Class ElevenLabsRealtime

The built-in default voices available in ElevenLabs — used to populate the voice picker.

ElevenLabs supports the client-direct topology natively: the signed websocket URL is a short-lived, agent-scoped credential the browser can open directly.

Mints the client-direct config: ensures the managed agent, fetches a signed URL (the ephemeral credential — connect window ≈ ELEVENLABS_SIGNED_URL_TTL_MS), and packs the private pact SessionConfig the same-keyed 'elevenlabs' client driver consumes: { agentId, overrides, config }, where overrides is the wire-shaped conversation_config_override carrying the server-authored system prompt and config passes params.Config through opaquely.

Opens a server-bridged conversation: ensures the agent, mints a signed URL, opens the websocket (via the connectConversation seam), sends the conversation_initiation_client_data frame carrying the per-session system prompt, and resolves only once the server's conversation_initiation_metadata confirms the session config is applied (driver obligation #7 — "ready only after the config is applied").

Builds the RAW-WIRE conversation_config_override for one session: the server-authored system prompt, plus the per-session TTS voice and opening utterance when the config bag carries them.

Mind the casing split. This object is forwarded VERBATIM onto the conversation websocket (by this driver server-bridged, by the 'elevenlabs' client driver client-direct), so it is snake_case — tts.voice_id, agent.first_message. The matching ENABLEMENT in buildRequiredOverrideEnablement goes out through the SDK's serializer instead and is therefore camelCase — tts.voiceId, agent.firstMessage. Only agent.prompt.prompt reads the same in both, so every override added after it has to be spelled twice, differently.

Each optional key is omitted ENTIRELY when unconfigured, so a session that configures neither is byte-for-byte the frame it was before either existed. That is what preserves the platform's wait-for-the-user default: an ENABLED-but-empty first_message means exactly "no opening utterance", so omitting the key and sending it blank are the same behavior — and omitting keeps the frame honest.

Extracts the CLIENT tools from a fetched agent as Core tool definitions, for fingerprint comparison against the requested set. Non-client tools (webhook/system/mcp) are ignored — they are deployment-side additions the managed flow must not fight over.

Maps a Core tool definition up to an ElevenLabs inline CLIENT tool config.

The desired-model-settings half of the ensure cache key (#3859): the config bag's llm and temperature in one stable string. Unconfigured values serialize as absent, so a bag that says nothing produces the same fingerprint forever.

Whether the agent's OWN configuration matches the model settings the config bag asks for — the drift the ensure flow repairs for settings that are AGENT STATE rather than per-session overrides (#3859).

temperature lives here because the platform offers no per-session override flag for it (PromptAgentApiModelOverrideConfig has prompt/llm/nativeMcpServerIds only), so the only honest delivery is: set it on the managed agent's body and re-PATCH when it changes. llm is checked here TOO, even though it is also per-session overridable, so an agent whose configured default model changed is repaired even for callers that send no override.

An UNCONFIGURED desire matches anything: half the point of the managed agent is that the deployment may tune it, and a config bag that says nothing must not stampede a PATCH war against a hand-tuned value.

Whether the agent's platform settings enable EVERY per-session override this driver sends — the drift condition the ensure flow repairs. Checking the whole required set (rather than the prompt override alone) is what lets an agent provisioned by an older MJ version pick up a newly-added override: it still matches on tools, so the enablement gap is the only signal that it needs a PATCH.

This is the READ half of the contract buildRequiredOverrideEnablement WRITES. The two are deliberately spelled out separately rather than derived from one another: the write side is the SDK's …Input shape and this reads the …Output shape, so a generic walker over either would have to erase both types and lose exactly the compile-time checking that catches a mistake here. They are kept honest instead by the round-trip test "considers an agent this driver just provisioned already satisfied" — add an override to one half only and that test fails (as a permanent re-PATCH loop).

Chooses ONE agent when the search returns several with the requested name — which happens whenever a duplicate was forked before this guard existed. The OLDEST wins (ties broken by agent id), so every process and every session converges on the SAME agent instead of picking whichever the API happened to list first and PATCHing them alternately.

Whether the agent's platform settings enable the per-session system-prompt override.

Reads the session's opening utterance out of the config bag — the firstMessage key, which maps to ElevenLabs' agent.first_message override.

This is spoken VERBATIM, not interpreted. It is the literal first thing the agent says, not an instruction to the model about how to open — which is also why it is immune to the ordering of ElevenLabsRealtimeSession.SendInitiation's deferred initial-context injection: there is no generation to race.

A missing, blank, or non-string value yields undefined so the override is omitted rather than sent empty. An empty first_message IS the platform's wait-for-the-user default, so this is a no-op either way — omitting simply keeps the frame free of meaningless keys.

Reads the DRIVER-NEUTRAL voice key out of the session's config bag — the same key the AssemblyAI and Inworld realtime drivers read, so a persona's configured voice is authored identically whichever provider ends up speaking it. Reached in practice via the effective config's per-provider bag, realtime.voice.providers.elevenlabs.voice.

A missing, blank, or non-string value yields undefined so the override is omitted rather than sent empty (ElevenLabs would reject an empty voice_id, killing the whole session over a misconfigured persona).

Canonical, order-insensitive fingerprint of a tool set (same scheme as the Gemini realtime driver). A schema round-trip through ElevenLabs that fails exact equality merely triggers a harmless idempotent PATCH.