The ElevenLabs API key. It is only ever sent as xi-api-key to mint client
secrets; the realtime socket itself is opened with a minted secret.
Static ReadonlyDriverThe provider's session ceiling is not documented in-repo (max_duration_seconds is a per-session setting, not a ceiling), so it stays unknown.
Static ReadonlySupportsWhether this driver subclass supports dynamic, multi-tool sets projected into the realtime session (e.g. direct action invocation on realtime co-agents).
Static capability descriptor read before a session instance exists.
Default: absent / false. Subclasses that support dynamic toolsets declare public static readonly SupportsDynamicToolSet = true;.
ProtectedapiOnly sub-classes can access the API key
Read-only accessor for the underlying OpenAI SDK client.
ProtectedProfileThe Eleven Realtime knobs + feature gates the shared protocol implementation runs with.
ElevenLabs' premade voices, by voice id — the voice picker's list. Any voice id in the
account's library also works through the Config bag's voice key.
OpenAI supports the client-direct topology: the server mints a short-lived ephemeral client secret that the browser uses to open its OWN connection to OpenAI's Realtime API, while the server still controls the system prompt + tools via the returned SessionConfig.
Whether this driver's sessions carry a video track in addition to audio.
false in every MJ driver.
Nothing reads it, and no driver overrides it: it is false for Gemini Live, which takes camera video
and renders avatars. What a session does with video is declared on its IRealtimeSession.Capabilities:
RealtimeSessionCapabilities.SupportedInboundTracks for video in and
RealtimeSessionCapabilities.SupportedOutboundTracks for video out, each track with its encoding. Whether a
model renders a live avatar is SupportsAvatarOutput. A driver declares video there, not here.
ProtectedClampReturns the smallest candidate that is a positive integer; see ClampSessionSecondsValue. The rule "the caller's bag wins only if smaller" lives there, once: pass the provider cap and the bag cap together.
ProtectedcreateOpens the realtime socket authenticated by a minted client secret. The SDK puts the secret in
the openai-insecure-api-key. subprotocol — the "use the OpenAI SDK as is" path ElevenLabs
documents. Overridable seam: unit tests return a fake connection.
Client-direct mint: the shared OpenAI flow, with max_duration_seconds clamped first (see withClampedMaxDuration).
The clamp is computed once: when it yields a wire value, that same value becomes MaxSessionSeconds for the
shared flow, so the ClientConfig reports exactly what went on the wire (bag clamp included).
ProtectedcreateCreates the realtime connection for a model. Overridable seam for testing.
Production returns a real OpenAIRealtimeWebSocket. Unit tests override this to return a
fake IOpenAIRealtimeConnection that emits OpenAI-shaped events and captures sends.
The provider realtime model id (e.g. gpt-realtime-2.1).
A connection implementing IOpenAIRealtimeConnection.
ProtectedcreateReturns the Eleven-branded session subclass so instanceof checks and logs stay provider-true.
OptionalExchangeOptional WebRTC SDP exchange seam for drivers that support server-brokered WebRTC topologies.
The local SDP offer generated by the browser.
Optional_sessionConfig: Record<string, unknown>The session configuration to initialize the session with.
The answer SDP, provider session ID, and any prebill accounting.
ProtectedmintMints an Eleven Realtime client secret. The openai client already resolves the right URL and
types the request and response; this only swaps the bearer header for xi-api-key, and reads
expires_at in seconds whichever unit ElevenLabs returns it in.
Opens a server-bridged session: mints a client secret for the model, opens the socket with
it, then applies the full session config as the shared driver does. The minted secret carries
only the model — the prompt and tools arrive in the session.update that follows, as on every
driver in the family.
Whether this driver's sessions can render a live avatar for the model, on the endpoint the driver serves.
Defaults to false: a driver whose models render no avatar says nothing more, and may ignore an avatar request. A
driver that renders one for some models (Gemini Enterprise) overrides it. The client-session service reads it to
tell a call or a meeting that asked for an avatar "this voice model can't show an avatar"
(RealtimeAvatarUnavailableReason 'endpoint'), whichever driver serves it.
The provider's API name for the model.
true when sessions on this model can render a live avatar; false otherwise.
Eleven Realtime implementation of the BaseRealtimeModel primitive —
eleven_realtime_v1_miniandeleven_realtime_v1_max.Eleven Realtime is a cascaded stack (ASR → LLM → TTS with ElevenLabs' turn-taking) sold as a single speech-to-speech model, and its API follows the OpenAI Realtime spec over a WebSocket. So, like xAI's Grok Voice, this driver subclasses OpenAIRealtime and inherits the whole protocol implementation — session config, tool loop, barge-in, error classification, client-direct minting — with ELEVEN_REALTIME_PROFILE gating the features.
It is a different product from ElevenLabsRealtime, which drives the ElevenLabs Agents Platform's own protocol against a server-side agent. Here there is no agent to provision: the session carries the prompt and tools, exactly as it does for OpenAI.
Two things differ from OpenAI:
xi-api-key, never as a bearer token, and documents the realtime socket as authenticated by an ephemeral client secret passed where an OpenAI client would put its API key. So mintClientSecret sendsxi-api-key, and the server-bridged StartSession mints a secret first and opens the socket with it — the same documented path the browser uses, rather than guessing whether the socket also accepts a raw key.mediumwhere MJ saysauto(see MapElevenRealtimeEagerness).max_duration_seconds(an ElevenLabs extension: the server closes the session with code 1000 once it is exceeded) ismin(provider cap, MaxSessionSeconds, Config-bag value)through the one clamp in BaseRealtimeModel. Only a positive-integer number in the bag counts; any other bag value (e.g. the string"600") is ignored AND never sent as-is. With no provider cap known, no hostMaxSessionSecondsand no usable bag value the field is removed, so ElevenLabs applies its own ceiling. The bag is adjusted in ElevenRealtime.withClampedMaxDuration before the shared OpenAI payload builder runs, on both the mint (ElevenRealtime.CreateClientSession) and the server-bridged (ElevenRealtime.StartSession) paths, so no payload code is duplicated.Registered as
ElevenRealtime; the matchingMJ: AI Model Vendorsrows name it as theirDriverClass.