Static ReadonlyDriverProvider ceiling unconfirmed; unknown until an in-repo source states it.
Static Optional ReadonlySupportsWhether this driver subclass supports dynamic, multi-tool sets projected into the realtime session (e.g. direct action invocation on realtime co-agents).
Static capability descriptor read before a session instance exists.
Default: absent / false. Subclasses that support dynamic toolsets declare public static readonly SupportsDynamicToolSet = true;.
ProtectedapiOnly sub-classes can access the API key
Readonly getter returning the model driver's connection duration in seconds.
Static driver default for a single connection's duration in seconds (for example, 600 on
GeminiRealtime and GeminiEnterpriseRealtime, 3600 on OpenAIRealtime and
OpenAILiveRealtime, 7200 on xAIRealtime). Defaults to DriverFacts.ProviderMaxSessionSeconds.
Buffer in seconds before CallDurationSeconds at which the framework initiates a
proactive reconnect. Defaults to DEFAULT_REALTIME_RECONNECT_BUFFER_SECONDS (10s).
Dynamic runtime override hook on the model driver (returns undefined unless overridden by a subclass).
Static fallback list of provider-native voices supported by this driver when metadata personas are not present or sparse.
NOTE: Database metadata (MJ: AI Personas + AI Model Personas) is authoritative where configured.
At runtime, GetRealtimeModelVoices unions metadata personas with this driver fallback list so that
newly uncatalogued provider voices remain selectable while catalogued personas carry curated names/descriptions.
The supported voice ids (id + human label), or [] when none are declared.
Whether this driver can mint an ephemeral, server-scoped client credential for a client-direct realtime session (the browser opens its own provider socket using a short-lived token the server minted).
Defaults to false. Providers that support browser-direct sessions override this to true
and implement CreateClientSession. The server-bridged topology (where the provider
socket lives on the server, via StartSession) is supported by every driver regardless
of this flag.
true if CreateClientSession is supported; false otherwise.
Whether this driver's sessions carry a video track in addition to audio.
false in every MJ driver.
Nothing reads it, and no driver overrides it: it is false for Gemini Live, which takes camera video
and renders avatars. What a session does with video is declared on its IRealtimeSession.Capabilities:
RealtimeSessionCapabilities.SupportedInboundTracks for video in and
RealtimeSessionCapabilities.SupportedOutboundTracks for video out, each track with its encoding. Whether a
model renders a live avatar is SupportsAvatarOutput. A driver declares video there, not here.
Whether this driver supports and trusts the provider's native session resumption on the
active endpoint when a handle is present. Subclasses override as needed (e.g., true on
server-side GeminiRealtime, GeminiEnterpriseRealtime, and xAIRealtime; false on
OpenAIRealtime).
ProtectedbuildBuilds the authenticated WebSocket URL. Auth rides as a ?token= query parameter (the
server-side API key for the server-bridged topology).
The full wss://…?token=… connect URL.
ProtectedClampReturns the smallest candidate that is a positive integer; see ClampSessionSecondsValue. The rule "the caller's bag wins only if smaller" lives there, once: pass the provider cap and the bag cap together.
Resolves whether native resumption is trusted for a client-direct session minted by this driver,
given the transport ('direct' vs 'relay'). Subclasses override when client-direct trust
differs from server-bridged trust (e.g., GeminiRealtime is true on server v1beta keys
but false on browser-direct v1alpha ephemeral tokens).
Optional_transport: RealtimeClientTransportProtectedconnectTransport seam for the realtime WebSocket. Production speaks the raw Inworld Realtime protocol
over the platform-global WebSocket (browsers / Node 22+) and resolves once the socket is
OPEN. Unit tests override this to return an in-memory fake — no network.
The authenticated URL plus inbound-message / error / close callbacks.
A promise resolving to the connected InworldRealtimeSocket once open.
Mints an ephemeral, server-scoped client credential plus a provider-native session config for a client-direct realtime session.
In the client-direct topology the browser owns the provider socket (e.g. WebRTC), but the
server still controls the prompt and tool set: it mints a short-lived token and hands back
a ClientRealtimeSessionConfig whose SessionConfig the matching client driver applies
when it opens its socket. This keeps prompt/tool authority server-side even though the media
plane is client-direct.
Not every provider supports this (some only expose a server-bridged socket), so this is a
concrete method that throws by default rather than an abstract one — that would force every
existing and future driver to implement it. Providers that support it override
SupportsClientDirect to true and override this method.
The session parameters (system prompt, tools, model, config bag).
A promise resolving to the minted ClientRealtimeSessionConfig.
OptionalExchangeOptional WebRTC SDP exchange seam for drivers that support server-brokered WebRTC topologies.
The local SDP offer generated by the browser.
Optional_sessionConfig: Record<string, unknown>The session configuration to initialize the session with.
The answer SDP, provider session ID, and any prebill accounting.
Internal bridge allowing BaseRealtimeSession to invoke the model driver's protected ReconnectSession.
ProtectedReconnectReconnect hook for server-hosted IRealtimeSession instances created by StartSession.
Called by BaseRealtimeSession's duration timer (CallDurationSeconds - ReconnectBufferSeconds)
or on unexpected transport close when resumable. Concrete provider drivers override this method.
Mints a fresh client credential (ephemeral token or signed URL) for an already-running client-direct session during a mid-call credential refresh or reconnect.
Defaults to delegating to CreateClientSession. Drivers where CreateClientSession
has side effects beyond credential minting (such as ElevenLabsRealtime, which provisions
a managed agent when no agentId is given) override this to mint a credential for the
existing resource in existingSessionConfig without creating duplicate vendor resources.
Optional_existingSessionConfig: JSONObjectResolves the effective single-connection duration in seconds from an open config bag
(connectionMaxSeconds), narrowing this.CallDurationSeconds via ResolveRealtimeTiming.
Optionalconfig: Record<string, unknown>Resolves the effective reconnect buffer in seconds from an open config bag
(reconnectBufferSeconds), falling back to this.ReconnectBufferSeconds.
Optionalconfig: Record<string, unknown>Resolves whether native session resumption is trusted for a session from an open config bag
(trustNativeResumption), falling back to defaultTrust (or this.TrustsNativeResumption).
Optionalconfig: Record<string, unknown>OptionaldefaultTrust: booleanOpens a server-bridged session: connects the realtime WebSocket authenticated with the API
key, sends the full session config as the FIRST frame (session.update — model, instructions,
audio/STT/TTS, tools, semantic-VAD turn-taking), and resolves only once the provider's
session-ready confirmation arrives (driver obligation #7 — "ready only after the config is
applied"). Mic frames sent before that would be dropped by the provider, and StartSession
not resolving until ready makes that unrepresentable for consumers.
Session configuration (model, system prompt, tools, initial context, config bag).
A promise resolving to the live IRealtimeSession handle, post-ready.
Whether this driver's sessions can render a live avatar for the model, on the endpoint the driver serves.
Defaults to false: a driver whose models render no avatar says nothing more, and may ignore an avatar request. A
driver that renders one for some models (Gemini Enterprise) overrides it. The client-session service reads it to
tell a call or a meeting that asked for an avatar "this voice model can't show an avatar"
(RealtimeAvatarUnavailableReason 'endpoint'), whichever driver serves it.
The provider's API name for the model.
true when sessions on this model can render a live avatar; false otherwise.
StaticMapMaps a Core RealtimeToolDefinition up to an Inworld realtime function tool schema —
the shape Inworld's "fluent tool calling" tools[] slot accepts. Shared by the initial
session.update and by mid-session IRealtimeSession.RegisterTools so both expose
byte-for-byte identical tool schemas.
The Core tool definition to map.
The Inworld realtime function-tool object.
StaticToolCanonical, order-insensitive fingerprint of a tool set (same scheme as the OpenAI / Gemini / AssemblyAI realtime drivers) — used by InworldRealtimeSession.RegisterTools to no-op identical re-registrations per the contract's idempotency rule.
The tool set to fingerprint.
A stable string fingerprint independent of tool ordering.
Real-time, full-duplex driver for the Inworld Realtime API, implementing the Core BaseRealtimeModel primitive. Registers as
InworldRealtimeand is resolved forMJ: AI ModelstypedRealtime(API-key env alias:AI_VENDOR_API_KEY__InworldRealtime).What the provider is: a single-WebSocket speech-to-speech stack. Sessions initialize with a
session.updateframe carrying model selection, instructions, audio settings, STT/TTS config, and tools; components are swappable mid-session without reconnect. Input is streamed audio (server-side STT with voice profiling); output is synthesized speech (Realtime TTS-2) supporting inline steering tags like[laugh]. Turn-taking is semantic VAD with adjustable eagerness, handling barge-in.Model resolution: Inworld brokers hundreds of LLMs; the reasoning model is selected via
modelId(e.g.anthropic/claude-sonnet-4-6) carried from RealtimeSessionParams.Model (falling back to INWORLD_DEFAULT_MODEL_ID). STT engine and TTS voice are configured independently viaConfig.Topology: server-bridged only (StartSession) — the driver opens the WebSocket, sends the session config, and resolves only once the provider confirms the config is applied (driver obligation #7). Inworld does not expose a documented ephemeral-token mint for browser-direct sessions, so SupportsClientDirect stays
false(inherited).Tool calling: "fluent tool calling" — functions declared at startup (or added mid-session via IRealtimeSession.RegisterTools) and executed mid-conversation; results fed back via IRealtimeSession.SendToolResult complete the loop.