The xAI API key used to authenticate the Grok Voice realtime socket.
Static ReadonlyDriverxAI's Grok Voice sessions run up to 120 minutes.
Static ReadonlySupportsWhether this driver subclass supports dynamic, multi-tool sets projected into the realtime session (e.g. direct action invocation on realtime co-agents).
Static capability descriptor read before a session instance exists.
Default: absent / false. Subclasses that support dynamic toolsets declare public static readonly SupportsDynamicToolSet = true;.
ProtectedapiOnly sub-classes can access the API key
Readonly getter returning the model driver's connection duration in seconds.
Static driver default for a single connection's duration in seconds (for example, 600 on
GeminiRealtime and GeminiEnterpriseRealtime, 3600 on OpenAIRealtime and
OpenAILiveRealtime, 7200 on xAIRealtime). Defaults to DriverFacts.ProviderMaxSessionSeconds.
Read-only accessor for the underlying OpenAI SDK client.
ProtectedProfileThe Grok Voice knobs + GA feature gates the shared protocol implementation runs with.
Buffer in seconds before CallDurationSeconds at which the framework initiates a
proactive reconnect. Defaults to DEFAULT_REALTIME_RECONNECT_BUFFER_SECONDS (10s).
Dynamic runtime override hook on the model driver (returns undefined unless overridden by a subclass).
Grok Voice does not (yet) document a selectable voice set on the realtime endpoint, so the dev voice picker gets none — overriding away the OpenAI voice list this class would otherwise inherit.
OpenAI supports the client-direct topology: the server mints a short-lived ephemeral client secret that the browser uses to open its OWN connection to OpenAI's Realtime API, while the server still controls the system prompt + tools via the returned SessionConfig.
Whether this driver's sessions carry a video track in addition to audio.
false in every MJ driver.
Nothing reads it, and no driver overrides it: it is false for Gemini Live, which takes camera video
and renders avatars. What a session does with video is declared on its IRealtimeSession.Capabilities:
RealtimeSessionCapabilities.SupportedInboundTracks for video in and
RealtimeSessionCapabilities.SupportedOutboundTracks for video out, each track with its encoding. Whether a
model renders a live avatar is SupportsAvatarOutput. A driver declares video there, not here.
False by default: live probing against wss://api.x.ai/v1/realtime confirmed that resuming via
?conversation_id=<id> drops all client-created conversation.item.create message items
(SendContextNote, SendText, and any history seeded after a network drop), whereas ConversationMemory
text seeding is lossless (~0.9 s). Explicit Config.trustNativeResumption = true still opts in.
ProtectedClampReturns the smallest candidate that is a positive integer; see ClampSessionSecondsValue. The rule "the caller's bag wins only if smaller" lives there, once: pass the provider cap and the bag cap together.
Resolves whether native resumption is trusted for a client-direct session minted by this driver,
given the transport ('direct' vs 'relay'). Subclasses override when client-direct trust
differs from server-bridged trust (e.g., GeminiRealtime is true on server v1beta keys
but false on browser-direct v1alpha ephemeral tokens).
Optional_transport: RealtimeClientTransportMints an ephemeral, server-scoped realtime session credential for a browser to open its own provider connection (client-direct topology). The server builds the session config (system prompt + tools + model) so it retains control of behavior even though the browser owns the socket.
The GA features (reasoning effort, parallel tool calls, MCP tools) and the output voice are extracted from the Config bag and applied here exactly as on the server-bridged path, so the two topologies stay behaviorally identical — the browser applies the minted SessionConfig verbatim with no client-side changes needed.
Session configuration (model, system prompt, tools).
The minted ClientRealtimeSessionConfig the browser authenticates + applies.
ProtectedcreateCreates the realtime connection for a model. Overridable seam for testing.
Production returns a real OpenAIRealtimeWebSocket. Unit tests override this to return a
fake IOpenAIRealtimeConnection that emits OpenAI-shaped events and captures sends.
The provider realtime model id (e.g. gpt-realtime-2.1).
OptionalresumptionHandle: stringOptional provider session/conversation handle to resume (?conversation_id=).
A connection implementing IOpenAIRealtimeConnection.
ProtectedcreateReturns the xAI-branded session subclass so instanceof checks and logs stay provider-true.
OptionalExchangeOptional WebRTC SDP exchange seam for drivers that support server-brokered WebRTC topologies.
The local SDP offer generated by the browser.
Optional_sessionConfig: Record<string, unknown>The session configuration to initialize the session with.
The answer SDP, provider session ID, and any prebill accounting.
Internal bridge allowing BaseRealtimeSession to invoke the model driver's protected ReconnectSession.
ProtectedmintMints the ephemeral client secret via the provider's Realtime client-secrets API (resolved from the SDK client's base URL, so OpenAI-compatible subclasses target their own endpoint). Overridable seam for testing — unit tests return a fake response so no network call is made.
The client-secret create request (carries the realtime session config).
The client-secret create response (token value + expiry + echoed session).
ProtectedReconnectReconnect hook for server-hosted IRealtimeSession instances created by StartSession.
Called by BaseRealtimeSession's duration timer (CallDurationSeconds - ReconnectBufferSeconds)
or on unexpected transport close when resumable. Concrete provider drivers override this method.
Mints a fresh client credential (ephemeral token or signed URL) for an already-running client-direct session during a mid-call credential refresh or reconnect.
Defaults to delegating to CreateClientSession. Drivers where CreateClientSession
has side effects beyond credential minting (such as ElevenLabsRealtime, which provisions
a managed agent when no agentId is given) override this to mint a credential for the
existing resource in existingSessionConfig without creating duplicate vendor resources.
Optional_existingSessionConfig: JSONObjectResolves the effective single-connection duration in seconds from an open config bag
(connectionMaxSeconds), narrowing this.CallDurationSeconds via ResolveRealtimeTiming.
Optionalconfig: Record<string, unknown>Resolves the effective reconnect buffer in seconds from an open config bag
(reconnectBufferSeconds), falling back to this.ReconnectBufferSeconds.
Optionalconfig: Record<string, unknown>Resolves whether native session resumption is trusted for a session from an open config bag
(trustNativeResumption), falling back to defaultTrust (or this.TrustsNativeResumption).
Optionalconfig: Record<string, unknown>OptionaldefaultTrust: booleanOpens a duplex realtime session, applies the session config, and returns the live handle.
Session configuration (model, system prompt, tools, initial context, config bag).
A promise resolving to the IRealtimeSession handle.
Whether this driver's sessions can render a live avatar for the model, on the endpoint the driver serves.
Defaults to false: a driver whose models render no avatar says nothing more, and may ignore an avatar request. A
driver that renders one for some models (Gemini Enterprise) overrides it. The client-session service reads it to
tell a call or a meeting that asked for an avatar "this voice model can't show an avatar"
(RealtimeAvatarUnavailableReason 'endpoint'), whichever driver serves it.
The provider's API name for the model.
true when sessions on this model can render a live avatar; false otherwise.
xAI Grok Voice implementation of the BaseRealtimeModel primitive.
xAI's Grok Voice Agent API is OpenAI-Realtime-API compatible (WebSocket endpoint
wss://api.x.ai/v1/realtime, Base64-encoded PCM16 @ 24 kHz audio, the same client/server event protocol and tool-calling shape). This driver therefore subclasses OpenAIRealtime with the SDK client pointed at xAI's base URL — inheriting the shared protocol implementation (session config, tool loop, barge-in gating, error classification, client-direct minting) and every future GA feature, gated per-provider by XAI_REALTIME_PROFILE, instead of maintaining a near-identical clone of the driver.Tool results complete the tool-call loop: the returned session implements the Core
IRealtimeSession.SendToolResultcontract method, which the agent layer calls after executing a tool to feed its result back to the model.Registered as
GrokRealtimevia@RegisterClass(BaseRealtimeModel, 'GrokRealtime'); the associatedMJ: AI Modelis typed with theRealtimeAIModelType.