Member Junction
    Preparing search index...

    xAI Grok Voice implementation of the BaseRealtimeModel primitive.

    xAI's Grok Voice Agent API is OpenAI-Realtime-API compatible (WebSocket endpoint wss://api.x.ai/v1/realtime, Base64-encoded PCM16 @ 24 kHz audio, the same client/server event protocol and tool-calling shape). This driver therefore subclasses OpenAIRealtime with the SDK client pointed at xAI's base URL — inheriting the shared protocol implementation (session config, tool loop, barge-in gating, error classification, client-direct minting) and every future GA feature, gated per-provider by XAI_REALTIME_PROFILE, instead of maintaining a near-identical clone of the driver.

    Tool results complete the tool-call loop: the returned session implements the Core IRealtimeSession.SendToolResult contract method, which the agent layer calls after executing a tool to feed its result back to the model.

    Registered as GrokRealtime via @RegisterClass(BaseRealtimeModel, 'GrokRealtime'); the associated MJ: AI Model is typed with the Realtime AIModelType.

    Hierarchy (View Summary)

    Index

    Constructors

    Properties

    DriverFacts: Readonly<RealtimeDriverFacts> = ...

    xAI's Grok Voice sessions run up to 120 minutes.

    SupportsDynamicToolSet: boolean = XAI_REALTIME_PROFILE.supportsDynamicToolSet

    Whether this driver subclass supports dynamic, multi-tool sets projected into the realtime session (e.g. direct action invocation on realtime co-agents).

    Static capability descriptor read before a session instance exists. Default: absent / false. Subclasses that support dynamic toolsets declare public static readonly SupportsDynamicToolSet = true;.

    Accessors

    • get apiKey(): string

      Only sub-classes can access the API key

      Returns string

    • get CallDurationSeconds(): number

      Readonly getter returning the model driver's connection duration in seconds.

      Returns number

    • get DefaultCallDurationSeconds(): number

      Static driver default for a single connection's duration in seconds (for example, 600 on GeminiRealtime and GeminiEnterpriseRealtime, 3600 on OpenAIRealtime and OpenAILiveRealtime, 7200 on xAIRealtime). Defaults to DriverFacts.ProviderMaxSessionSeconds.

      Returns number

    • get OpenAI(): OpenAI

      Read-only accessor for the underlying OpenAI SDK client.

      Returns OpenAI

    • get ReconnectBufferSeconds(): number

      Buffer in seconds before CallDurationSeconds at which the framework initiates a proactive reconnect. Defaults to DEFAULT_REALTIME_RECONNECT_BUFFER_SECONDS (10s).

      Returns number

    • get RuntimeCallDurationSeconds(): number

      Dynamic runtime override hook on the model driver (returns undefined unless overridden by a subclass).

      Returns number

    • get SupportsClientDirect(): boolean

      OpenAI supports the client-direct topology: the server mints a short-lived ephemeral client secret that the browser uses to open its OWN connection to OpenAI's Realtime API, while the server still controls the system prompt + tools via the returned SessionConfig.

      Returns boolean

    • get TrustsNativeResumption(): boolean

      False by default: live probing against wss://api.x.ai/v1/realtime confirmed that resuming via ?conversation_id=<id> drops all client-created conversation.item.create message items (SendContextNote, SendText, and any history seeded after a network drop), whereas ConversationMemory text seeding is lossless (~0.9 s). Explicit Config.trustNativeResumption = true still opts in.

      Returns boolean

    Methods

    • Returns the smallest candidate that is a positive integer; see ClampSessionSecondsValue. The rule "the caller's bag wins only if smaller" lives there, once: pass the provider cap and the bag cap together.

      Parameters

      • ...candidates: readonly number[]

      Returns number

    • Resolves whether native resumption is trusted for a client-direct session minted by this driver, given the transport ('direct' vs 'relay'). Subclasses override when client-direct trust differs from server-bridged trust (e.g., GeminiRealtime is true on server v1beta keys but false on browser-direct v1alpha ephemeral tokens).

      Parameters

      Returns boolean

    • Mints an ephemeral, server-scoped realtime session credential for a browser to open its own provider connection (client-direct topology). The server builds the session config (system prompt + tools + model) so it retains control of behavior even though the browser owns the socket.

      The GA features (reasoning effort, parallel tool calls, MCP tools) and the output voice are extracted from the Config bag and applied here exactly as on the server-bridged path, so the two topologies stay behaviorally identical — the browser applies the minted SessionConfig verbatim with no client-side changes needed.

      Parameters

      Returns Promise<ClientRealtimeSessionConfig>

      The minted ClientRealtimeSessionConfig the browser authenticates + applies.

    • Creates the realtime connection for a model. Overridable seam for testing.

      Production returns a real OpenAIRealtimeWebSocket. Unit tests override this to return a fake IOpenAIRealtimeConnection that emits OpenAI-shaped events and captures sends.

      Parameters

      • model: string

        The provider realtime model id (e.g. gpt-realtime-2.1).

      • OptionalresumptionHandle: string

        Optional provider session/conversation handle to resume (?conversation_id=).

      Returns IOpenAIRealtimeConnection

      A connection implementing IOpenAIRealtimeConnection.

    • Optional WebRTC SDP exchange seam for drivers that support server-brokered WebRTC topologies.

      Parameters

      • _offerSdp: string

        The local SDP offer generated by the browser.

      • Optional_sessionConfig: Record<string, unknown>

        The session configuration to initialize the session with.

      Returns Promise<{ answerSdp: string; prebillSeconds: number; sessionId: string }>

      The answer SDP, provider session ID, and any prebill accounting.

    • Mints the ephemeral client secret via the provider's Realtime client-secrets API (resolved from the SDK client's base URL, so OpenAI-compatible subclasses target their own endpoint). Overridable seam for testing — unit tests return a fake response so no network call is made.

      Parameters

      • body: ClientSecretCreateParams

        The client-secret create request (carries the realtime session config).

      Returns Promise<ClientSecretCreateResponse>

      The client-secret create response (token value + expiry + echoed session).

    • Reconnect hook for server-hosted IRealtimeSession instances created by StartSession. Called by BaseRealtimeSession's duration timer (CallDurationSeconds - ReconnectBufferSeconds) or on unexpected transport close when resumable. Concrete provider drivers override this method.

      Parameters

      Returns Promise<void>

    • Resolves the effective single-connection duration in seconds from an open config bag (connectionMaxSeconds), narrowing this.CallDurationSeconds via ResolveRealtimeTiming.

      Parameters

      • Optionalconfig: Record<string, unknown>

      Returns number

    • Resolves the effective reconnect buffer in seconds from an open config bag (reconnectBufferSeconds), falling back to this.ReconnectBufferSeconds.

      Parameters

      • Optionalconfig: Record<string, unknown>

      Returns number

    • Resolves whether native session resumption is trusted for a session from an open config bag (trustNativeResumption), falling back to defaultTrust (or this.TrustsNativeResumption).

      Parameters

      • Optionalconfig: Record<string, unknown>
      • OptionaldefaultTrust: boolean

      Returns boolean

    • Whether this driver's sessions can render a live avatar for the model, on the endpoint the driver serves.

      Defaults to false: a driver whose models render no avatar says nothing more, and may ignore an avatar request. A driver that renders one for some models (Gemini Enterprise) overrides it. The client-session service reads it to tell a call or a meeting that asked for an avatar "this voice model can't show an avatar" (RealtimeAvatarUnavailableReason 'endpoint'), whichever driver serves it.

      Parameters

      • _model: string

        The provider's API name for the model.

      Returns boolean

      true when sessions on this model can render a live avatar; false otherwise.