The provider API key.
OptionalbaseURL: stringOptional override for OpenAI-compatible providers (subclasses pass their own
endpoint; the SDK's buildRealtimeURL() derives the wss:// realtime endpoint from it).
ProtectedapiOnly sub-classes can access the API key
Read-only accessor for the underlying OpenAI SDK client.
ProtectedProfileThe provider profile driving per-provider knobs and GA feature gates. Subclasses override this single seam instead of re-implementing the protocol.
The voices the OpenAI Realtime API can speak with — used to populate the dev voice picker so two agents in one room can be given distinct voices. Kept in sync with OpenAI's realtime voice set.
OpenAI supports the client-direct topology: the server mints a short-lived ephemeral client secret that the browser uses to open its OWN connection to OpenAI's Realtime API, while the server still controls the system prompt + tools via the returned SessionConfig.
Whether this driver's sessions carry a video track in addition to audio — i.e. the model accepts video input (it can "see" the user's camera) and/or emits video output (a talking-head avatar / generated video), in sync with audio.
Defaults to false (audio-only — today's realtime models). Video-capable drivers (a native
multimodal realtime model, or an avatar provider) override this to true. The session's media
plane is media-tagged (IRealtimeSession.SendInput takes a RealtimeMediaKind;
IRealtimeSession.OnVideoOutput delivers video-out), so a video session reuses the entire
realtime contract — only the media frames gain a video kind. Resolution prefers a video-capable
model when an agent requests video, and degrades to audio-only otherwise.
true if sessions can carry video; false (audio-only) otherwise.
Mints an ephemeral, server-scoped realtime session credential for a browser to open its own provider connection (client-direct topology). The server builds the session config (system prompt + tools + model) so it retains control of behavior even though the browser owns the socket.
The GA features (reasoning effort, parallel tool calls, MCP tools) and the output voice are extracted from the Config bag and applied here exactly as on the server-bridged path, so the two topologies stay behaviorally identical — the browser applies the minted SessionConfig verbatim with no client-side changes needed.
Session configuration (model, system prompt, tools).
The minted ClientRealtimeSessionConfig the browser authenticates + applies.
ProtectedcreateCreates the realtime connection for a model. Overridable seam for testing.
Production returns a real OpenAIRealtimeWebSocket. Unit tests override this to return a
fake IOpenAIRealtimeConnection that emits OpenAI-shaped events and captures sends.
The provider realtime model id (e.g. gpt-realtime-2.1).
A connection implementing IOpenAIRealtimeConnection.
ProtectedcreateCreates the session wrapper for a freshly-opened connection. Overridable seam so subclasses can return their own session subclass while StartSession stays shared.
The open provider connection.
The session bound to this driver's Profile.
ProtectedmintMints the ephemeral client secret via the provider's Realtime client-secrets API (resolved from the SDK client's base URL, so OpenAI-compatible subclasses target their own endpoint). Overridable seam for testing — unit tests return a fake response so no network call is made.
The client-secret create request (carries the realtime session config).
The client-secret create response (token value + expiry + echoed session).
Opens a duplex realtime session, applies the session config, and returns the live handle.
Session configuration (model, system prompt, tools, initial context, config bag).
A promise resolving to the IRealtimeSession handle.
OpenAI implementation of the BaseRealtimeModel primitive, backed by OpenAI's Realtime API over a WebSocket (
OpenAIRealtimeWebSocketfrom theopenaiSDK, v6.18.0).The driver opens a duplex session, configures it (system prompt + tools + optional initial context), and translates the provider's server-event stream into the modality-agnostic IRealtimeSession contract.
This class is also the shared implementation for OpenAI-Realtime-compatible providers. Compatible providers (e.g. xAI Grok Voice) subclass it, pass their base URL to the constructor, and override OpenAIRealtime.Profile — inheriting the whole protocol implementation and every future GA feature (gated per-provider by the profile) instead of maintaining a clone.
GA features (gpt-realtime-2 / 2.1 era) are driven from the open RealtimeSessionParams.Config bag with MJ-idiomatic keys, translated to provider-native session fields only when the profile confirms support:
reasoningEffort: 'minimal'|'low'|'medium'|'high'|'xhigh'→reasoning.effortparallelToolCalls: boolean→parallel_tool_callsmcpTools: [{ type:'mcp', server_label, server_url|connector_id, ... }]→ appended tosession.toolsTool results complete the tool-call loop: the returned session implements the Core
IRealtimeSession.SendToolResultcontract method, which the agent layer calls after executing a tool to feed its result back to the model. See OpenAIRealtimeSession.SendToolResult.