Class OpenAIRealtimeSession

Applies the initial session config: system prompt + tools via session.update, optional initial context as a user message. Called once by OpenAIRealtime.StartSession.

Deferral is profile-driven. On OpenAI the realtime WebSocket is NOT open when StartSession returns — sending session.update synchronously races the handshake and the instructions (the system prompt + tools) are silently dropped, so the model runs with NO prompt (no identity, no companion framing). We therefore wait for the server's session.created frame — the first event once the socket is open and the session exists, and the canonical moment to configure a realtime session — exactly the point the browser/client-direct path applies its config. Idempotent (a re-emitted session.created can't double-apply); the listener removes itself once it fires. Providers whose socket accepts config immediately (xAI) set the profile flag false and send synchronously.

Closes the session and releases the underlying provider connection.

Optional capability — registers a handler invoked when the underlying provider connection closes WITHOUT the consumer having called IRealtimeSession.Close (provider-side hangup, network drop). Not fired for a consumer-initiated Close() — the caller already knows about that one.

Optional because not every provider surface exposes a close signal cheaply; callers must feature-detect (if (session.OnClose) { ... }). An unexpected close is typically ALSO surfaced as a Fatal IRealtimeSession.OnError, which is the signal consumers should drive finalization from.

Registers a handler for session errors.

Fatality semantics mirror the client-side BaseRealtimeClient contract:

• Fatal: true — the session is unusable (transport/socket failure, credential/token expiry, unexpected connection loss). The consumer should finalize the session (e.g. RealtimeSessionRunner calls Stop()) instead of idling forever on a dead socket.
• Fatal: false — a provider-reported, recoverable error frame; the session stays open and the consumer should log and continue.

Registers a handler for provider-detected interruptions (barge-in).

True barge-in only: the handler fires ONLY when user speech interrupts ACTIVE model output — NOT on every user utterance. A user simply taking their normal turn while the model is idle is not an interruption, and drivers must not report it as one (e.g. a raw "speech started" frame must be gated on whether a model response is actually in flight).

Turn detection / VAD is owned by the provider. The agent layer uses this hook to cancel the model's current turn and to fire the cancellationToken of any in-flight delegated agent run — a stale delegated result must never be narrated into a conversation that has moved on.

Registers a handler for model audio output frames (the audio media plane).

Registers a handler for model tool-call requests.

Consumers execute the requested tool (under the session's context user) and feed the result back to the model.

Registers a handler for transcript events (the text stream).

Consumers typically forward these to the control plane and persist them as ConversationDetail records.

Registers a handler for usage/telemetry updates.

Usage is checkpointed incrementally by the agent layer (debounced onto the prompt run) so partial usage is never lost if the session is force-closed after a crash.

Registers the set of tools the model may call, translating them into the provider's native function-calling format.

The Core-level RealtimeToolDefinition is intentionally minimal: BaseRealtimeModel lives in the lowest AI layer and cannot depend on the richer tool metadata defined in higher packages (the agent layer) — doing so would create an illegal upward/circular dependency. The agent layer is responsible for mapping its richer tool metadata down to this Core type before calling RegisterTools, and the concrete driver maps this Core type up to the provider's native function-calling schema.

Note that some providers (e.g. Eleven Labs) bind to a pre-declared tool set on a server-side agent configuration; for those, the driver maps these definitions onto the pre-declared tool names rather than registering arbitrary schemas at session start.

Idempotency rule: a post-start registration of a set IDENTICAL to the set supplied at connect time (via RealtimeSessionParams.Tools) MUST be a no-op. Providers that bind their tool set at connect time and cannot re-declare schemas on an open session MUST no-op (and may log) rather than degrade the conversation — e.g. by injecting schema text into the conversation as content. A genuinely DIFFERENT post-start set on such a provider is unsupported and should be surfaced as a warning, not silently mangled.

Sends a client media frame to the model.

Fire-and-forget: frames are streamed straight to the provider with no JSON intermediation. The optional kind tags the media plane — 'audio' (default, back-compatible: existing callers and audio-only drivers need not pass or read it) or 'video' for a camera frame to a video-capable model (one that BaseRealtimeModel.SupportsVideo). Audio-only drivers ignore 'video' frames.

Resolves once the initial session config has been APPLIED (sent on the socket) — immediately for providers that configure synchronously, or on the server's session.created frame for deferring providers. Rejects if the transport dies (fatal error or unexpected close) or the consumer closes the session before the config went out.

The base OpenAIRealtime.StartSession deliberately does NOT await this (OpenAI semantics: the session handle is returned while the handshake completes). Drivers whose contract promises "ready only after config is applied" (HuggingFace) await it in their StartSession override.