Binds the underlying socket. Called by the driver once the websocket is open.
Entry point for an inbound websocket frame. Multiplexes on type to focused
per-concern handlers so each translation unit stays small and testable.
Surfaces an UNEXPECTED socket close as a fatal error (expected closes — the consumer called Close — are silent). The provider hard-closes at token expiry and when it ends the session itself, so this is also how credential / session death reaches the consumer.
Optionalcode: numberOptionalreason: stringSurfaces a websocket-level failure as a FATAL session error — the transport is gone, so the consumer should finalize cleanly instead of idling (driver obligation #6).
Registers a handler for session errors.
Fatality semantics mirror the client-side BaseRealtimeClient contract:
Fatal: true — the session is unusable (transport/socket failure, credential/token
expiry, unexpected connection loss). The consumer should finalize the session (e.g.
RealtimeSessionRunner calls Stop()) instead of idling forever on a dead socket.Fatal: false — a provider-reported, recoverable error frame; the session stays open
and the consumer should log and continue.Invoked with each RealtimeSessionError.
Registers a handler for provider-detected interruptions (barge-in).
True barge-in only: the handler fires ONLY when user speech interrupts ACTIVE model output — NOT on every user utterance. A user simply taking their normal turn while the model is idle is not an interruption, and drivers must not report it as one (e.g. a raw "speech started" frame must be gated on whether a model response is actually in flight).
Turn detection / VAD is owned by the provider. The agent layer uses this hook to cancel
the model's current turn and to fire the cancellationToken of any in-flight
delegated agent run — a stale delegated result must never be narrated into a conversation
that has moved on.
Invoked when the provider reports a true barge-in interruption.
Registers a handler for model audio output frames (the audio media plane).
Invoked with each output audio frame as an ArrayBuffer.
Registers a handler for model tool-call requests.
Consumers execute the requested tool (under the session's context user) and feed the result back to the model.
Invoked with each RealtimeToolCall.
Registers a handler for transcript events (the text stream).
Consumers typically forward these to the control plane and persist them as
ConversationDetail records.
Invoked with each RealtimeTranscript (partial or final).
Registers the set of tools the model may call, translating them into the provider's native function-calling format.
The Core-level RealtimeToolDefinition is intentionally minimal: BaseRealtimeModel
lives in the lowest AI layer and cannot depend on the richer tool metadata defined in
higher packages (the agent layer) — doing so would create an illegal upward/circular
dependency. The agent layer is responsible for mapping its richer tool metadata down
to this Core type before calling RegisterTools, and the concrete driver maps this Core
type up to the provider's native function-calling schema.
Note that some providers (e.g. Eleven Labs) bind to a pre-declared tool set on a server-side agent configuration; for those, the driver maps these definitions onto the pre-declared tool names rather than registering arbitrary schemas at session start.
Idempotency rule: a post-start registration of a set IDENTICAL to the set supplied at connect time (via RealtimeSessionParams.Tools) MUST be a no-op. Providers that bind their tool set at connect time and cannot re-declare schemas on an open session MUST no-op (and may log) rather than degrade the conversation — e.g. by injecting schema text into the conversation as content. A genuinely DIFFERENT post-start set on such a provider is unsupported and should be surfaced as a warning, not silently mangled.
The tools to expose to the model.
via the mutable system_prompt (the protocol has no purpose-built context
channel): the note is appended under a "Background updates" heading and the FULL
prompt is re-sent via session.update — a config write, so it never triggers or
disturbs generation and is sent immediately even mid-reply. The model has the notes
available the next time it speaks.
Sends the initial session.update frame carrying the full server-authored session
config (prompt, tools, voice, turn detection). Always the FIRST client frame — the
provider drops audio sent before the session is configured and ready.
tool.result correlated by call_id. Sent IMMEDIATELY (never queued): the
provider's own guidance is "send it the moment your tool returns — no buffering, no
waiting on reply.done"; it owns the continuation and speaks the result as soon as the
floor allows. The wire result slot expects a JSON-STRING (not an object), which is
exactly the contract's output shape — it passes through verbatim. The reply flag is
set eagerly so a queued narration can't slip in before the spoken result.
Records the tool set declared at connect time. Called by AssemblyAIRealtime.StartSession.
Resolves once the server's session.ready arrives (the provider's confirmation that
the session config is applied); rejects if the transport dies first. Awaited by
AssemblyAIRealtime.StartSession so the session is never handed to a consumer
before it is actually configured.
Concrete IRealtimeSession backed by a raw AssemblyAI Voice Agent websocket.
Owns the inbound translation (flat snake_case wire events → Core events) and the outbound translation (Core calls → wire frames). Created by AssemblyAIRealtime.StartSession; never instantiated directly by consumers.
Provider-behavior notes (the contract deltas a consumer should know):
transcript.user.deltafragments followed by a finaltranscript.user; agent turns arrive as one finaltranscript.agent(carryinginterrupted: trueafter a barge-in, in which case the text reflects the truncated turn — already the authoritative record, no ElevenLabs-style correction event follows).reply.donewithstatus: 'interrupted'— the provider owns turn detection AND tracks its own audio emission, so this is the authoritative "user speech cut off active model output" signal (rawinput.speech.startedfires on EVERY user utterance and is NOT an interruption per the base contract; the server bridge also cannot see host-side playback, so it defers to the provider's verdict).reply.createcarries per-response instructions — no ElevenLabs-style user-turn emulation, no fidelity caveat), queued behind any in-flight response per the contract's collision rule.system_prompt: the protocol has no non-interrupting context channel, butsession.updatemay rewrite the prompt mid-session without triggering generation — notes are appended under a "Background updates" heading and the full prompt is re-sent. Never interrupts; the model sees the notes the next time it speaks.toolsis a mutablesession.updatefield, so (unlike ElevenLabs) a genuinely different post-start set is applied to the live session; an identical set is a silent no-op per the idempotency rule.