Member Junction
    Preparing search index...

    Class OpenAIProtocolRealtimeClientAbstract

    The shared OpenAI-protocol brain for realtime CLIENT drivers — the transport-agnostic middle layer between BaseRealtimeClient and the per-provider drivers.

    Every OpenAI-protocol provider (OpenAI itself over WebRTC, xAI Grok Voice and self-hosted HuggingFace over websockets) shares the exact same event vocabulary and turn discipline, so this class owns — ONCE — the pieces that were previously cloned per driver:

    • Inbound event dispatch (handleEvent): GA + beta transcript names, input transcription, tool calls, barge-in gating, playback-buffer events, provider error frames.
    • The response state machine: responseActive set on response.created / cleared on response.done; tool-result response.creates queued while a response is in flight and flushed on response.done so the model ALWAYS voices delegated results.
    • Narration-kind tagging: RequestSpokenUpdate marks the NEXT response as 'narration' so its transcripts are emitted with Kind: 'narration' (ephemeral).
    • The outbound actions: SendText (implies barge-in), SendContextNote, SendToolResult, CancelActiveResponse, SetMuted.

    Transport specifics stay in subclasses through a small set of seams: canSendEvents / sendProtocolEvent (the wire), stopAudioOutput (how already-generated speech is silenced), handleAudioDeltaFrame (websocket transports enqueue PCM; WebRTC ignores — audio rides the peer connection), and per-provider behavioral overrides (onUserTranscriptFrame, onSpeechStartedFrame, releasesBusyFlagOnToolCall).

    Hierarchy (View Summary)

    Index

    Constructors

    Properties

    Accessors

    Methods

    AddTrack ApplySessionConfigTiming ArmConnectionDurationTimer AssertTransportSupported attachInputAudioMeter attachOutputAudioMeter buildSpokenUpdateEvent CancelActiveResponse canSendEvents clearRemoteMediaStream closeAudioMeters Connect ConnectionClosesAtCredentialExpiry CredentialConnectionLimit currentSessionInstructions Disconnect DisposeReconnectTimers doSendToolResult dropInputHeldDuringReconnect emitError emitInterruption emitRemoteVideo emitStateChange emitToolCall emitTrackStateChange emitTranscript emitUsage emitUserSpeechStart flushPendingResultResponse GetAudioActivity GetRemoteMediaStream handleAudioDeltaFrame handleEvent handleProtocolMessage holdsHardReconnectPoint isCredentialNearExpiry isHostNetworkOnline isToolResultSilent IsTrackEstablished isTransportUsable isTurnActiveForReconnect isUserSpeaking logInboundEvent logOutboundEvent negotiateTracks NotifyConnectionEnding OnError OnInterruption onNonJsonFrame onOutputAudioBufferStarted onOutputAudioBufferStopped onReconnected onReconnectFailed onReconnecting OnRemoteMediaStream OnRemoteVideo onResponseStarted onSessionCreatedFrame onSpeechStartedFrame OnStateChange OnToolCall OnTrackStateChange OnTranscript OnUsage OnUserSpeechStart onUserTranscriptFrame publishRemoteMediaStream Reconnect RecordResumptionHandle recordToolResult recoverLostTransport refreshCredentialForReconnect RemoveTrack ReplaceMicrophone? requestResultResponse RequestSpokenTurn RequestSpokenUpdate resetResponseState resetResponseStateForNewConnection resolveToolName ResumeOnOldConnection seedConversationAfterMove seedConversationHistory SendContextNote sendEvent sendProtocolEvent SendText SendToolResult SendVideoFrame? SetMuted setState SetTokenRefreshCallback stopAudioOutput TriggerReconnect readInstructions

    Constructors

    Properties

    activeResponseKind: "normal" | "narration" = 'normal'

    The kind of the response currently in flight. Event ordering (confirmed against the live OpenAI API): response.created → transcript deltas → *_audio_transcript.done → response.done. The transcript-done frame therefore arrives while the kind is still set, letting onAssistantDone classify the turn; response.done then resets it.

    callsFromBeforeReseed: Set<string> = ...

    Tool call IDs that were in flight when a non-native seed reconnect occurred.

    configuredCallDurationSeconds: number

    Duration of a single provider connection in seconds from ClientRealtimeSessionConfig.ConnectionMaxSeconds.

    configuredReconnectBufferSeconds: number = DEFAULT_REALTIME_RECONNECT_BUFFER_SECONDS

    Buffer in seconds before CallDurationSeconds expiry at which Reconnect is triggered.

    configuredSessionMaxSeconds: number

    Credential/session token lifetime in seconds from ClientRealtimeSessionConfig.SessionMaxSeconds (or MaxSessionSeconds).

    configuredTrustNativeResumption: boolean

    Optional metadata override for whether native resumption is trusted on this endpoint.

    confirmedResponseActive: boolean = false

    True while a response the provider has CONFIRMED (response.created seen, response.done not yet) is in flight. Distinct from responseActive, which is set EAGERLY before a local response.create is even sent: if the provider REJECTS that create (an error frame with no response.created), responseActive would otherwise stay stuck true forever. This flag lets onErrorFrame tell "the eager flag is a phantom for a rejected create" (no confirmed response) from "a real response is genuinely active" (e.g. a concurrent VAD turn), so it clears the phantom without disturbing a live turn. Set on response.created, cleared on response.done.

    conversationMemory: RealtimeConversationMemory = ...

    Rolling conversation memory used to re-seed provider context on reconnect.

    currentState: RealtimeClientState = 'closed'

    The client's own view of the session state — mirrors what emitStateChange last reported, EXCEPT after a tool call is emitted: the host typically shows its own busy state then, so the client silently leaves 'speaking' (no emission) to preserve the host's indicator until the result reply starts (see onToolCallFrame).

    inputAudioMeter: IRealtimeAudioMeter = null

    Meter over the USER's microphone, when the driver attached one.

    micStream: MediaStream = null

    The mic stream owned by the current connection (used by the shared SetMuted).

    outputAudioMeter: IRealtimeAudioMeter = null

    Meter over the AGENT's audio output, when the driver attached one.

    pendingAssistantText: string = ''

    Accumulates the in-flight assistant transcript across delta frames.

    pendingLocalResponseCreates: number = 0

    Count of locally-initiated response.creates whose response.created echo has not arrived yet. While non-zero, a response.done belongs to an EARLIER (typically just-cancelled) response — it must not clear the busy flag, flush the queued trigger, or flip the state for the response we just started (usage is still emitted). Consumed by response.created.

    pendingNarrationKind: boolean = false

    Set by RequestSpokenUpdate just before it sends its response.create, and CONSUMED by the very next response.created frame, which stamps activeResponseKind for that turn. Narration is only requested while the model is idle (hosts gate on IsBusy), so under normal ordering the next response.created is ours.

    pendingResultResponse: boolean = false

    Set when a tool result is ready while a response is active; sent on the next response.done.

    reconnectController: RealtimeReconnectController = ...

    Unified I/O-free reconnect controller shared with server-side BaseRealtimeSession.

    responseActive: boolean = false

    True while the model has a response in flight; gates narration + queues the tool result.

    tokenRefreshCallback: RealtimeClientTokenRefreshCallback = null

    Optional callback installed by RealtimeSessionRuntime to refresh the provider token mid-call.

    tracks: RealtimeTrack[] = []

    Tracks requested, established, or denied on this session.

    Accessors

    • get CallDurationSeconds(): number

      Effective connection ceiling in seconds. Metadata and runtime caps narrow the driver's provider ceiling (ClampSessionSecondsValue(DefaultCallDurationSeconds, RuntimeCallDurationSeconds, configuredCallDurationSeconds)), never widen it.

      Returns number

    • get InboundVideoRate(): number

      The frame rate (frames per second) the live inbound video track was negotiated at — the model's ceiling or the session's request, whichever is lower — or undefined when no inbound video track is live or none declared a rate. Anything pacing a video feed reads THIS, never a constant of its own.

      Returns number

    • get IsAudioPlaying(): boolean

      true while model audio is AUDIBLY playing in the browser. Distinct from IsBusy — audio plays at realtime while generation finishes early, so the model can be "idle" while speech is still coming out of the speaker. Hosts must gate narration on BOTH, or queued utterances come out late and stale.

      Returns boolean

    • get IsReconnectCycleActive(): boolean

      True from the start of a reconnect cycle until it succeeds or fails. Unlike IsReconnecting, it stays true while a planned move waits between attempts with the call live again on its old connection. A driver that would otherwise end the call on a loss of that connection checks this and hands the loss to the running cycle (TriggerReconnect('connection-lost')) instead.

      Returns boolean

    • get MaxInboundVideoStreams(): number

      How many concurrent inbound video streams the model accepts: the ceiling the driver declared when it negotiated tracks, else 1 when an inbound video track is live and 0 when it is not.

      A source arbiter reads this to decide whether several live video sources must be narrowed to one (with a note to the model on each switch) or passed through untouched.

      Returns number

    • get RecoversLostTransport(): boolean

      Whether an unexpected loss of a live call's transport is handed to the shared connection-lost reconnect. On by default for every driver that can reconnect mid-call; a driver whose transport closes benignly and whose host reads that close as the end of the session overrides it to false.

      Returns boolean

    • get RequiresPostTurnResumptionHandle(): boolean

      Whether this driver's provider emits a fresh resumable handle shortly after each turnComplete (Gemini / Vertex AI Live), so planned moves wait for that post-turn handle.

      Returns boolean

    • get RuntimeCallDurationSeconds(): number

      Optional runtime ceiling override (e.g. Gemini's 120s ceiling when uncompressed video is active). Narrowed against DefaultCallDurationSeconds and configuredCallDurationSeconds via ClampSessionSecondsValue.

      Returns number

    • get StateToRestoreOnOldConnection(): RealtimeClientState

      The state to report again when ResumeOnOldConnection goes back to the old connection, or null when the old connection has reported a state of its own since the move started (it keeps delivering events during the attempt), which is current. 'speaking' while the turn is open, else 'listening': the turn, not the state the move replaced, says whether the agent is answering, since a driver can leave 'speaking' without reporting it (a tool call) and the old connection can start an answer during the attempt. A 'speaking' report opens a turn, and one with no answer behind it would hold the next attempt until the last call. 'connected' is never reported: a host keeps showing the move for it.

      Returns RealtimeClientState

    • get SupportsInboundVideo(): boolean

      Whether the model takes inbound video on this session: the driver declared an inbound video track it supports when it negotiated tracks, and a stream limit above zero. No video track need be live; one is added when a capture starts (AddTrack). false before tracks are negotiated, and for a driver that does not negotiate them.

      Returns boolean

    • get SupportsRelayTransport(): boolean

      Whether this driver can open a relay session (ClientRealtimeSessionConfig.Transport 'relay'): connect to MJAPI's realtime relay at the session's RelayUrl rather than to its provider. false here; a driver that speaks its provider's protocol through the relay opts in by overriding it (the Gemini Enterprise client does).

      Returns boolean

    Methods

    • Adds a track to a running session, such as inbound video when the user starts their camera, resolved against what the driver supported when it negotiated at connect. The new track is 'live', or 'unsupported' with the reason (the model takes no such track, or no more inbound video streams). A live track with the same direction, modality and source is returned as it is. Other tracks keep their ids.

      The base re-negotiates locally, which is all a driver needs when its provider accepts the media without a setup change (Gemini Live takes video frames at any time). A driver whose wire must change overrides this and calls it.

      Parameters

      Returns RealtimeTrack

      The track, or null when the session has not negotiated tracks: before it connects, or with a driver that does not negotiate them (only the Gemini driver does today).

    • Arms (or re-arms) the soft and hard connection-ceiling timers for the current transport socket.

      • Soft timer opens the turn-boundary window before CallDurationSeconds.
      • Hard timer fires at CallDurationSeconds - ReconnectBufferSeconds and forces a reconnect; while the model is answering it waits for the turn to end, up to shortly before CallDurationSeconds. Both measure from the connection deadline instead when it comes before CallDurationSeconds (connectionDeadlineAtMs: the expiry of the credential the connection opened on, on a driver whose connections close with it).

      Returns void

    • The response.create frame for one spoken update — carrying the session identity, not replacing it.

      ⚠️ response.instructions IS A FULL OVERRIDE OF THE SESSION SYSTEM PROMPT for that one response, not an addition to it. That is the whole reason this method exists. The server-side twin of this driver (openAIRealtime.ts) already states the rule and guards the BLANK case on it — forwarding '' "would wipe the co-agent identity framing" — but the NON-blank case was never guarded, so every caller that supplied a real direction wiped it just as thoroughly and much less visibly.

      What that cost, live: the one turn that rides this method had no persona, so the model fell back to its vendor default and opened a hiring interview with "I'm ChatGPT, your friendly voice companion" while every other turn in the same session correctly said "I'm Sam Rivera, Support Team Lead" (bizapps-caliber#397 / MJ#4591). Every commentary-style caller is exposed the same way — an opening turn, a silence check-in, a delegation narration — and none of them should have to restate an identity the session already carries.

      Three cases, and the middle one is the fix:

      • Blank direction → a bare response.create. "Respond now, under the session prompt" (the meeting-mode bridge trigger passes ''). Matches the server twin exactly.
      • Direction + session instructions → identity, then direction. The identity leads because it is context, and the direction is LAST because it is the thing being asked for now — the most recent line is the one a model weights hardest.
      • Direction, no session instructions → the direction alone. A session minted without instructions has no identity to lose, and prefixing a blank line would be noise.

      Parameters

      • instructions: string

      Returns OAIProtocolResponseCreateEvent

    • Returns void

      response.cancel (only when a response is actually in flight) and silences already-generated speech via the transport's stopAudioOutput. Resets the local response state machine (active flag, narration kind, accumulated transcript) but PRESERVES any queued tool-result trigger: delegated work is never affected by a floor-control cancel, and the queued trigger still fires on the cancelled response's trailing response.done. No-op when idle or when the transport is not open.

    • Opens the provider connection using the server-minted ephemeral credential and applies config.SessionConfig once the control channel is ready — so prompt and tool authority stay server-side even though the browser owns the socket. The payload's SHAPE is a private pact between this driver and the same-keyed server driver that minted it (see driver obligation #8); how it is applied is entirely driver-specific.

      Parameters

      • config: ClientRealtimeSessionConfig

        The server-minted client session config (provider, model, ephemeral token or relay transport, session config). A driver refuses a transport it does not speak first (see AssertTransportSupported).

      • micStream: MediaStream

        The user's microphone capture stream. The caller acquires it (so IT owns the permission prompt UX); the client attaches it to the transport and stops its tracks on Disconnect.

      • OptionalcameraStream: MediaStream

        OPTIONAL camera capture stream for a VIDEO session (the model "sees" the user). Only attached by video-capable drivers; audio-only drivers ignore it (back-compatible — existing callers and drivers need not pass or read it). The model/avatar's video comes BACK via OnRemoteVideo.

      Returns Promise<void>

    • The session-level instructions currently in force — the co-agent's identity, persona and standing directives, as applied by session.update.

      A SEAM, not a field read, because the pact lives in a different place per transport: the websocket subclass keeps it in sessionObject, the WebRTC driver in its own sessionConfig. The base holds no copy of either, so it answers "no identity on record" and each transport overrides with the one it actually applied. A transport that forgets to override loses only the carry — buildSpokenUpdateEvent still sends the caller's direction — which is the old behaviour rather than a new failure.

      Null rather than '' for "this session carries no identity", so buildSpokenUpdateEvent can tell "nothing to preserve" from "preserve this".

      Returns string

    • Sends the tool result back as a function_call_output conversation item, then triggers a reply — immediately if the model is idle, otherwise queued until the current response (e.g. a progress narration) finishes. Without the queueing the result's response.create would collide with an in-flight narration and be dropped, leaving the model silent when delegated work comes back.

      Parameters

      • callID: string
      • outputJson: string

      Returns boolean | void

    • Returns the AGENT's remote-audio MediaStream when this driver owns a tappable agent-audio plane (a WebRTC driver's remote track, or a PCM driver's playout output), or null otherwise. Hosts use it to MIX the agent's voice into a browser-side recording alongside the mic; null (drivers with no tappable plane, or before the stream lands) degrades gracefully to mic-only capture. Hosts may still call it as client.GetRemoteMediaStream?.(). Pure read — never mutates.

      Returns MediaStream

    • Handles one base64 PCM16 audio delta. Default: no-op — on the WebRTC transport the agent's audio rides the peer connection's remote track, not data-channel frames. Websocket transports override to enqueue into their local playout engine.

      Parameters

      • _deltaBase64: string

      Returns void

    • Whether the user said something the current connection has not answered yet, with no turn open for it. The hard-point move then waits as it does for an open turn, until shortly before the provider limit. false here: a driver whose provider opens the turn only once the answer starts overrides it, and calls reconnectController.NotifyIdleStateChanged() when the hold ends without a turn.

      Returns boolean

    • Registers the (single) interruption handler.

      True barge-in only: fires ONLY when USER INPUT CUT OFF ACTIVE MODEL OUTPUT — a model response in flight or audio audibly playing when the user took the floor. A user simply taking their normal turn while the model is idle is NOT an interruption, and drivers must not report it as one (e.g. a raw "speech started" frame must be gated on whether a response is actually active or audio is playing).

      On interruption, drivers must also flush locally-owned playback and report IsAudioPlaying === false promptly (driver obligation #3). Hosts typically use this hook to abort in-flight delegated work so a stale result is never narrated into a conversation that has moved on.

      Parameters

      • handler: () => void

      Returns void

    • Registers a handler invoked when the agent's remote-audio stream becomes available — IMMEDIATELY (synchronously) if it has already landed, otherwise when the driver publishes it (a WebRTC track typically lands AFTER Connect resolves). Lets a host attach the agent voice to a recording that began before the stream landed. Handler errors are isolated (logged) so a host bug never disturbs the call. Handlers are session-scoped: clearRemoteMediaStream (called from Disconnect) drops them.

      Parameters

      • handler: (stream: MediaStream) => void

      Returns void

    • Registers the (single) remote-VIDEO handler — the model/avatar's video for a VIDEO session (a talking-head the host renders, e.g. as the agent's tile). Invoked once the provider's video is available.

      The video is a MediaVideoSource: a live stream (a WebRTC track), or a player that must own the <video> element (MSE or WebCodecs playout of encoded avatar video). Hosts show either with AttachVideoSource.

      Optional capability: audio-only drivers (the default) never emit — registering is always safe, but hosts must not assume video arrives. A driver whose model sends video (the server driver declares an outbound video track in its session's Capabilities.SupportedOutboundTracks, or renders an avatar, BaseRealtimeModel.SupportsAvatarOutput) calls emitRemoteVideo when it is live.

      Parameters

      • handler: (video: MediaVideoSource) => void

        Invoked with the remote video when it becomes available.

      Returns void

    • The user started speaking. TRUE barge-in only when it cut off active model output (a response in flight or audio audibly playing) — a normal turn while the model is idle is NOT an interruption, so the emission is gated (base-contract rule). Transports that own their playback override to also flush the local playout queue.

      Returns void

    • Registers the (single) usage handler.

      Emissions carry token deltas for the response/turn that just completed (see RealtimeClientUsage — deltas preferred; cumulative-only providers must convert in the driver). Hosts accumulate and relay/persist on their own cadence (e.g. the voice session service debounces a RelayRealtimeUsage mutation onto the co-agent prompt run).

      Optional capability: drivers whose provider exposes no usage telemetry simply never emit — registering a handler is always safe, but hosts must not assume emissions arrive. See RealtimeClientUsage for per-driver availability.

      Parameters

      Returns void

    • Registers the single handler invoked when the provider's voice-activity detection saw the user start to speak. Fires once per detected onset and may fire again inside one turn after a pause; it carries no text. When the provider's event names the conversation item this speech belongs to (OpenAI-protocol input_audio_buffer.speech_started.item_id), itemId carries that id so the host can pair the onset with the matching user transcript (RealtimeClientTranscript.ItemId); otherwise itemId is undefined. Unlike OnInterruption, it fires whether or not the model was speaking.

      Parameters

      • handler: (itemId?: string) => void

      Returns void

    • Emits the user's spoken-input transcription. Default (OpenAI/HuggingFace): each .completed frame is one final caption. Providers that STREAM the completed event (Grok re-sends the full growing text each time) override to collapse the stream into a single in-place-updating bubble.

      Parameters

      • transcript: string
      • OptionalitemId: string

      Returns void

    • Hands an unexpected loss of the live call's transport (a close or error the consumer did not ask for) to the shared reconnect controller, which retries within its recovery window and reports reconnect-failed when the window closes. Returns false when there is no live call to save, so the caller keeps its terminal handling: the connect has not finished (Connect reports that failure itself), the call already ended, or the driver opts out.

      A loss while a reconnect cycle runs is the loss of the transport the cycle may go back to after a failed attempt (the callers' identity checks drop events of a transport the cycle opened or already let go of). It goes to the cycle, even on a driver that opts out of starting one, and returns true: the cycle then never returns the call to a dead transport, and goes on as a lost connection's.

      Live means the call went live and has not ended (callWentLive), not a speech state: during a tool run the state is 'connected' (see onToolCallFrame), and a loss then is as recoverable as one mid-sentence.

      Parameters

      • detail: string

      Returns boolean

    • Moves a connected session onto the microphone stream's current audio track: call it after the track changes (a device switch, or a lost device replaced by the default) or to use another stream. The connection stays up and the provider hears the new track. The driver now owns this stream as it owned the one given to Connect; the previous stream's tracks are the caller's to stop.

      Mute follows the track: the new track's enabled flag decides it, so a caller replacing a muted track carries enabled over (the LocalMediaController does).

      Optional capability (driver obligation #10): a driver that can't rebind its transport leaves it undefined. Does nothing when the session isn't connected.

      Parameters

      • micStream: MediaStream

        The stream to follow, usually the one given to Connect after the controller swapped its track.

      Returns Promise<void>

      (rejects) When the stream has no audio track; the session keeps its current microphone.

    • Asks the model to speak (a tool result or typed-text reply) — immediately if it's idle, otherwise queued until the current response finishes. An immediate trigger also CONSUMES any queued trigger debt: every payload item is already in the conversation, so one response.create voices everything (e.g. typed text barging in over a narration that had tool results queued behind it).

      Returns void

    • Triggers ONE short spoken update with the given instructions. Marks the upcoming response as 'narration' (flag consumed by the next response.created) so its transcripts are emitted with Kind: 'narration' — ephemeral by contract. Sets responseActive eagerly so a tool result landing mid-narration queues instead of colliding.

      Skips when busy (base-contract collision rule — drivers MUST queue or skip): a response.create sent while a response is in flight would be rejected/garbled by the provider, and narration is disposable by contract, so the update is dropped with a debug log rather than queued to come out late and stale. Hosts SHOULD still gate on IsBusy / IsAudioPlaying for timing quality.

      The wire frame is built by buildSpokenUpdateEvent, which is where the identity rule lives — see that method for why the caller's direction is never sent on its own.

      Parameters

      • instructions: string

      Returns void

    • Resets the response bookkeeping for a connection that has just replaced the old one, and returns whether an answer was in flight (or queued) on the old one. Every count and flag here describes the old connection: a response.create it never confirmed would otherwise make the new connection's real response.done look stale and leave IsBusy stuck.

      Returns boolean

    • Seeds the new connection after a move (when sendsSeed; a natively resumed one already has the history) and notes whether the move cut off an answer the user is still owed: one was in flight on the old connection (answerWasInFlight, from resetResponseStateForNewConnection) and the conversation ends with what it answers (a user turn or a tool result), not counting the context notes after it. Neither a seeded nor a resumed session replies by itself, so without this the question sits unanswered in every later seed. An answer that completed is in memory as an assistant turn and is not asked for again.

      A seeded connection knows the conversation as the seed sent it. A resumed one knows it as the provider kept it, which includes what the old connection heard while the attempt was opening, after context.Seed was built: so then the memory as it is now decides.

      Parameters

      Returns void

    • Injects typed text as a user-role message conversation item, then triggers a reply through the SAME collision-safe path tool results use (requestResultResponse). No-op when the transport isn't open.

      SendText implies barge-in (base-contract rule): an active spoken response is cancelled via CancelActiveResponse before the text is injected, so the typed turn takes the floor immediately instead of waiting behind stale speech. When nothing is active the cancel is a no-op and the reply triggers immediately.

      Parameters

      • text: string

      Returns void

    • Parameters

      • callID: string
      • outputJson: string

      Returns void

      a move a result is held, like typed text, and goes out once the call is live again: on the new connection after the swap (onReconnected), or on the old one when a planned move goes back to it (ResumeOnOldConnection). Sent on now, it would be lost: a lost transport cannot carry it, and a closing one takes it along after the seed for the new connection was built. It is recorded into memory when it goes out, so it reaches the model and the seed of a later move exactly once.

    • Streams one video frame as base64-encoded image data to the model. Optional capability — drivers that support inbound video tracks implement this.

      Parameters

      • base64Image: string

        Base64-encoded image data.

      • OptionalmimeType: string

        Image MIME type (defaults to 'image/jpeg').

      • OptionalsourceId: string

        Which source this frame came from. A driver whose model takes ONE inbound video stream ignores it (the source arbiter already chose what to send); a driver whose model takes several routes the frame to the track bound to that SourceID.

      Returns boolean

      true if accepted and sent; false if dropped (throttled, unestablished, etc.).