Member Junction
    Preparing search index...

    Real-time, full-duplex driver for Google's Gemini Live API, implementing the Core BaseRealtimeModel primitive.

    The driver opens a bidirectional Gemini Live session, streams client audio in, and translates the provider's LiveServerMessage frames into the modality-agnostic Core events (RealtimeTranscript, RealtimeToolCall, RealtimeUsage, output media, and interruption). It registers via the MemberJunction class factory as GeminiRealtime and is resolved for MJ: AI Models typed Realtime.

    Testability: the live-session creation is isolated behind the overridable connectLiveSession seam, so unit tests inject a fake GeminiLiveSession and exercise the full message→event translation with no network.

    Hierarchy (View Summary)

    Index

    Constructors

    Properties

    DriverFacts: Readonly<RealtimeDriverFacts> = ...

    The ephemeral token MJ mints lives 30 minutes (GEMINI_CLIENT_TOKEN_EXPIRY_MS) and may open one connection per ~10 minutes of the cap plus a resume margin, so session resumption continues a conversation past Gemini's ~10-minute connection, and the token bounds the session.

    SupportsDynamicToolSet: true

    Gemini Live sessions accept dynamically-defined tools at connect/mint time.

    Accessors

    • get apiKey(): string

      Only sub-classes can access the API key

      Returns string

    • get CallDurationSeconds(): number

      Readonly getter returning the model driver's connection duration in seconds.

      Returns number

    • get DefaultCallDurationSeconds(): number

      Gemini Live caps each WebSocket connection at ~10 minutes (600 s).

      Returns number

    • get ReconnectBufferSeconds(): number

      Buffer in seconds before CallDurationSeconds at which the framework initiates a proactive reconnect. Defaults to DEFAULT_REALTIME_RECONNECT_BUFFER_SECONDS (10s).

      Returns number

    • get RuntimeCallDurationSeconds(): number

      Dynamic runtime override hook on the model driver (returns undefined unless overridden by a subclass).

      Returns number

    • get SupportsClientDirect(): boolean

      Gemini Live supports the client-direct topology: the server mints a short-lived ephemeral auth token (v1alpha auth_tokens API) that the browser uses to open its OWN Live websocket, while the server keeps prompt/tool authority by LOCKING the connect config into the token via liveConnectConstraints.

      Returns boolean

    • get TrustsNativeResumption(): boolean

      Server-side Gemini Live sessions trust Google's native sessionResumption handles (with Attempt 2 fallback to seed).

      Returns boolean

    Methods

    • The minted session config's avatar block: present only when this session renders an avatar, so the browser driver knows to expect video parts, their encoding, and whether they carry the voice.

      Parameters

      • config: LiveConnectConfig

        The connect config the session was built with.

      • profile: GeminiLiveResolvedProfile

        The model's profile on this driver's endpoint.

      Returns JSONObject

    • Returns the smallest candidate that is a positive integer; see ClampSessionSecondsValue. The rule "the caller's bag wins only if smaller" lives there, once: pass the provider cap and the bag cap together.

      Parameters

      • ...candidates: readonly number[]

      Returns number

    • Resolves whether a client-direct session minted by this driver trusts native resumption handles. On the Developer API (transport === 'direct'), browser sessions authenticate with v1alpha ephemeral tokens, where resuming with a handle connects and restores an empty session; those sessions use RealtimeConversationMemory seeding (false). Relay sessions on Gemini Enterprise (transport === 'relay') use server-side OAuth/API-key credentials on v1, where native sessionResumption handles preserve full audio context (true).

      Parameters

      Returns boolean

    • Creation seam for the underlying Gemini Live session.

      Production code routes through ai.live.connect; unit tests override this method to inject a fake GeminiLiveSession. Kept as a thin, single-responsibility method so the network boundary is the only thing tests need to replace.

      Parameters

      • args: GeminiConnectArgs

        Resolved model, connect config, and the server-message callback.

      Returns Promise<GeminiLiveSession>

      A promise resolving to the live session handle; it rejects if the socket errors or closes before Gemini acknowledges the setup.

    • Opens a Live connection with connect, which passes the given callbacks to live.connect, and rejects if the socket errors or closes before Gemini acknowledges the setup. live.connect (@google/genai 2.11+) resolves only on the server's setupComplete frame and never rejects when the socket errors or closes first (a rejected key, model or setup field), so without this the connect waits forever. A promise settles once, so callbacks after a successful connect only reach the handlers. Every connectLiveSession override should open through this.

      Parameters

      Returns Promise<GeminiLiveSession>

    • Mints an ephemeral, server-scoped Live credential for a client-direct session.

      The connect config is built EXACTLY as StartSession builds it (same BuildConnectConfig: audio modality, input+output transcription, system instruction, mapped tools) and is locked into the token via liveConnectConstraints + lockAdditionalFields: [] — so the API ignores any attempt by the browser to change the locked fields. The same config is ALSO carried in SessionConfig (as { model, config }) because the SDK still expects the client to pass a model/config at live.connect time; the token-side lock is what makes the server's prompt and tool set authoritative.

      Expiry: the token opens connections (the first, and each resume, which Gemini counts as a new session) until the session cap plus GEMINI_CLIENT_TOKEN_CAP_SKEW_SECONDS (at most GEMINI_CLIENT_TOKEN_EXPIRY_MS), one per effective connection length (ConnectionMaxSeconds, at most GEMINI_LIVE_CONNECTION_SECONDS) of the cap plus GEMINI_CLIENT_TOKEN_RESUME_MARGIN; the session's ability to send messages dies with it. The count rides SessionConfig.tokenUses so the browser refreshes before it spends the last use.

      Parameters

      Returns Promise<ClientRealtimeSessionConfig>

      The minted ClientRealtimeSessionConfig the browser authenticates + applies.

    • Optional WebRTC SDP exchange seam for drivers that support server-brokered WebRTC topologies.

      Parameters

      • _offerSdp: string

        The local SDP offer generated by the browser.

      • Optional_sessionConfig: Record<string, unknown>

        The session configuration to initialize the session with.

      Returns Promise<{ answerSdp: string; prebillSeconds: number; sessionId: string }>

      The answer SDP, provider session ID, and any prebill accounting.

    • Mint seam for the ephemeral auth token. Production routes through the SDK's authTokens.create on a v1alpha client (ephemeral tokens are v1alpha-only); unit tests override this to return a fake token with no network.

      Parameters

      • params: CreateAuthTokenParameters

        The auth-token create parameters (expiry, uses, live-connect constraints).

      Returns Promise<AuthToken>

      The created AuthToken (its name is the credential the browser presents).

    • Reconnect hook for server-hosted IRealtimeSession instances created by StartSession. Called by BaseRealtimeSession's duration timer (CallDurationSeconds - ReconnectBufferSeconds) or on unexpected transport close when resumable. Concrete provider drivers override this method.

      Parameters

      Returns Promise<void>

    • Resolves the effective single-connection duration in seconds from an open config bag (connectionMaxSeconds), narrowing this.CallDurationSeconds via ResolveRealtimeTiming.

      Parameters

      • Optionalconfig: Record<string, unknown>

      Returns number

    • Resolves the effective reconnect buffer in seconds from an open config bag (reconnectBufferSeconds), falling back to this.ReconnectBufferSeconds.

      Parameters

      • Optionalconfig: Record<string, unknown>

      Returns number

    • Resolves whether native session resumption is trusted for a session from an open config bag (trustNativeResumption), falling back to defaultTrust (or this.TrustsNativeResumption).

      Parameters

      • Optionalconfig: Record<string, unknown>
      • OptionaldefaultTrust: boolean

      Returns boolean

    • The minted session config, the private pact with the browser driver: a plain-JSON copy of the model and the config the browser passes to live.connect, plus the model's facts on this driver's endpoint (idle signal, tooling, inbound video limits, and the avatar block when the session renders one).

      Parameters

      • model: string

        The model id.

      • config: LiveConnectConfig

        The session's connect config, as BuildConnectConfig built it. It decides the avatar block.

      • browserConfig: LiveConnectConfig = config

        The config the browser passes to live.connect: the full config here; a driver whose sessions go through MJAPI's relay, which writes the setup, passes a smaller one.

      Returns JSONObject

    • Projects the full connect config down to the fields Gemini's ephemeral-token API accepts as liveConnectConstraints.config. The token mint converts the provided keys into a field mask over BidiGenerateContentSetup, and only generation-level fields are valid there — systemInstruction, tools, and the transcription configs are NOT, and their presence 400s the entire mint. Only defined fields are copied (an absent key must stay absent so it doesn't enter the mask).

      contextWindowCompression is left out on purpose. No test or doc shows the mask accepts it, and a rejected mask would fail every client-direct session. The browser applies it from SessionConfig, like the other server-built keys the token cannot lock.

      sessionResumption: {} is locked as in Google's ephemeral-token example. The browser adds the handle when it resumes, and Google accepts the same token for that until expireTime.

      Parameters

      • config: LiveConnectConfig

      Returns LiveConnectConfig

    • The context-window compression every Gemini Live session gets unless the config bag sets its own: a server-side sliding window with Google's default trigger and target sizes.

      Without compression, Google ends a session when its context fills: about 15 minutes for audio only and about 2 minutes for audio plus video (Google's Live API session-management docs). With compression the server drops the oldest turns and the session continues. Session resumption handles a different limit (a single connection lasts about 10 minutes) and does not lift this one.

      Returns a new object on each call so no session shares mutable config with another.

      Returns ContextWindowCompressionConfig

    • Extracts scheduling hints (__mj_scheduling or legacy scheduling) from the tool output, strips both keys so they do not leak into the model's response payload, resolves the scheduling directive accepting both 'INTERRUPT' and 'INTERRUPTED', and warns on unrecognized values or unsupported models (delegates normalization to Core ExtractToolSchedulingHint).

      Parameters

      • parsed: Record<string, unknown>
      • supportsScheduling: boolean
      • toolName: string

      Returns FunctionResponseScheduling

    • Maps MJ's lowercase thinking level onto the SDK's uppercase ThinkingLevel enum.

      Parameters

      • level: GeminiThinkingLevel

      Returns ThinkingLevel

    • Maps Core RealtimeToolDefinitions up to Gemini FunctionDeclarations.

      The Core ParametersSchema is a JSON-schema object, so it rides in parametersJsonSchema (the SDK's JSON-schema slot) rather than the OpenAPI-style parameters slot.

      In Gemini Live, function calling defaults to asynchronous execution (Behavior.NON_BLOCKING). On models that forbid synchronous blocking execution (e.g. gemini-3.8-live-extended-thinking), Behavior.BLOCKING is refused locally and forced to Behavior.NON_BLOCKING with a warning, preventing a hard error from the Live API server.

      Parameters

      Returns FunctionDeclaration[]