The Vertex AI key. Never throws: a key that cannot be read fails the session it is used for, so code that constructs a driver per key without opening a session (the voice list) keeps working.
Static ReadonlySupportsGemini Live sessions accept dynamically-defined tools at connect/mint time.
ProtectedapiOnly sub-classes can access the API key
ProtectedEndpointGemini Enterprise: what a model renders (live avatars) is looked up on this endpoint.
The voices Gemini Live can speak with — used to populate the voice picker. Includes all 30 built-in Gemini voices supported in prebuiltVoiceConfig.
Gemini Live supports the client-direct topology: the server mints a short-lived ephemeral
auth token (v1alpha auth_tokens API) that the browser uses to open its OWN Live
websocket, while the server keeps prompt/tool authority by LOCKING the connect config into
the token via liveConnectConstraints.
Whether this driver's sessions carry a video track in addition to audio.
false in every MJ driver.
Nothing reads it, and no driver overrides it: it is false for Gemini Live, which takes camera video
and renders avatars. What a session does with video is declared on its IRealtimeSession.Capabilities:
RealtimeSessionCapabilities.SupportedInboundTracks for video in and
RealtimeSessionCapabilities.SupportedOutboundTracks for video out, each track with its encoding. Whether a
model renders a live avatar is SupportsAvatarOutput. A driver declares video there, not here.
ProtectedAvatarThe minted session config's avatar block: present only when this session renders an avatar, so the browser driver knows to expect video parts, their encoding, and whether they carry the voice.
The connect config the session was built with.
The model's profile on this driver's endpoint.
ProtectedAvatarWhat the mint tells the call about the avatar the session asked for: granted, or audio only and why. undefined when
it asked for none. Read from the connect config BuildConnectConfig built, so it says what the session does:
an avatarConfig there is the grant. Every Gemini driver returns it on its minted config.
The session parameters (the avatar request).
The connect config the session was built with.
ProtectedBuildBuilds the LiveConnectConfig from the Core session params: audio response modality,
input/output transcription, system instruction, sliding-window context compression
(GeminiRealtime.DefaultContextWindowCompression), session resumption (unless the
session is zero-data-retention), mapped tools, the neutral voice key mapped via
GeminiRealtime.buildSpeechConfig, plus any provider-specific overrides from the open
config bag.
ProtectedconnectOpens one Live connection of a bridged session on Gemini Enterprise. The SDK writes the model's resource name from its project and location, or the short name in API-key mode; for an API key on the regional route it gets the full name, which it keeps as given.
ProtectedconnectOpens a Live connection with connect, which passes the given callbacks to live.connect, and rejects if the
socket errors or closes before Gemini acknowledges the setup. live.connect (@google/genai 2.11+) resolves only on
the server's setupComplete frame and never rejects when the socket errors or closes first (a rejected key, model or
setup field), so without this the connect waits forever. A promise settles once, so callbacks after a successful
connect only reach the handlers. Every connectLiveSession override should open through this.
Mints a client-direct session through MJAPI's relay: a relay session whose policy opens every upstream connection
with the setup written from the connect config and a fresh bearer token (or the key's API key); Transport
'relay' with the relay URL as RelayUrl and an empty EphemeralToken (the browser holds no Google credential;
the URL carries the ticket); a pact whose config holds only what the browser needs to state (the response
modalities, and the avatar's name when one is granted); and, when the session asked for an avatar, the avatar status
the call reads to say why it shows none.
The session parameters (model, system prompt, tools, config bag, avatar request).
ProtectedCreateCreation seam for the GoogleAuth that mints this driver's tokens (tests return a fake).
The credentials' google-auth-library options, with the cloud-platform scope.
ProtectedCreateCreation seam for the Vertex-mode @google/genai client of bridged sessions (tests return a fake).
The credentials' Vertex options (VertexGenAIOptions), with the Live API version and any Live base
URL (see LiveUrl).
OptionalExchangeOptional WebRTC SDP exchange seam for drivers that support server-brokered WebRTC topologies.
The local SDP offer generated by the browser.
Optional_sessionConfig: Record<string, unknown>The session configuration to initialize the session with.
The answer SDP, provider session ID, and any prebill accounting.
ProtectedLiveThe Vertex AI Live websocket a session connects to. MJAPI's relay opens each upstream connection of a client-direct
session on this URL. A bridged session's @google/genai client opens it too; the driver reads it once, when it
builds that client, and gives the SDK everything before the socket path
(/ws/google.cloud.aiplatform.<version>.LlmBidiService/BidiGenerateContent) as its base URL, path prefix included,
with the version the socket path names. The SDK writes the socket path back after it; with nothing before the
socket path, it writes //ws/…, as it does on Google's own hosts. By default: the key's liveHost when it names
one, else the host @google/genai picks for the location (Google's global host for an API key without a project),
with the key's liveApiVersion, else v1.
A seam for a subclass registered at a higher priority (a mock upstream, a private endpoint, a proxy), not a setting.
Both paths send the session's credential to the host returned (an OAuth bearer token, or the key's API key), without
TLS over ws://. The token does not depend on the URL: the auth clients on both paths ask for the cloud-platform
scope, so google-auth-library fetches an access token and never makes the URL its audience.
The session's key, checked.
The session's location, checked; null for an API key on Google's global route.
A wss:// URL, or ws:// for a host without TLS, that ends in the socket path. A bridged session fails on
a moved URL the SDK cannot open as given: one without the socket path at its end, or with a user, a query or a
fragment.
ProtectedmintGemini Enterprise has no ephemeral tokens; a client-direct session goes through the relay instead.
ProtectedSessionThe minted session config, the private pact with the browser driver: a plain-JSON copy of the model and the config
the browser passes to live.connect, plus the model's facts on this driver's endpoint (idle signal, tooling, inbound
video limits, and the avatar block when the session renders one).
The model id.
The session's connect config, as BuildConnectConfig built it. It decides the avatar block.
OptionalbrowserConfig: LiveConnectConfigThe config the browser passes to live.connect: the full config here; a driver whose sessions
go through MJAPI's relay, which writes the setup, passes a smaller one.
Opens a server-side (bridged) session through @google/genai in Vertex mode: audio only, or with the avatar when
its host publishes it into a meeting room (RealtimeAvatarSettings.Delivery 'room'; see GeminiRealtime).
Whether the model renders a live avatar on this driver's endpoint, from its Live profile: never on the Developer API; on Gemini Enterprise for the models whose profile says so.
The model's API name.
StaticBuildProjects the full connect config down to the fields Gemini's ephemeral-token API accepts
as liveConnectConstraints.config. The token mint converts the provided keys into a
field mask over BidiGenerateContentSetup, and only generation-level fields are valid
there — systemInstruction, tools, and the transcription configs are NOT, and their
presence 400s the entire mint. Only defined fields are copied (an absent key must stay
absent so it doesn't enter the mask).
contextWindowCompression is left out on purpose. No test or doc shows the mask accepts
it, and a rejected mask would fail every client-direct session. The browser applies it from
SessionConfig, like the other server-built keys the token cannot lock.
sessionResumption: {} is locked as in Google's ephemeral-token example. The browser adds
the handle when it resumes, and Google accepts the same token for that until expireTime.
StaticDefaultThe context-window compression every Gemini Live session gets unless the config bag sets its own: a server-side sliding window with Google's default trigger and target sizes.
Without compression, Google ends a session when its context fills: about 15 minutes for audio only and about 2 minutes for audio plus video (Google's Live API session-management docs). With compression the server drops the oldest turns and the session continues. Session resumption handles a different limit (a single connection lasts about 10 minutes) and does not lift this one.
Returns a new object on each call so no session shares mutable config with another.
StaticExtractExtracts scheduling hints (__mj_scheduling or legacy scheduling) from the tool output,
strips both keys so they do not leak into the model's response payload, resolves the scheduling
directive accepting both 'INTERRUPT' and 'INTERRUPTED', and warns on unrecognized values or
unsupported models (delegates normalization to Core ExtractToolSchedulingHint).
StaticMapMaps MJ's lowercase thinking level onto the SDK's uppercase ThinkingLevel enum.
StaticMapMaps Core RealtimeToolDefinitions up to Gemini FunctionDeclarations.
The Core ParametersSchema is a JSON-schema object, so it rides in parametersJsonSchema
(the SDK's JSON-schema slot) rather than the OpenAPI-style parameters slot.
In Gemini Live, function calling defaults to asynchronous execution (Behavior.NON_BLOCKING).
On models that forbid synchronous blocking execution (e.g. gemini-3.8-live-extended-thinking),
Behavior.BLOCKING is refused locally and forced to Behavior.NON_BLOCKING with a warning,
preventing a hard error from the Live API server.
Optionalmodel: stringOptionalrequestedBehavior: string
Gemini Live on Gemini Enterprise (Vertex AI), with live avatars where the model renders them.
Registered as
GeminiEnterpriseRealtime. Its key (AI_VENDOR_API_KEY__GeminiEnterpriseRealtime, or the run's key) is a JSON string in one of theVertexAICredentialsshapes: an inline service account (aJWTclient), a key-file path (only in the environment key; any other key that names one is refused), neither for Application Default Credentials, or a Google Cloud API key (apiKey). Tokens are minted withgoogle-auth-libraryon each upstream open, never at the mint; an API key is sent asx-goog-api-keyon each upstream open. Neither leaves MJAPI. The Live socket iswss://<host>/ws/google.cloud.aiplatform.<version>.LlmBidiService/BidiGenerateContent: the location's host andv1, unless the key namesliveHost(environment key only) orliveApiVersion, or a subclass overrides LiveUrl. An API key picks its route by its key:alone, Google's global route, as
@google/genai's API-key mode does: the global host and the model's short name (publishers/google/models/<id>);with a
project(and alocation), the regional route: the location's host and the model's fullprojects/<p>/locations/<l>/…name, as for the other credentials. The SDK has no mode for it, so a bridged session gives the SDK the host and the full name itself.Client-direct (CreateClientSession): MJ writes the Live setup from the connect config, MJAPI's relay sends it upstream with a bearer token (or the key's API key) and filters what the browser sends after it, and the browser gets a relay session (
Transport'relay', the relay URL asRelayUrl, with the ticket in its path) and a pact with a minimal config: no system prompt and no tools.Bridged (StartSession):
@google/genaiin Vertex mode; audio only, unless the host publishes the avatar into a meeting room (RealtimeAvatarSettings.Delivery'room'): then the model renders it and the session sends its pieces to the host as frames throughOnVideoFrame.