AbstractProtectedapiOnly sub-classes can access the API key
The provider-native voice ids this model can speak with (e.g. OpenAI alloy/echo/shimmer). The
model/driver is the authoritative owner of "what voices do I support", so each driver declares its
own — used to populate the dev voice picker. Default empty (a driver that hasn't declared voices
yields no picker options, falling back to the configured/default voice).
NOTE: this is the near-term, driver-owned source of truth. Long term this should move to metadata so providers that let users add their OWN voices (e.g. ElevenLabs) can be enumerated dynamically.
The supported voice ids (id + human label), or [] when none are declared.
Whether this driver can mint an ephemeral, server-scoped client credential for a client-direct realtime session (the browser opens its own provider socket using a short-lived token the server minted).
Defaults to false. Providers that support browser-direct sessions override this to true
and implement CreateClientSession. The server-bridged topology (where the provider
socket lives on the server, via StartSession) is supported by every driver regardless
of this flag.
true if CreateClientSession is supported; false otherwise.
Whether this driver's sessions carry a video track in addition to audio — i.e. the model accepts video input (it can "see" the user's camera) and/or emits video output (a talking-head avatar / generated video), in sync with audio.
Defaults to false (audio-only — today's realtime models). Video-capable drivers (a native
multimodal realtime model, or an avatar provider) override this to true. The session's media
plane is media-tagged (IRealtimeSession.SendInput takes a RealtimeMediaKind;
IRealtimeSession.OnVideoOutput delivers video-out), so a video session reuses the entire
realtime contract — only the media frames gain a video kind. Resolution prefers a video-capable
model when an agent requests video, and degrades to audio-only otherwise.
true if sessions can carry video; false (audio-only) otherwise.
Mints an ephemeral, server-scoped client credential plus a provider-native session config for a client-direct realtime session.
In the client-direct topology the browser owns the provider socket (e.g. WebRTC), but the
server still controls the prompt and tool set: it mints a short-lived token and hands back
a ClientRealtimeSessionConfig whose SessionConfig the matching client driver applies
when it opens its socket. This keeps prompt/tool authority server-side even though the media
plane is client-direct.
Not every provider supports this (some only expose a server-bridged socket), so this is a
concrete method that throws by default rather than an abstract one — that would force every
existing and future driver to implement it. Providers that support it override
SupportsClientDirect to true and override this method.
The session parameters (system prompt, tools, model, config bag).
A promise resolving to the minted ClientRealtimeSessionConfig.
AbstractStartOpens a stateful duplex session with the provider.
The returned IRealtimeSession is the long-lived handle that streams media and transcripts in both directions, surfaces tool calls and usage telemetry, and reports provider-detected interruptions (barge-in). The caller is responsible for closing the session via IRealtimeSession.Close.
Configuration for the session (system prompt, tools, initial context, model, and an open config bag).
A promise resolving to the live session handle.
Base AI model class, used for everything else in the MemberJunction AI environment