Protected ReadonlysessionMaps session id -> direct actions config resolved during session prep. Bounded for the same reason as sessionWireActionMaps.
Protected ReadonlysessionMaps session id -> (wireName -> MJActionEntityExtended) built during tool projection.
Bounded with MJLruCache (rather than a plain Map): the resolver holds ONE shared service
instance for the process lifetime (see class doc above), and there is no "session ended" hook
this service can key eviction off — every voice session ever prepared would otherwise leave an
entry here forever. Same TTL/maxSize rationale as promptRunWriteChains below.
Protected ReadonlytargetMaps targetAgentID -> (wireName -> MJActionEntityExtended) fallback built during tool projection. Bounded for the same reason as sessionWireActionMaps.
Static ReadonlyLegacyDEPRECATED legacy name of the narration prompt, from before the co-agent's rename from "Voice Co-Agent" to "Realtime Co-Agent". Deployments that have not re-synced the prompt seed still carry this name, so resolveNarrationInstructionsTemplate falls back to it (with a deprecation log) when RealtimeClientSessionService.NarrationPromptName is absent.
Static ReadonlyNarrationThe seeded name of the MJ: AI Prompts row whose TemplateText carries the first-person
progress-narration instructions (with a {{ progressMessage }} placeholder). Resolved at
session prepare time so the browser narrates with DB-driven, product-tunable wording.
Canonical value lives in realtime-narration.ts (shared with the server-bridged runner path).
Accumulates relayed usage DELTAS onto the co-agent AIPromptRun's TokensPrompt / TokensCompletion
(recomputing TokensUsed). Serialized against AppendPromptRunMessage on the same run so the
high-frequency usage checkpoint never overwrites freshly-appended transcript turns (and vice-versa).
Best-effort: load/save failures log and return false, never throw.
The per-modality details add into the usage record the run keeps in ModelSpecificResponseDetails
(RealtimeUsage.Input / .Output, which pricing reads at finalize); every other key there is kept. The
stored output video seconds never exceed the run's elapsed time plus 30 seconds, and the stored output video
tokens never exceed 6,192 a second of that: usage is client-reported, and avatar video carries a price, per
minute or per token.
The co-agent observability prompt run.
Input-token delta to add (caller clamps to >= 0).
Output-token delta to add (caller clamps to >= 0).
Optionaldetails: RealtimeUsageRecordPer-modality usage to add (input and output blocks, avatar video seconds included).
true when the accumulated usage was persisted.
ProtectedappendAppends the host-declared tools to a stable tool set, dropping any whose name is already taken.
OptionalhostTools: RealtimeToolDefinition[]Appends (or replaces) one transcript turn onto the co-agent's long-lived AIPromptRun.Messages,
so the realtime co-agent's conversation is captured on its run exactly like every other MJ agent
run — closing the observability gap where the run held only token totals, never the turns. The
run viewer can then show what the co-agent heard and said. Mirrors accumulatePromptRunUsage's
load/append/save pattern; best-effort and tolerant (logs, never throws).
replacePrevious swaps the last same-role message instead of appending — the streaming-correction
case (an interim assistant turn finalized into its full text). The stored shape is the standard
chat-message array ([{ role, content }, …]) the rest of MJ already reads from Messages.
NOTE: load-append-save carries the same benign race as usage accumulation; realtime turns are sequential per session so collisions are rare. A dedicated child turn-row entity would remove the race (and the blob rewrite) entirely — a future increment. Tool-call turns (the browser_ and Whiteboard_ channel tools) are a separate increment that requires the client to relay them.
true when the turn was persisted onto the prompt run.
ProtectedassembleAssembles the same memory/context block a loop agent injects, reusing AgentMemoryContextBuilder so there is no duplicated retrieval logic. The builder unshifts a system message onto a throwaway array, which we pull back out as plain text.
The prepare-session input.
The resolved co-agent.
The calling user.
The concatenated context text (empty string when nothing was injected).
ProtectedbuildBuilds the session-start app-context section from the AppContextSnapshot: where the user
is, what they see, and the live capability manifest, rendered via the shared
FormatAppContextNote (one wording for mint-time and streaming notes) under a clear heading.
Returns '' when no snapshot was supplied or nothing salient survives.
OptionalappContext: AppContextSnapshotThe app-context snapshot, or undefined.
The prompt section, or ''.
ProtectedbuildMaps the effective config's union-accumulated allowedAgents into RealtimeColleague
entries for the framing — resolving each target's display name + description from the agent cache
and its effective per-target disclosure (GetDisclosureForTarget). The LEAD (the voiced
target) is excluded so the co-agent never lists itself as a colleague. Empty when none configured.
The resolved effective configuration.
OptionalexcludeAgentId: stringThe lead/target agent id to exclude from the colleague list.
The colleague set (possibly empty).
ProtectedbuildAssembles the companion system prompt: the framing ("you are the voice for the target"), the co-agent's own system prompt text, the TARGET agent's identity/capabilities (Name + Description), the conversation history, and the same memory/context a loop agent assembles.
When the effective configuration carries a voice persona (realtime.voice.default), a
short "Voice & manner" section (tone / speaking style) is appended after the co-agent's
own prompt so the model speaks in the configured manner.
The prepare-session input.
The resolved co-agent.
The calling user.
The request-scoped metadata provider.
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration (voice persona source).
The concatenated system prompt (never empty — the framing is always present).
OptionaleffectiveConfig: RealtimeCoAgentConfigOptionaldriverClass: stringOptionalagentSessionID: stringBuilds the projected direct action tool definitions for a target agent if supported by the driver.
Gated by driver capability (SupportsDynamicToolSet) and explicit configuration opt-in
(Configuration.realtime.directActions.enabled = true).
With agentSessionID, the projection is recorded for the session, even when it is empty: relayed
direct-action calls for the session may run only the projected actions.
The target agent ID being voiced.
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration.
OptionaldriverClass: stringThe resolved vendor's DriverClass.
OptionalagentSessionID: stringThe array of projected direct action tools (empty if unsupported or disabled).
ProtectedbuildBuilds the "interactive-surface tools" exception clause appended to the co-agent framing when
the client supplied channel tools (browser_, Whiteboard_, …) as ExtraTools. Without it the
co-agent — told to route ALL work through invoke-target-agent — delegates browser/whiteboard
requests to the target agent (which has no live channel of its own) instead of driving the
surface itself, then hallucinates a "missing session id". The tools ARE already in its set
(buildStableToolSet merges [invokeTarget, ...extraTools]); this clause tells the model
to USE them directly. Returns empty for pure-voice sessions (no ExtraTools), keeping that
framing untouched. Generic by design — it names browser_ and Whiteboard_ tools only as
examples, so any future client channel is covered automatically.
OptionalextraTools: RealtimeToolDefinition[]The client-supplied channel tools, when any.
The exception clause (leading space included), or '' when there are no extra tools.
ProtectedbuildBuilds the meeting-mode discipline clause — present only for a multi-agent meeting session
(PrepareClientSessionInput.DisableAutoResponse). It tells the agent to hear the whole
conversation but speak only when addressed (named) or clearly called on, and never to talk over
others. This is the prompt half of "hear always, speak selectively"; the enforcement half is the
model's disabled auto-response + the bridge's addressing gate. Empty for a 1:1 call (prompt unchanged).
See plans/realtime/multi-agent-meeting-turn-taking.md.
The prepare-session input (carries the meeting flag + self names).
The meeting clause, or '' for a non-meeting session.
ProtectedbuildBuilds the provider-pact Config bag for the session: the effective config's resolved voice
settings (per GetProviderVoiceSettings — the persona's agnostic wire-level slots
under realtime.voice.default
and/or a matching realtime.voice.providers.<provider> bag) merged UNDER any caller-supplied
PrepareClientSessionInput.Config (the runtime bag wins per key). The settings objects
are OPAQUE driver pacts — each server driver consumes its own keys exactly as it consumes any
other entry of the open config bag (OpenAI spreads it into session.update, AssemblyAI reads
voice, Gemini merges it last). Returns the original input.Config (possibly undefined)
when the config contributes no voice and no session tuning.
The prepare-session input (carries the runtime config bag).
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration.
OptionaldriverClass: stringThe resolved vendor's DriverClass.
OptionalmodelID: stringThe resolved model id — keys the model-catalog ModelConfiguration cascade.
OptionalmodelVendorID: stringThe resolved model-vendor ROW id — the cascade's most-specific layer.
The merged config bag, or undefined when nothing contributes.
ProtectedbuildBuilds the RealtimeSessionParams for the client-direct session: the companion system prompt plus the registered realtime tool set.
The prepare-session input.
The resolved co-agent.
The vendor API name of the resolved realtime model.
The calling user.
The request-scoped metadata provider.
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration (voice persona + provider settings).
OptionaldriverClass: stringThe resolved vendor's DriverClass — matches per-provider voice settings.
OptionalmodelID: stringThe resolved MJ: AI Models id — keys the model-catalog ModelConfiguration cascade.
OptionalmodelVendorID: stringThe resolved MJ: AI Model Vendors ROW id — the cascade's most-specific layer.
OptionalresolvedAvatar: RealtimeAvatarResolutionThe session's avatar resolution, when the caller already made it; resolved here otherwise.
The assembled session params.
ProtectedbuildBuilds the tool set every voice session exposes: the core invoke-target-agent tool plus
any caller-supplied extra tools or projected direct actions. The target is a runtime
argument inside the delegation call, keeping the provider delegation contract identical.
OptionalextraTools: RealtimeToolDefinition[]Optional additional tools to register.
The tools to register at session start.
ProtectedbuildBuilds the RealtimeToolBroker for a relayed tool call, wiring DelegateToTarget to a
target-agent run and ExecuteTool to direct-action execution or a structured "not available" placeholder.
The relayed tool input.
The calling user.
The request-scoped metadata provider.
The constructed broker.
Use BuildWireActionMap.
Builds a wire-name to action map from candidate actions, sanitizing each action's name and deduplicating collisions (first action wins).
The action entities to index by wire name.
Map of wire-name to action entity.
Aborts in-flight relayed delegations for a session — the server half of the client-direct CANCEL channel (see the registry note on inFlightDelegations).
The session whose in-flight delegations to abort.
OptionalcallID: stringWhen supplied, only the delegation for this specific call is aborted; when omitted, EVERY in-flight delegation for the session is aborted.
The number of in-flight delegations aborted. Tolerant by design: an unknown
session, an unknown call id, or a session with nothing in flight returns 0 — never throws
(the call the user wanted dead may simply have finished already, which is a fine outcome).
ProtectedconfigureEnsures AIEngine metadata is loaded before resolution. Overridable seam so tests can skip the DB-backed config load.
The calling user.
The request-scoped metadata provider.
ProtectedcreateCreates the server-side co-agent observability runs for a voice session: an AIAgentRun
(Status Running), and — when a co-agent system prompt resolved — a linked AIPromptRun
(Status Running, AgentRunID = the co-agent run, AgentID = the co-agent) plus a single
MJ: AI Agent Run Steps row (StepType Prompt) so the co-agent run's Timeline is non-empty.
Delegated target-agent runs nest under the returned CoAgentRunID via ParentRunID.
Best-effort: returns null (and logs) when the co-agent run cannot be saved, so callers can
continue without observability rather than failing the whole prepare. A failed prompt-run or
run-step save just omits that id.
The resolved co-agent (its id stamps AgentID on both runs).
The co-agent system prompt id, or null to skip the prompt run + run step.
The resolved realtime model id (stamps the prompt run's ModelID).
Optional owning user id for the agent run.
The session id grouping this voice session's runs.
The calling user.
The request-scoped metadata provider.
OptionalconversationID: stringThe { CoAgentRunID, PromptRunID, CoAgentRunStepID } ids, or null when the agent run failed.
ProtectedcreateCreates artifact(s) from a completed delegated run's payload — the voice-path equivalent of
the chat path's artifact step in AgentRunner.RunAgentInConversation. Delegated voice runs
execute via AgentRunner.RunAgent directly (no conversation detail), so without this step
they would never produce artifacts at all.
Eligibility guards (all must hold, mirroring the chat path's processArtifacts):
The DB work is delegated to processRunArtifacts (an overridable seam), which reuses
AgentRunner.ProcessAgentArtifacts — so ArtifactCreationMode, DefaultArtifactTypeID,
name extraction, and duplicate-version dedup all behave exactly as in chat. Best-effort:
any failure is logged and returns undefined; artifact surfacing never fails the delegation.
The delegated agent execution result.
The calling user.
The request-scoped metadata provider.
The produced artifact descriptor(s), or undefined when none were created.
ProtectedcreateInstantiates the realtime driver for a vendor driver class via the ClassFactory. Overridable seam so tests can return a mock driver.
The vendor's DriverClass (the ClassFactory key).
The resolved API key (constructor argument).
The driver instance, or null when the factory cannot create one.
ProtecteddelegateDelegates an invoke-target-agent call to the target agent via AgentRunner.RunAgent.
Threads the broker-owned abort signal (combined with any caller signal) into the child run's
cancellationToken, links the child run to the co-agent run via parentRunID, and propagates
agentSessionID so both runs group under the same session. Mirrors
BaseAgent.delegateRealtimeToTarget.
The relayed tool input (target id + linkage).
The broker's delegation request (call id + arguments + abort signal).
The calling user.
The request-scoped metadata provider.
The delegated result for the model's tool_response.
ProtecteddescribeComma-joined display names of the session's allowed colleagues (labels preferred), for error guidance.
ProtecteddriverInspects whether the resolved realtime driver class supports dynamic tool registration.
Dynamic tool sets (e.g. per-agent direct action projection) are registered on the provider
socket only when the driver explicitly declares SupportsDynamicToolSet === true.
OptionaldriverClass: stringThe vendor's DriverClass (the ClassFactory key).
True if the driver supports dynamic tool sets, false otherwise.
ProtectedexecuteRoutes a non-target tool call by resolving the action through the wire map, checking the
direct-actions allowlist, and executing via ActionEngineServer.Instance.RunAction under a
configured timeout. If the tool is unrecognized, disallowed, or target resolution fails,
returns a structured "not available" result.
When this service holds the session's start-time projection, only the projected actions may run, whatever target the input names. When it holds none, the target agent's own metadata decides: another process prepared the session, the entry was evicted or lost on a restart, or the session was never started through this service (for example a row created through the entity API). That fallback skips the driver-capability gate but is still limited to the target's own direct-actions allowlist.
The non-target tool call.
Optionalinput: ExecuteRelayedToolInputThe optional relayed tool input context.
OptionalcontextUser: UserInfoThe calling user context.
A ToolExecutionResult for the model's tool_response.
Executes a single tool call relayed from the browser and returns its serialized result.
Builds a RealtimeToolBroker whose DelegateToTarget runs the target agent (threading
the abort signal, parent run, and session id) and whose ExecuteTool returns a structured
"not available" result for non-target tools (action wiring is a later phase). The broker
routes the call and always resolves with structured JSON — failures become tool_response
errors the model can narrate rather than thrown exceptions.
The relayed tool call plus delegation linkage.
The calling user (threaded into the delegated agent run).
The request-scoped metadata provider (threaded into the delegated agent run).
{ ResultJson, Success, PausedRunID?, Artifacts? } — the serialized tool result for
the browser to relay back, the paused run id when the delegated target agent paused awaiting
feedback (so the resolver can persist it and resume that run on the next answer), and the
artifacts the delegated run produced (so the resolver can junction-link them into the
session's conversation history — the same info is embedded in ResultJson for the client).
Finalizes the server-side co-agent observability records when a voice session ends. Loads
each (when its id is supplied) and, only if it is still Running, sets it to Completed
(or Failed when success is false) with a CompletedAt + Success stamp. Idempotent and
tolerant: a missing/already-finalized record is a no-op; a load/save failure is logged,
never thrown.
The co-agent run id, or null to skip.
The co-agent prompt run id, or null to skip.
The calling user.
The request-scoped metadata provider.
Whether the session ended successfully (controls Completed vs Failed).
The co-agent run's single MJ: AI Agent Run Steps row id, or null to skip.
OptionalerrorMessage: stringWhy the session failed (for example the bridge start's error), stamped as ErrorMessage on each
record it fails. Ignored when success is true. Without it the step gets a generic line and the runs none.
Finalizes the co-agent observability run(s) for an agent session that were left Running because
the session was reaped WITHOUT a live in-memory handle — a prior-boot orphan or a cross-host teardown,
where the Close()-wrapped finalizer never ran. This is the by-AgentSessionID analogue of
FinalizeCoAgentRun: the same-process path already knows its run ids (no query), but here that
state died with the prior process, so we locate the session's TOP-LEVEL co-agent run (delegated target
runs nest under it and finalize on their own runner) and finalize it + its prompt run + step via the
same idempotent helpers. A clean teardown already marked them Completed, so this finds nothing.
MJ: AI Agent Runs is a high-volume transactional table no engine caches, so a narrow ids-only query
is the right tool (not a cache reuse). Tolerant — never throws.
The agent session whose dangling co-agent runs to finalize.
Mark them Completed (true) or Failed (false).
The user the writes run as.
The request-scoped metadata provider.
The number of co-agent runs finalized (0 when none were dangling).
ProtectedfindLooks up a model by id in AIEngine's cached models. Overridable seam for tests.
The MJ: AI Models.ID to find.
The model entity, or null when not present.
ProtectedfindLooks up a model by ID (UUID-insensitive) or, failing that, by case/whitespace-insensitive Name in AIEngine's cached models. Overridable seam; tolerant of an unloaded cache.
The MJ: AI Models ID or Name.
The model entity, or null.
ProtectedgetReads the co-agent's TYPE-level DefaultConfiguration from AIEngine's cached agent
types. Overridable seam; tolerant — an absent type or unloaded cache returns null.
ProtectedgetResolves the API key for a vendor driver class. Overridable seam (wraps the module-level
GetAIAPIKey) so tests can simulate present/absent keys without environment setup.
Consulted for every driver class the session's run keys do not cover, so an override (a vault,
a different key, or undefined to disable a vendor) applies whether or not the environment
also holds a key.
The vendor's DriverClass.
The API key, or a falsy value when none is configured.
ProtectedgetReads the co-agent's own system prompt text from its highest-priority active agent prompt,
mirroring BaseAgent.loadAgentConfiguration's child-prompt resolution.
The resolved co-agent.
The co-agent's system prompt template text, or empty string when none is configured.
ProtectedgetResolves the active database actions assigned to the target agent. Overridable seam for tests to provide mock action sets without an engine cache.
The target agent ID.
The active action entities assigned to the agent.
Keeps a voice session's co-agent run alive as far as the AgentRunWatchdog is concerned. The
run spans the whole call, but no agent loop owns it, so nothing stamped its heartbeat: the watchdog
force-failed every call that ran past ~5 minutes ("no liveness heartbeat … owning process presumed
dead") while the call carried on. Called when the run is created and again on each persisted
session heartbeat (SessionManager), so whichever server instance the session is talking to keeps
it fresh; the watchdog drops it once it is finalized. Only a database provider can stamp heartbeats —
any other provider is a no-op, as for every agent run.
The session's co-agent run id (from its Config), or nothing.
The request-scoped metadata provider.
The user the heartbeat writes run as.
ProtectedloadReads and parses an application's AgentSettings (one tiny by-ID read, served from the provider's
RunView cache on repeat). The RAW column is parsed with ParseAgentSettings rather than read
through the CodeGen-generated AgentSettingsObject accessor, so a field added to the settings
interface works the moment the code reading it ships, before CodeGen next refreshes the copy.
Tolerant — a blank id, a failed read, a missing app, or an app with no usable settings returns null.
ProtectedloadLoads the co-agent run entity behind ExecuteRelayedToolInput.ParentRunID so the
delegated run can link to it via parentRun (→ ParentRunID). Returns null when no id was
supplied or the run cannot be loaded (delegation proceeds without parent linkage rather than
failing the whole call).
The co-agent run id, or undefined.
The calling user.
The request-scoped metadata provider.
The loaded parent run entity, or null.
ProtectedmapMaps an action entity and its metadata parameters to a RealtimeToolDefinition.
The action entity to map.
OptionalwireName: stringOptional pre-sanitized wire name. If omitted, derives via SanitizeWireToolName.
The constructed tool definition.
ProtectedmodelWhether the session model's effective catalog configuration (type < model < vendor < model-vendor)
declares Privacy.ZeroDataRetention: true. Fails closed: a model or vendor row that cannot be
resolved, or a catalog lookup that throws, reads as "not declared", so an agent that REQUIRES zero
data retention loses exposure rather than gaining it. Overridable seam.
The resolved MJ: AI Models id.
The resolved model-vendor row id (the cascade's most specific layer).
ProtectedparseParses JSON arguments string into a key-value record of parameters. Distinguishes "no arguments" (valid for zero-param action) from "unparseable JSON".
OptionalargumentsJson: stringThe raw arguments string emitted by the model.
The parsed arguments record.
Prepares a client-direct realtime session: resolves the model, assembles the companion system prompt + stable tool set, and mints the ClientRealtimeSessionConfig.
Returns a failure result (never throws) when no Realtime model/key resolves or the provider cannot mint a client-direct session.
The co-agent/target/session inputs.
The calling user (threaded to metadata + memory retrieval).
The request-scoped metadata provider.
The prep result (Success + ClientConfig/SessionParams, or Success: false + ErrorMessage).
The single source of truth for realtime session prep. Builds the RealtimeSessionParams
for a co-agent voicing a target: resolves the co-agent, the effective config via the full precedence
cascade (type-default < co-agent < target < runtime override), the realtime model, then assembles
the companion system prompt (first-person as the TARGET — this is what gives every host the right
identity), the stable tool set (always including invoke-target-agent), voice, and memory.
EVERY realtime host consumes this — native chat via PrepareClientSession → CreateClientSession,
and the server-bridged hosts (LiveKit, future Zoom/Teams) via StartSession. Hosts differ ONLY in how
they OPEN the session and their media transport; identity/precedence/prompt/tools live here, once. Do
NOT re-implement this in a host. See plans/realtime/realtime-core-host-convergence.md.
Pure-ish and side-effect-free (no session opened, no observability run created) — those are the
opener's concern. Never throws — returns Success: false on failure.
The co-agent/target/session inputs (the runtime override rides ConfigOverridesJson).
The calling user (threaded to metadata + memory retrieval).
The request-scoped metadata provider.
The prep result: Success + co-agent/resolution/effective-config/session-params, or Success: false.
ProtectedprocessThe DB-backed artifact-creation seam: runs AgentRunner.ProcessAgentArtifacts WITHOUT a
conversation detail (the voice path has none — the artifact + version are created and the
junction link is skipped), then loads the artifact header for its display name.
Artifacts whose Visibility resolved to System Only (the agent's ArtifactCreationMode) are
created but NOT surfaced to the overlay — matching how chat hides them from users.
Overridable seam so tests can exercise createDelegatedRunArtifacts' eligibility guards without a DB.
The delegated agent execution result (payload + agentRun).
The calling user.
The request-scoped metadata provider.
The produced artifact descriptor(s), or undefined.
ProtectedreadThe server's view of the channel registry (MJ: AI Agent Channels), read from the request
provider's cached AIEngineBase rows (already configured by the prepare). Overridable seam.
Tolerant: an unloaded cache yields an empty registry (every candidate then reads as having no
row), never a throw.
ProtectedresolveResolves the APP cascade layer JSON from Application.AgentSettings: the Realtime overrides
(persona/disclosure/model) plus RelevantAgents mapped to the union-accumulated allowed-agent
set, all translated into the canonical {"realtime":{…}} shape via BuildAppRealtimeOverridesJson.
One tiny by-ID read (served from the provider's RunView cache on repeat). Tolerant — any failure,
a missing app, or an app with no AgentSettings returns null (no app layer), never throws.
The app the session runs in, or absent/blank for no app layer.
The calling user (RunView scope).
The request-scoped metadata provider (reserved; RunView uses the default).
The canonical app-layer JSON string, or null.
ProtectedResolveFor a co-agent with video on: the first candidate that shows an avatar and supports client-direct sessions, in the default walk's order (highest PowerRank first; a model's Active vendor rows by Priority, each with a usable key). A candidate shows an avatar when its model's Video/Output row allows video or it has none, and its driver renders avatars for the vendor's API name on its endpoint (RealtimeModelShowsAvatar). Every keyed vendor of a model is tried, not only the first. Overridable seam.
The co-agent (reserved for future per-agent model preference).
Optionalresolve: AIAPIKeyResolverThe session's run-scoped key resolver (run keys only; see resolveVendorAndInstantiate).
The session's credential scope; see resolveVendorAndInstantiate.
The resolution, or null when no candidate shows an avatar.
ProtectedresolveResolves the co-agent from either the supplied entity or its id (from cached metadata).
The prepare-session input.
The co-agent entity, or null when neither form resolves.
ProtectedresolveResolves the co-agent's highest-priority active system prompt, returning both its template
text and its prompt id. The id is surfaced so PrepareClientSession can create a linked
co-agent AIPromptRun for observability. Mirrors BaseAgent.loadAgentConfiguration's
child-prompt resolution.
The resolved co-agent.
The prompt text + id, or { Text: '', PromptID: null } when none is configured.
ProtectedresolveResolves the effective config's realtime.modelPreference (an MJ: AI Models Name OR ID)
into a usable realtime model. TOLERANT by design — this is a METADATA preference, so any
failure (unknown model, inactive, wrong type, no vendor/key) logs a warning and returns
null, falling through to the default highest-PowerRank resolution. Contrast with the
explicit runtime choice (resolvePreferredRealtimeModel), which fails loud.
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration.
Optionalresolve: AIAPIKeyResolverThe session's run-scoped key resolver (run keys only; see resolveVendorAndInstantiate).
The session's credential scope; see resolveVendorAndInstantiate.
Whether the session will show an avatar; see resolveVendorAndInstantiate.
The resolution, or null when no preference is configured or it can't be satisfied.
ProtectedresolveResolves the primary-scope pair a returning visitor's memory is filed under, mirroring the
recap side (writeReturningVisitorRecap's resolveRecapScope):
(Conversation.LinkedEntityID, Conversation.LinkedRecordID)(the "MJ: Conversations" entity, Conversation.LastConversationID)undefined (no scoped memory)Best-effort: any failure (no conversation id, load failure, entity not found) resolves to
undefined so memory injection silently falls back to unscoped behavior.
the current session's conversation id, if any.
the request/session metadata provider (multi-provider-safe — never global Metadata).
the calling user.
the scope pair, or undefined when this isn't a returning visitor.
ProtectedresolveResolves which agent a delegation should run, given an optional colleague name the model passed.
The allowed union is an affordance/UX filter; CanRun on the resolved agent (inside the agent run)
remains the security boundary — both apply.
The colleague name the model named, or undefined.
The relayed tool input (lead id + allowed union).
{ Agent } on success, or { Error } describing why no agent resolved.
ProtectedresolveResolves the EFFECTIVE realtime configuration via the surface-agnostic precedence cascade:
agent-TYPE DefaultConfiguration (base) < co-agent TypeConfiguration < target agent
TypeConfiguration < (pre-authorized) runtime override — deep-merged per key and normalized.
The target layer is what makes a voiced agent (Sage, Marketing Agent, …) carry its own voice/model
regardless of host. Tolerant end-to-end: malformed layers contribute nothing and an unloaded metadata
cache yields no type defaults. See plans/realtime/realtime-core-host-convergence.md.
The resolved co-agent.
OptionaloverridesJson: stringThe pre-authorized runtime override layer, when present.
OptionaltargetAgent: MJAIAgentEntityExtendedThe TARGET agent being voiced, when distinct from the co-agent — contributes the per-voiced-agent layer (above the co-agent, below the runtime override). Omit when there is none.
OptionalappSettingsJson: stringThe normalized effective configuration (possibly empty, never null).
ProtectedresolveResolves the realtime model for a session, honoring an explicit user choice when present.
realtime.modelPreference (name or id): resolve it via
resolveConfiguredModelPreference. METADATA preferences degrade gracefully — an
unsatisfiable preference logs and FALLS THROUGH to the default (mirroring the co-agent
resolution chain's tolerant metadata steps), it never breaks calls.'room'), the first candidate that shows one (ResolveAvatarRealtimeModel); else the existing default via
resolveRealtimeModel (highest-PowerRank active Realtime model), with the generic
noModelMessage on failure. The two explicit choices above win over the video preference.On every path the chosen model's vendor follows whether the session will show an avatar (the video setting is on and the session could show one): among the model's keyed vendors, one that renders avatars (Vertex AI for Gemini 3.8 Live) only when it will, else one that renders none; see resolveVendorAndInstantiate.
The prepare-session input (carries the optional preferred model id).
The resolved co-agent (threaded to the default-resolution seam).
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration (carries modelPreference and the video setting).
The resolution outcome (resolution or failure reason).
ProtectedresolveResolves the DB-driven progress-narration instruction template: the Active MJ: AI Prompts
row named RealtimeClientSessionService.NarrationPromptName, read from
AIEngine's cached prompts. When the current name is absent, falls back to the
DEPRECATED RealtimeClientSessionService.LegacyNarrationPromptName (pre-rename seed)
with a deprecation log. Tolerant: returns null (never throws) when neither prompt is
present, the text is empty, or the engine cache is unavailable — clients fall back to their
built-in narration instruction text.
The template text (containing a {{ progressMessage }} placeholder), or null.
ProtectedresolveStrictly resolves an EXPLICITLY requested realtime model. Each precondition failure returns a clear, user-facing reason naming the model — there is NO fallback to another model, because the caller's user explicitly chose this one.
The MJ: AI Models.ID the user chose.
Optionalresolve: AIAPIKeyResolverThe session's run-scoped key resolver (run keys only; see resolveVendorAndInstantiate).
The session's credential scope; see resolveVendorAndInstantiate.
Whether the session will show an avatar; see resolveVendorAndInstantiate.
The resolution outcome (resolution or a specific failure reason).
ProtectedresolveResolves the Realtime model + vendor driver + API key, mirroring BaseAgent's server-bridged
resolution: highest-power active model of AIModelType Realtime; highest-priority active
vendor whose DriverClass has a resolvable API key; instantiated via the ClassFactory.
Overridable seam. Test subclasses override this to return a mock model so the service can
be exercised without provider SDKs or DB metadata. Returns null (never throws) when any
step can't be satisfied.
The co-agent being voiced (reserved for future per-agent model preference).
Optionalresolve: AIAPIKeyResolverThe session's run-scoped key resolver (run keys only; see resolveVendorAndInstantiate).
The session's credential scope; see resolveVendorAndInstantiate.
Whether the session will show an avatar; see resolveVendorAndInstantiate.
The resolved model + identifiers, or null.
ProtectedResolveThe live avatar this session asks for, if any: from the voiced agent's persona, or realtime.video.avatarId,
when the agent's video setting is on (see ResolveRealtimeAvatar). A call that picked an avatar (its runtime
override names realtime.video.avatarId) while that setting is off asks for none, with reason video-off. A model
whose MJ: AI Model Modalities
Video/Output row turns video off asks for none (reason endpoint, one log line); otherwise whether the session can
render it is the driver's call. A seam so tests can supply a resolution without the engine's persona metadata.
The prepare-session input (the voiced agent).
The co-agent.
The effective configuration.
The resolved model.
The resolved model-vendor row; its vendor scopes the persona bindings.
ProtectedresolveResolves the client-tool tiers the browser is told about at mint: the APP tier
(Application.AgentSettings.ClientTools resolved against the tool-definition catalog) and the
STATIC tier (the target agent's MJ: AI Agent Client Tools junction). Overridable seam; tolerant —
any failure yields empty tiers and never fails the mint.
ProtectedresolveResolves the target agent entity from cached metadata.
The target agent id.
The target agent entity, or null when not found.
ProtectedresolveShared tail of model resolution: picks the vendor (with a usable API key) for an already-chosen model entity and instantiates its realtime driver.
Key precedence, per driver class: resolve (the session's run keys), then this service's
getAPIKeyForDriver seam, whose default is the environment key. Vendor selection and the
mint use that one chain, so a session is never routed to a vendor it cannot then be minted on.
The chosen model entity.
Optionalresolve: AIAPIKeyResolverThe session's run-scoped key resolver. Expected to answer with run keys only: one that falls back to the environment itself answers before getAPIKeyForDriver and so bypasses an override of it.
A scope that rules out the 'Environment' source ('RuntimeOnly') drops the
getAPIKeyForDriver seam: the run's keys are the whole chain, so a vendor they do not key
is never selected on the platform's key.
Whether the session will show an avatar. The highest-Priority keyed vendor stands when it renders avatars (RealtimeModelShowsAvatar) exactly when the session will show one; otherwise the first other keyed vendor that does is taken, when there is one. So a vendor that renders avatars (Vertex AI for Gemini 3.8 Live, with its own billing and limits) takes a session only when it will show an avatar, or when it is the only keyed vendor; and a session that will show one goes to such a vendor whatever the Priority order.
The full resolution, or null when no vendor/key/driver can be satisfied.
ProtectedscopeApplies channel scoping and the client-tool tiers to a prepare input.
Application.AgentSettings.ClientTools is resolved to metadata
and layered, through the unified ResolveClientTools, beneath the live surface's own
manifest in the app-context snapshot — so the co-agent is told about tools the app declares
even when the active surface did not republish them. (Only the HOST can run them; the browser
reports "declared but not registered" for one it has no handler for.) The agent's static
junction tools are deliberately NOT added to the voice prompt — voice prompts are
token-sensitive and a tool with no browser handler is noise — but they ride the returned policy
so the browser can describe them.channels section, and the declared tools narrowed to match
(BuildSessionChannelPolicy).The (possibly adjusted) input, and the policy to hand back when candidates were reported.
ProtectedselectSelects the highest-priority active vendor for a model whose DriverClass has a resolvable
API key. Delegates to SelectRealtimeVendorForModel (the one copy of the rule), threading
THIS service's getAPIKeyForDriver so subclasses and tests keep their key-resolution seam.
The chosen model's id.
Optionalresolve: AIAPIKeyResolverThe complete key chain to select against; defaults to getAPIKeyForDriver.
The vendor driver/api identifiers, or null when none has a usable key.
Wires a server long-lived (bridged) realtime session onto the SAME core machinery the
client-direct path uses — so a LiveKit (or future Zoom/Teams) agent does real work and is tracked
identically, with zero host-local re-implementation. This is the Phase 2 counterpart to
PrepareClientSession: the browser relays tool calls back over GraphQL to ExecuteRelayedTool,
whereas here the server holds the live IRealtimeSession and we wire its OnToolCall directly to
the SAME ExecuteRelayedTool (so invoke-target-agent runs the target via AgentRunner, nests
under the co-agent run, supports barge-in cancel + paused-run resume — all of it, for free).
Responsibilities, in order:
0. Report the avatar status the driver had nothing to say about: a prep that asked the driver for no avatar (a
phone call, no face for the model's vendor, an unknown avatar, a Video/Output row that turns video off) leaves
the driver nothing to report, and a driver that renders no avatar ignores a request, so the session's
AvatarStatus says why instead of saying nothing (ResolveBridgedAvatarStatus).
session.OnUsage →
BridgeRealtimeUsageRecorder → AccumulatePromptRunUsage, every 10 s and once more at finalize.session.OnToolCall → ExecuteRelayedTool → session.SendToolResult.session.Close() and listen for an unexpected drop (OnClose), both
routed through one idempotent finalizer. The bridge teardown calls Close(), so the run finalizes
on graceful end; a dropped socket finalizes via OnClose, and a Close() after it waits for that
finalize. The wrapped Close() closes the model session first (waiting up to
BRIDGE_SESSION_CLOSE_WAIT_MS), so the usage it reports while closing is recorded; the
finalizer then writes the session's last usage before it finalizes the runs, so the prompt run is
priced from its final counts.The live realtime session the bridge owns (from model.StartSession).
The same prep input used to build the session (carries AgentSessionID, TargetAgentID, …).
The successful PrepareRealtimeSessionParams result (CoAgent + Resolution).
The calling user (threaded into observability + delegated runs).
The request-scoped metadata provider.
A BridgeRealtimeRuntime the bridge holds for the session lifetime.
Server-agnostic service that prepares a client-direct realtime session and executes the tool calls the browser relays back. Constructed per-request (a normal injectable service — NOT a singleton) so the UserInfo and IMetadataProvider are always request-scoped.
Every public method takes the
contextUserandproviderexplicitly — this service never reaches for the global default provider, so it is safe in multi-provider/multi-tenant servers.