Static ReadonlyLegacyDEPRECATED legacy name of the narration prompt, from before the co-agent's rename from "Voice Co-Agent" to "Realtime Co-Agent". Deployments that have not re-synced the prompt seed still carry this name, so resolveNarrationInstructionsTemplate falls back to it (with a deprecation log) when RealtimeClientSessionService.NarrationPromptName is absent.
Static ReadonlyNarrationThe seeded name of the MJ: AI Prompts row whose TemplateText carries the first-person
progress-narration instructions (with a {{ progressMessage }} placeholder). Resolved at
session prepare time so the browser narrates with DB-driven, product-tunable wording.
Canonical value lives in realtime-narration.ts (shared with the server-bridged runner path).
Accumulates relayed usage DELTAS onto the co-agent AIPromptRun's TokensPrompt / TokensCompletion
(recomputing TokensUsed). Serialized against AppendPromptRunMessage on the same run so the
high-frequency usage checkpoint never overwrites freshly-appended transcript turns (and vice-versa).
Best-effort: load/save failures log and return false, never throw.
The co-agent observability prompt run.
Input-token delta to add (caller clamps to >= 0).
Output-token delta to add (caller clamps to >= 0).
true when the accumulated usage was persisted.
Appends (or replaces) one transcript turn onto the co-agent's long-lived AIPromptRun.Messages,
so the realtime co-agent's conversation is captured on its run exactly like every other MJ agent
run — closing the observability gap where the run held only token totals, never the turns. The
run viewer can then show what the co-agent heard and said. Mirrors accumulatePromptRunUsage's
load/append/save pattern; best-effort and tolerant (logs, never throws).
replacePrevious swaps the last same-role message instead of appending — the streaming-correction
case (an interim assistant turn finalized into its full text). The stored shape is the standard
chat-message array ([{ role, content }, …]) the rest of MJ already reads from Messages.
NOTE: load-append-save carries the same benign race as usage accumulation; realtime turns are sequential per session so collisions are rare. A dedicated child turn-row entity would remove the race (and the blob rewrite) entirely — a future increment. Tool-call turns (the browser_ and Whiteboard_ channel tools) are a separate increment that requires the client to relay them.
true when the turn was persisted onto the prompt run.
ProtectedassembleAssembles the same memory/context block a loop agent injects, reusing AgentMemoryContextBuilder so there is no duplicated retrieval logic. The builder unshifts a system message onto a throwaway array, which we pull back out as plain text.
The prepare-session input.
The resolved co-agent.
The calling user.
The concatenated context text (empty string when nothing was injected).
ProtectedbuildBuilds the session-start app-context section from the AppContextSnapshot: where the user
is, what they see, and the live capability manifest, rendered via the shared
FormatAppContextNote (one wording for mint-time and streaming notes) under a clear heading.
Returns '' when no snapshot was supplied or nothing salient survives.
OptionalappContext: AppContextSnapshotThe app-context snapshot, or undefined.
The prompt section, or ''.
ProtectedbuildMaps the effective config's union-accumulated allowedAgents into RealtimeColleague
entries for the framing — resolving each target's display name + description from the agent cache
and its effective per-target disclosure (GetDisclosureForTarget). The LEAD (the voiced
target) is excluded so the co-agent never lists itself as a colleague. Empty when none configured.
The resolved effective configuration.
OptionalexcludeAgentId: stringThe lead/target agent id to exclude from the colleague list.
The colleague set (possibly empty).
ProtectedbuildAssembles the companion system prompt: the framing ("you are the voice for the target"), the co-agent's own system prompt text, the TARGET agent's identity/capabilities (Name + Description), the conversation history, and the same memory/context a loop agent assembles.
When the effective configuration carries a voice persona (realtime.voice.default), a
short "Voice & manner" section (tone / speaking style) is appended after the co-agent's
own prompt so the model speaks in the configured manner.
The prepare-session input.
The resolved co-agent.
The calling user.
The request-scoped metadata provider.
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration (voice persona source).
The concatenated system prompt (never empty — the framing is always present).
ProtectedbuildBuilds the "interactive-surface tools" exception clause appended to the co-agent framing when
the client supplied channel tools (browser_, Whiteboard_, …) as ExtraTools. Without it the
co-agent — told to route ALL work through invoke-target-agent — delegates browser/whiteboard
requests to the target agent (which has no live channel of its own) instead of driving the
surface itself, then hallucinates a "missing session id". The tools ARE already in its set
(buildStableToolSet merges [invokeTarget, ...extraTools]); this clause tells the model
to USE them directly. Returns empty for pure-voice sessions (no ExtraTools), keeping that
framing untouched. Generic by design — it names browser_ and Whiteboard_ tools only as
examples, so any future client channel is covered automatically.
OptionalextraTools: RealtimeToolDefinition[]The client-supplied channel tools, when any.
The exception clause (leading space included), or '' when there are no extra tools.
ProtectedbuildBuilds the meeting-mode discipline clause — present only for a multi-agent meeting session
(PrepareClientSessionInput.DisableAutoResponse). It tells the agent to hear the whole
conversation but speak only when addressed (named) or clearly called on, and never to talk over
others. This is the prompt half of "hear always, speak selectively"; the enforcement half is the
model's disabled auto-response + the bridge's addressing gate. Empty for a 1:1 call (prompt unchanged).
See plans/realtime/multi-agent-meeting-turn-taking.md.
The prepare-session input (carries the meeting flag + self names).
The meeting clause, or '' for a non-meeting session.
ProtectedbuildBuilds the provider-pact Config bag for the session: the effective config's matching
per-provider voice settings (realtime.voice.providers.<provider>) merged UNDER any
caller-supplied PrepareClientSessionInput.Config (the runtime bag wins per key).
The settings objects are OPAQUE driver pacts — each server driver consumes its own keys
exactly as it consumes any other entry of the open config bag (OpenAI spreads it into
session.update, AssemblyAI reads voice, Gemini merges it last). Returns the original
input.Config (possibly undefined) when no provider settings match, preserving the
pre-config behavior byte-for-byte.
The prepare-session input (carries the runtime config bag).
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration.
OptionaldriverClass: stringThe resolved vendor's DriverClass.
The merged config bag, or undefined when nothing contributes.
ProtectedbuildBuilds the RealtimeSessionParams for the client-direct session: the companion system prompt plus the stable, target-independent tool set.
The prepare-session input.
The resolved co-agent.
The vendor API name of the resolved realtime model.
The calling user.
The request-scoped metadata provider.
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration (voice persona + provider settings).
OptionaldriverClass: stringThe resolved vendor's DriverClass — matches per-provider voice settings.
The assembled session params.
ProtectedbuildBuilds the stable, target-independent tool set every voice session exposes: the single
invoke-target-agent tool plus any caller-supplied extra tools. The target is a runtime
argument inside the call, never a per-target tool — this keeps the provider contract
identical across targets.
OptionalextraTools: RealtimeToolDefinition[]Optional additional target-independent tools.
The tools to register at session start.
ProtectedbuildBuilds the RealtimeToolBroker for a relayed tool call, wiring DelegateToTarget to a
target-agent run and ExecuteTool to a structured "not available" placeholder.
The relayed tool input.
The calling user.
The request-scoped metadata provider.
The constructed broker.
Aborts in-flight relayed delegations for a session — the server half of the client-direct CANCEL channel (see the registry note on inFlightDelegations).
The session whose in-flight delegations to abort.
OptionalcallID: stringWhen supplied, only the delegation for this specific call is aborted; when omitted, EVERY in-flight delegation for the session is aborted.
The number of in-flight delegations aborted. Tolerant by design: an unknown
session, an unknown call id, or a session with nothing in flight returns 0 — never throws
(the call the user wanted dead may simply have finished already, which is a fine outcome).
ProtectedconfigureEnsures AIEngine metadata is loaded before resolution. Overridable seam so tests can skip the DB-backed config load.
The calling user.
The request-scoped metadata provider.
ProtectedcreateCreates the server-side co-agent observability runs for a voice session: an AIAgentRun
(Status Running), and — when a co-agent system prompt resolved — a linked AIPromptRun
(Status Running, AgentRunID = the co-agent run, AgentID = the co-agent) plus a single
MJ: AI Agent Run Steps row (StepType Prompt) so the co-agent run's Timeline is non-empty.
Delegated target-agent runs nest under the returned CoAgentRunID via ParentRunID.
Best-effort: returns null (and logs) when the co-agent run cannot be saved, so callers can
continue without observability rather than failing the whole prepare. A failed prompt-run or
run-step save just omits that id.
The resolved co-agent (its id stamps AgentID on both runs).
The co-agent system prompt id, or null to skip the prompt run + run step.
The resolved realtime model id (stamps the prompt run's ModelID).
Optional owning user id for the agent run.
The session id grouping this voice session's runs.
The calling user.
The request-scoped metadata provider.
OptionalconversationID: stringThe { CoAgentRunID, PromptRunID, CoAgentRunStepID } ids, or null when the agent run failed.
ProtectedcreateCreates artifact(s) from a completed delegated run's payload — the voice-path equivalent of
the chat path's artifact step in AgentRunner.RunAgentInConversation. Delegated voice runs
execute via AgentRunner.RunAgent directly (no conversation detail), so without this step
they would never produce artifacts at all.
Eligibility guards (all must hold, mirroring the chat path's processArtifacts):
The DB work is delegated to processRunArtifacts (an overridable seam), which reuses
AgentRunner.ProcessAgentArtifacts — so ArtifactCreationMode, DefaultArtifactTypeID,
name extraction, and duplicate-version dedup all behave exactly as in chat. Best-effort:
any failure is logged and returns undefined; artifact surfacing never fails the delegation.
The delegated agent execution result.
The calling user.
The request-scoped metadata provider.
The produced artifact descriptor(s), or undefined when none were created.
ProtectedcreateInstantiates the realtime driver for a vendor driver class via the ClassFactory. Overridable seam so tests can return a mock driver.
The vendor's DriverClass (the ClassFactory key).
The resolved API key (constructor argument).
The driver instance, or null when the factory cannot create one.
ProtecteddelegateDelegates an invoke-target-agent call to the target agent via AgentRunner.RunAgent.
Threads the broker-owned abort signal (combined with any caller signal) into the child run's
cancellationToken, links the child run to the co-agent run via parentRunID, and propagates
agentSessionID so both runs group under the same session. Mirrors
BaseAgent.delegateRealtimeToTarget.
The relayed tool input (target id + linkage).
The broker's delegation request (call id + arguments + abort signal).
The calling user.
The request-scoped metadata provider.
The delegated result for the model's tool_response.
ProtecteddescribeComma-joined display names of the session's allowed colleagues (labels preferred), for error guidance.
ProtectedexecuteRoutes a non-target tool call. For now this returns a structured "not available" result — the richer client/UI/action routing is wired in a later phase. Documented minimal seam.
The non-target tool call.
A failed ToolExecutionResult the model can narrate.
Executes a single tool call relayed from the browser and returns its serialized result.
Builds a RealtimeToolBroker whose DelegateToTarget runs the target agent (threading
the abort signal, parent run, and session id) and whose ExecuteTool returns a structured
"not available" result for non-target tools (action wiring is a later phase). The broker
routes the call and always resolves with structured JSON — failures become tool_response
errors the model can narrate rather than thrown exceptions.
The relayed tool call plus delegation linkage.
The calling user (threaded into the delegated agent run).
The request-scoped metadata provider (threaded into the delegated agent run).
{ ResultJson, Success, PausedRunID?, Artifacts? } — the serialized tool result for
the browser to relay back, the paused run id when the delegated target agent paused awaiting
feedback (so the resolver can persist it and resume that run on the next answer), and the
artifacts the delegated run produced (so the resolver can junction-link them into the
session's conversation history — the same info is embedded in ResultJson for the client).
Finalizes the server-side co-agent observability records when a voice session ends. Loads
each (when its id is supplied) and, only if it is still Running, sets it to Completed
(or Failed when success is false) with a CompletedAt + Success stamp. Idempotent and
tolerant: a missing/already-finalized record is a no-op; a load/save failure is logged,
never thrown.
The co-agent run id, or null to skip.
The co-agent prompt run id, or null to skip.
The calling user.
The request-scoped metadata provider.
Whether the session ended successfully (controls Completed vs Failed).
The co-agent run's single MJ: AI Agent Run Steps row id, or null to skip.
Finalizes the co-agent observability run(s) for an agent session that were left Running because
the session was reaped WITHOUT a live in-memory handle — a prior-boot orphan or a cross-host teardown,
where the Close()-wrapped finalizer never ran. This is the by-AgentSessionID analogue of
FinalizeCoAgentRun: the same-process path already knows its run ids (no query), but here that
state died with the prior process, so we locate the session's TOP-LEVEL co-agent run (delegated target
runs nest under it and finalize on their own runner) and finalize it + its prompt run + step via the
same idempotent helpers. A clean teardown already marked them Completed, so this finds nothing.
MJ: AI Agent Runs is a high-volume transactional table no engine caches, so a narrow ids-only query
is the right tool (not a cache reuse). Tolerant — never throws.
The agent session whose dangling co-agent runs to finalize.
Mark them Completed (true) or Failed (false).
The user the writes run as.
The request-scoped metadata provider.
The number of co-agent runs finalized (0 when none were dangling).
ProtectedfindLooks up a model by id in AIEngine's cached models. Overridable seam for tests.
The MJ: AI Models.ID to find.
The model entity, or null when not present.
ProtectedfindLooks up a model by ID (UUID-insensitive) or, failing that, by case/whitespace-insensitive Name in AIEngine's cached models. Overridable seam; tolerant of an unloaded cache.
The MJ: AI Models ID or Name.
The model entity, or null.
ProtectedgetReads the co-agent's TYPE-level DefaultConfiguration from AIEngine's cached agent
types. Overridable seam; tolerant — an absent type or unloaded cache returns null.
ProtectedgetResolves the API key for a vendor driver class. Overridable seam (wraps the module-level GetAIAPIKey) so tests can simulate present/absent keys without environment setup.
The vendor's DriverClass.
The API key, or a falsy value when none is configured.
ProtectedgetReads the co-agent's own system prompt text from its highest-priority active agent prompt,
mirroring BaseAgent.loadAgentConfiguration's child-prompt resolution.
The resolved co-agent.
The co-agent's system prompt template text, or empty string when none is configured.
ProtectedloadLoads the co-agent run entity behind ExecuteRelayedToolInput.ParentRunID so the
delegated run can link to it via parentRun (→ ParentRunID). Returns null when no id was
supplied or the run cannot be loaded (delegation proceeds without parent linkage rather than
failing the whole call).
The co-agent run id, or undefined.
The calling user.
The request-scoped metadata provider.
The loaded parent run entity, or null.
Prepares a client-direct realtime session: resolves the model, assembles the companion system prompt + stable tool set, and mints the ClientRealtimeSessionConfig.
Returns a failure result (never throws) when no Realtime model/key resolves or the provider cannot mint a client-direct session.
The co-agent/target/session inputs.
The calling user (threaded to metadata + memory retrieval).
The request-scoped metadata provider.
The prep result (Success + ClientConfig/SessionParams, or Success: false + ErrorMessage).
The single source of truth for realtime session prep. Builds the RealtimeSessionParams
for a co-agent voicing a target: resolves the co-agent, the effective config via the full precedence
cascade (type-default < co-agent < target < runtime override), the realtime model, then assembles
the companion system prompt (first-person as the TARGET — this is what gives every host the right
identity), the stable tool set (always including invoke-target-agent), voice, and memory.
EVERY realtime host consumes this — native chat via PrepareClientSession → CreateClientSession,
and the server-bridged hosts (LiveKit, future Zoom/Teams) via StartSession. Hosts differ ONLY in how
they OPEN the session and their media transport; identity/precedence/prompt/tools live here, once. Do
NOT re-implement this in a host. See plans/realtime/realtime-core-host-convergence.md.
Pure-ish and side-effect-free (no session opened, no observability run created) — those are the
opener's concern. Never throws — returns Success: false on failure.
The co-agent/target/session inputs (the runtime override rides ConfigOverridesJson).
The calling user (threaded to metadata + memory retrieval).
The request-scoped metadata provider.
The prep result: Success + co-agent/resolution/effective-config/session-params, or Success: false.
ProtectedprocessThe DB-backed artifact-creation seam: runs AgentRunner.ProcessAgentArtifacts WITHOUT a
conversation detail (the voice path has none — the artifact + version are created and the
junction link is skipped), then loads the artifact header for its display name.
Artifacts whose Visibility resolved to System Only (the agent's ArtifactCreationMode) are
created but NOT surfaced to the overlay — matching how chat hides them from users.
Overridable seam so tests can exercise createDelegatedRunArtifacts' eligibility guards without a DB.
The delegated agent execution result (payload + agentRun).
The calling user.
The request-scoped metadata provider.
The produced artifact descriptor(s), or undefined.
ProtectedresolveResolves the APP cascade layer JSON from Application.AgentSettings: the Realtime overrides
(persona/disclosure/model) plus RelevantAgents mapped to the union-accumulated allowed-agent
set, all translated into the canonical {"realtime":{…}} shape via BuildAppRealtimeOverridesJson.
One tiny by-ID read (served from the provider's RunView cache on repeat). Tolerant — any failure,
a missing app, or an app with no AgentSettings returns null (no app layer), never throws.
The app the session runs in, or absent/blank for no app layer.
The calling user (RunView scope).
The request-scoped metadata provider (reserved; RunView uses the default).
The canonical app-layer JSON string, or null.
ProtectedresolveResolves the co-agent from either the supplied entity or its id (from cached metadata).
The prepare-session input.
The co-agent entity, or null when neither form resolves.
ProtectedresolveResolves the co-agent's highest-priority active system prompt, returning both its template
text and its prompt id. The id is surfaced so PrepareClientSession can create a linked
co-agent AIPromptRun for observability. Mirrors BaseAgent.loadAgentConfiguration's
child-prompt resolution.
The resolved co-agent.
The prompt text + id, or { Text: '', PromptID: null } when none is configured.
ProtectedresolveResolves the effective config's realtime.modelPreference (an MJ: AI Models Name OR ID)
into a usable realtime model. TOLERANT by design — this is a METADATA preference, so any
failure (unknown model, inactive, wrong type, no vendor/key) logs a warning and returns
null, falling through to the default highest-PowerRank resolution. Contrast with the
explicit runtime choice (resolvePreferredRealtimeModel), which fails loud.
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration.
The resolution, or null when no preference is configured or it can't be satisfied.
ProtectedresolveResolves the primary-scope pair a returning visitor's memory is filed under, mirroring the
recap side (writeReturningVisitorRecap's resolveRecapScope):
(Conversation.LinkedEntityID, Conversation.LinkedRecordID)(the "MJ: Conversations" entity, Conversation.LastConversationID)undefined (no scoped memory)Best-effort: any failure (no conversation id, load failure, entity not found) resolves to
undefined so memory injection silently falls back to unscoped behavior.
the current session's conversation id, if any.
the request/session metadata provider (multi-provider-safe — never global Metadata).
the calling user.
the scope pair, or undefined when this isn't a returning visitor.
ProtectedresolveResolves which agent a delegation should run, given an optional colleague name the model passed.
The allowed union is an affordance/UX filter; CanRun on the resolved agent (inside the agent run)
remains the security boundary — both apply.
The colleague name the model named, or undefined.
The relayed tool input (lead id + allowed union).
{ Agent } on success, or { Error } describing why no agent resolved.
ProtectedresolveResolves the EFFECTIVE realtime configuration via the surface-agnostic precedence cascade:
agent-TYPE DefaultConfiguration (base) < co-agent TypeConfiguration < target agent
TypeConfiguration < (pre-authorized) runtime override — deep-merged per key and normalized.
The target layer is what makes a voiced agent (Sage, Marketing Agent, …) carry its own voice/model
regardless of host. Tolerant end-to-end: malformed layers contribute nothing and an unloaded metadata
cache yields no type defaults. See plans/realtime/realtime-core-host-convergence.md.
The resolved co-agent.
OptionaloverridesJson: stringThe pre-authorized runtime override layer, when present.
OptionaltargetAgent: MJAIAgentEntityExtendedThe TARGET agent being voiced, when distinct from the co-agent — contributes the per-voiced-agent layer (above the co-agent, below the runtime override). Omit when there is none.
OptionalappSettingsJson: stringThe normalized effective configuration (possibly empty, never null).
ProtectedresolveResolves the realtime model for a session, honoring an explicit user choice when present.
realtime.modelPreference (name or id): resolve it via
resolveConfiguredModelPreference. METADATA preferences degrade gracefully — an
unsatisfiable preference logs and FALLS THROUGH to the default (mirroring the co-agent
resolution chain's tolerant metadata steps), it never breaks calls.The prepare-session input (carries the optional preferred model id).
The resolved co-agent (threaded to the default-resolution seam).
OptionaleffectiveConfig: RealtimeCoAgentConfigThe resolved effective configuration (carries modelPreference).
The resolution outcome (resolution or failure reason).
ProtectedresolveResolves the DB-driven progress-narration instruction template: the Active MJ: AI Prompts
row named RealtimeClientSessionService.NarrationPromptName, read from
AIEngine's cached prompts. When the current name is absent, falls back to the
DEPRECATED RealtimeClientSessionService.LegacyNarrationPromptName (pre-rename seed)
with a deprecation log. Tolerant: returns null (never throws) when neither prompt is
present, the text is empty, or the engine cache is unavailable — clients fall back to their
built-in narration instruction text.
The template text (containing a {{ progressMessage }} placeholder), or null.
ProtectedresolveStrictly resolves an EXPLICITLY requested realtime model. Each precondition failure returns a clear, user-facing reason naming the model — there is NO fallback to another model, because the caller's user explicitly chose this one.
The MJ: AI Models.ID the user chose.
The resolution outcome (resolution or a specific failure reason).
ProtectedresolveResolves the Realtime model + vendor driver + API key, mirroring BaseAgent's server-bridged
resolution: highest-power active model of AIModelType Realtime; highest-priority active
vendor whose DriverClass has a resolvable API key; instantiated via the ClassFactory.
Overridable seam. Test subclasses override this to return a mock model so the service can
be exercised without provider SDKs or DB metadata. Returns null (never throws) when any
step can't be satisfied.
The co-agent being voiced (reserved for future per-agent model preference).
The resolved model + identifiers, or null.
ProtectedresolveResolves the target agent entity from cached metadata.
The target agent id.
The target agent entity, or null when not found.
ProtectedresolveShared tail of model resolution: picks the vendor (with a usable API key) for an already-chosen model entity and instantiates its realtime driver.
The chosen model entity.
The full resolution, or null when no vendor/key/driver can be satisfied.
ProtectedselectSelects the highest-priority active vendor for a model whose DriverClass has a resolvable
API key. Mirrors BaseAgent.selectRealtimeVendor.
The chosen model's id.
The vendor driver/api identifiers, or null when none has a usable key.
Wires a server long-lived (bridged) realtime session onto the SAME core machinery the
client-direct path uses — so a LiveKit (or future Zoom/Teams) agent does real work and is tracked
identically, with zero host-local re-implementation. This is the Phase 2 counterpart to
PrepareClientSession: the browser relays tool calls back over GraphQL to ExecuteRelayedTool,
whereas here the server holds the live IRealtimeSession and we wire its OnToolCall directly to
the SAME ExecuteRelayedTool (so invoke-target-agent runs the target via AgentRunner, nests
under the co-agent run, supports barge-in cancel + paused-run resume — all of it, for free).
Responsibilities, in order:
session.OnToolCall → ExecuteRelayedTool → session.SendToolResult.session.Close() and listen for an unexpected drop (OnClose), both
routed through one idempotent finalizer. The bridge teardown calls Close(), so the run finalizes
on graceful end; a dropped socket finalizes via OnClose.The live realtime session the bridge owns (from model.StartSession).
The same prep input used to build the session (carries AgentSessionID, TargetAgentID, …).
The successful PrepareRealtimeSessionParams result (CoAgent + Resolution).
The calling user (threaded into observability + delegated runs).
The request-scoped metadata provider.
A BridgeRealtimeRuntime the bridge holds for the session lifetime.
Server-agnostic service that prepares a client-direct realtime session and executes the tool calls the browser relays back. Constructed per-request (a normal injectable service — NOT a singleton) so the UserInfo and IMetadataProvider are always request-scoped.
Every public method takes the
contextUserandproviderexplicitly — this service never reaches for the global default provider, so it is safe in multi-provider/multi-tenant servers.