Member Junction
    Preparing search index...

    Server-agnostic service that prepares a client-direct realtime session and executes the tool calls the browser relays back. Constructed per-request (a normal injectable service — NOT a singleton) so the UserInfo and IMetadataProvider are always request-scoped.

    Every public method takes the contextUser and provider explicitly — this service never reaches for the global default provider, so it is safe in multi-provider/multi-tenant servers.

    Index

    Constructors

    Properties

    LegacyNarrationPromptName: "Voice Co-Agent - Progress Narration" = LEGACY_NARRATION_PROMPT_NAME

    DEPRECATED legacy name of the narration prompt, from before the co-agent's rename from "Voice Co-Agent" to "Realtime Co-Agent". Deployments that have not re-synced the prompt seed still carry this name, so resolveNarrationInstructionsTemplate falls back to it (with a deprecation log) when RealtimeClientSessionService.NarrationPromptName is absent.

    NarrationPromptName: "Realtime Co-Agent - Progress Narration" = NARRATION_PROMPT_NAME

    The seeded name of the MJ: AI Prompts row whose TemplateText carries the first-person progress-narration instructions (with a {{ progressMessage }} placeholder). Resolved at session prepare time so the browser narrates with DB-driven, product-tunable wording. Canonical value lives in realtime-narration.ts (shared with the server-bridged runner path).

    Methods

    • Accumulates relayed usage DELTAS onto the co-agent AIPromptRun's TokensPrompt / TokensCompletion (recomputing TokensUsed). Serialized against AppendPromptRunMessage on the same run so the high-frequency usage checkpoint never overwrites freshly-appended transcript turns (and vice-versa). Best-effort: load/save failures log and return false, never throw.

      Parameters

      • promptRunID: string

        The co-agent observability prompt run.

      • inputDelta: number

        Input-token delta to add (caller clamps to >= 0).

      • outputDelta: number

        Output-token delta to add (caller clamps to >= 0).

      • contextUser: UserInfo
      • provider: IMetadataProvider

      Returns Promise<boolean>

      true when the accumulated usage was persisted.

    • Appends (or replaces) one transcript turn onto the co-agent's long-lived AIPromptRun.Messages, so the realtime co-agent's conversation is captured on its run exactly like every other MJ agent run — closing the observability gap where the run held only token totals, never the turns. The run viewer can then show what the co-agent heard and said. Mirrors accumulatePromptRunUsage's load/append/save pattern; best-effort and tolerant (logs, never throws).

      replacePrevious swaps the last same-role message instead of appending — the streaming-correction case (an interim assistant turn finalized into its full text). The stored shape is the standard chat-message array ([{ role, content }, …]) the rest of MJ already reads from Messages.

      NOTE: load-append-save carries the same benign race as usage accumulation; realtime turns are sequential per session so collisions are rare. A dedicated child turn-row entity would remove the race (and the blob rewrite) entirely — a future increment. Tool-call turns (the browser_ and Whiteboard_ channel tools) are a separate increment that requires the client to relay them.

      Parameters

      • promptRunID: string
      • role: "system" | "user" | "assistant"
      • content: string
      • replacePrevious: boolean
      • contextUser: UserInfo
      • provider: IMetadataProvider

      Returns Promise<boolean>

      true when the turn was persisted onto the prompt run.

    • Maps the effective config's union-accumulated allowedAgents into RealtimeColleague entries for the framing — resolving each target's display name + description from the agent cache and its effective per-target disclosure (GetDisclosureForTarget). The LEAD (the voiced target) is excluded so the co-agent never lists itself as a colleague. Empty when none configured.

      Parameters

      • effectiveConfig: RealtimeCoAgentConfig

        The resolved effective configuration.

      • OptionalexcludeAgentId: string

        The lead/target agent id to exclude from the colleague list.

      Returns RealtimeColleague[]

      The colleague set (possibly empty).

    • Assembles the companion system prompt: the framing ("you are the voice for the target"), the co-agent's own system prompt text, the TARGET agent's identity/capabilities (Name + Description), the conversation history, and the same memory/context a loop agent assembles.

      When the effective configuration carries a voice persona (realtime.voice.default), a short "Voice & manner" section (tone / speaking style) is appended after the co-agent's own prompt so the model speaks in the configured manner.

      Parameters

      Returns Promise<string>

      The concatenated system prompt (never empty — the framing is always present).

    • Builds the "interactive-surface tools" exception clause appended to the co-agent framing when the client supplied channel tools (browser_, Whiteboard_, …) as ExtraTools. Without it the co-agent — told to route ALL work through invoke-target-agent — delegates browser/whiteboard requests to the target agent (which has no live channel of its own) instead of driving the surface itself, then hallucinates a "missing session id". The tools ARE already in its set (buildStableToolSet merges [invokeTarget, ...extraTools]); this clause tells the model to USE them directly. Returns empty for pure-voice sessions (no ExtraTools), keeping that framing untouched. Generic by design — it names browser_ and Whiteboard_ tools only as examples, so any future client channel is covered automatically.

      Parameters

      Returns string

      The exception clause (leading space included), or '' when there are no extra tools.

    • Builds the meeting-mode discipline clause — present only for a multi-agent meeting session (PrepareClientSessionInput.DisableAutoResponse). It tells the agent to hear the whole conversation but speak only when addressed (named) or clearly called on, and never to talk over others. This is the prompt half of "hear always, speak selectively"; the enforcement half is the model's disabled auto-response + the bridge's addressing gate. Empty for a 1:1 call (prompt unchanged). See plans/realtime/multi-agent-meeting-turn-taking.md.

      Parameters

      Returns string

      The meeting clause, or '' for a non-meeting session.

    • Builds the provider-pact Config bag for the session: the effective config's matching per-provider voice settings (realtime.voice.providers.<provider>) merged UNDER any caller-supplied PrepareClientSessionInput.Config (the runtime bag wins per key). The settings objects are OPAQUE driver pacts — each server driver consumes its own keys exactly as it consumes any other entry of the open config bag (OpenAI spreads it into session.update, AssemblyAI reads voice, Gemini merges it last). Returns the original input.Config (possibly undefined) when no provider settings match, preserving the pre-config behavior byte-for-byte.

      Parameters

      • input: PrepareClientSessionInput

        The prepare-session input (carries the runtime config bag).

      • OptionaleffectiveConfig: RealtimeCoAgentConfig

        The resolved effective configuration.

      • OptionaldriverClass: string

        The resolved vendor's DriverClass.

      Returns JSONObject

      The merged config bag, or undefined when nothing contributes.

    • Aborts in-flight relayed delegations for a session — the server half of the client-direct CANCEL channel (see the registry note on inFlightDelegations).

      Parameters

      • agentSessionID: string

        The session whose in-flight delegations to abort.

      • OptionalcallID: string

        When supplied, only the delegation for this specific call is aborted; when omitted, EVERY in-flight delegation for the session is aborted.

      Returns number

      The number of in-flight delegations aborted. Tolerant by design: an unknown session, an unknown call id, or a session with nothing in flight returns 0 — never throws (the call the user wanted dead may simply have finished already, which is a fine outcome).

    • Creates the server-side co-agent observability runs for a voice session: an AIAgentRun (Status Running), and — when a co-agent system prompt resolved — a linked AIPromptRun (Status Running, AgentRunID = the co-agent run, AgentID = the co-agent) plus a single MJ: AI Agent Run Steps row (StepType Prompt) so the co-agent run's Timeline is non-empty. Delegated target-agent runs nest under the returned CoAgentRunID via ParentRunID.

      Best-effort: returns null (and logs) when the co-agent run cannot be saved, so callers can continue without observability rather than failing the whole prepare. A failed prompt-run or run-step save just omits that id.

      Parameters

      • coAgent: MJAIAgentEntityExtended

        The resolved co-agent (its id stamps AgentID on both runs).

      • promptID: string

        The co-agent system prompt id, or null to skip the prompt run + run step.

      • modelID: string

        The resolved realtime model id (stamps the prompt run's ModelID).

      • vendorID: string
      • userID: string

        Optional owning user id for the agent run.

      • agentSessionID: string

        The session id grouping this voice session's runs.

      • contextUser: UserInfo

        The calling user.

      • provider: IMetadataProvider

        The request-scoped metadata provider.

      • OptionalconversationID: string

      Returns Promise<
          {
              CoAgentRunID: string;
              CoAgentRunStepID?: string;
              PromptRunID?: string;
          },
      >

      The { CoAgentRunID, PromptRunID, CoAgentRunStepID } ids, or null when the agent run failed.

    • Creates artifact(s) from a completed delegated run's payload — the voice-path equivalent of the chat path's artifact step in AgentRunner.RunAgentInConversation. Delegated voice runs execute via AgentRunner.RunAgent directly (no conversation detail), so without this step they would never produce artifacts at all.

      Eligibility guards (all must hold, mirroring the chat path's processArtifacts):

      • the run succeeded and did NOT pause awaiting feedback (a paused run has no deliverable yet);
      • the run returned a non-empty payload.

      The DB work is delegated to processRunArtifacts (an overridable seam), which reuses AgentRunner.ProcessAgentArtifacts — so ArtifactCreationMode, DefaultArtifactTypeID, name extraction, and duplicate-version dedup all behave exactly as in chat. Best-effort: any failure is logged and returns undefined; artifact surfacing never fails the delegation.

      Parameters

      Returns Promise<DelegatedRunArtifact[]>

      The produced artifact descriptor(s), or undefined when none were created.

    • Instantiates the realtime driver for a vendor driver class via the ClassFactory. Overridable seam so tests can return a mock driver.

      Parameters

      • driverClass: string

        The vendor's DriverClass (the ClassFactory key).

      • apiKey: string

        The resolved API key (constructor argument).

      Returns BaseRealtimeModel

      The driver instance, or null when the factory cannot create one.

    • Executes a single tool call relayed from the browser and returns its serialized result.

      Builds a RealtimeToolBroker whose DelegateToTarget runs the target agent (threading the abort signal, parent run, and session id) and whose ExecuteTool returns a structured "not available" result for non-target tools (action wiring is a later phase). The broker routes the call and always resolves with structured JSON — failures become tool_response errors the model can narrate rather than thrown exceptions.

      Parameters

      • input: ExecuteRelayedToolInput

        The relayed tool call plus delegation linkage.

      • contextUser: UserInfo

        The calling user (threaded into the delegated agent run).

      • provider: IMetadataProvider

        The request-scoped metadata provider (threaded into the delegated agent run).

      Returns Promise<
          {
              Artifacts?: DelegatedRunArtifact[];
              PausedRunID?: string;
              ResultJson: string;
              Success: boolean;
          },
      >

      { ResultJson, Success, PausedRunID?, Artifacts? } — the serialized tool result for the browser to relay back, the paused run id when the delegated target agent paused awaiting feedback (so the resolver can persist it and resume that run on the next answer), and the artifacts the delegated run produced (so the resolver can junction-link them into the session's conversation history — the same info is embedded in ResultJson for the client).

    • Finalizes the server-side co-agent observability records when a voice session ends. Loads each (when its id is supplied) and, only if it is still Running, sets it to Completed (or Failed when success is false) with a CompletedAt + Success stamp. Idempotent and tolerant: a missing/already-finalized record is a no-op; a load/save failure is logged, never thrown.

      Parameters

      • coAgentRunID: string

        The co-agent run id, or null to skip.

      • promptRunID: string

        The co-agent prompt run id, or null to skip.

      • contextUser: UserInfo

        The calling user.

      • provider: IMetadataProvider

        The request-scoped metadata provider.

      • success: boolean = true

        Whether the session ended successfully (controls Completed vs Failed).

      • coAgentRunStepID: string = null

        The co-agent run's single MJ: AI Agent Run Steps row id, or null to skip.

      Returns Promise<void>

    • Finalizes the co-agent observability run(s) for an agent session that were left Running because the session was reaped WITHOUT a live in-memory handle — a prior-boot orphan or a cross-host teardown, where the Close()-wrapped finalizer never ran. This is the by-AgentSessionID analogue of FinalizeCoAgentRun: the same-process path already knows its run ids (no query), but here that state died with the prior process, so we locate the session's TOP-LEVEL co-agent run (delegated target runs nest under it and finalize on their own runner) and finalize it + its prompt run + step via the same idempotent helpers. A clean teardown already marked them Completed, so this finds nothing.

      MJ: AI Agent Runs is a high-volume transactional table no engine caches, so a narrow ids-only query is the right tool (not a cache reuse). Tolerant — never throws.

      Parameters

      • agentSessionID: string

        The agent session whose dangling co-agent runs to finalize.

      • success: boolean

        Mark them Completed (true) or Failed (false).

      • contextUser: UserInfo

        The user the writes run as.

      • provider: IMetadataProvider

        The request-scoped metadata provider.

      Returns Promise<number>

      The number of co-agent runs finalized (0 when none were dangling).

    • Resolves the API key for a vendor driver class. Overridable seam (wraps the module-level GetAIAPIKey) so tests can simulate present/absent keys without environment setup.

      Parameters

      • driverClass: string

        The vendor's DriverClass.

      Returns string

      The API key, or a falsy value when none is configured.

    • The single source of truth for realtime session prep. Builds the RealtimeSessionParams for a co-agent voicing a target: resolves the co-agent, the effective config via the full precedence cascade (type-default < co-agent < target < runtime override), the realtime model, then assembles the companion system prompt (first-person as the TARGET — this is what gives every host the right identity), the stable tool set (always including invoke-target-agent), voice, and memory.

      EVERY realtime host consumes this — native chat via PrepareClientSessionCreateClientSession, and the server-bridged hosts (LiveKit, future Zoom/Teams) via StartSession. Hosts differ ONLY in how they OPEN the session and their media transport; identity/precedence/prompt/tools live here, once. Do NOT re-implement this in a host. See plans/realtime/realtime-core-host-convergence.md.

      Pure-ish and side-effect-free (no session opened, no observability run created) — those are the opener's concern. Never throws — returns Success: false on failure.

      Parameters

      • input: PrepareClientSessionInput

        The co-agent/target/session inputs (the runtime override rides ConfigOverridesJson).

      • contextUser: UserInfo

        The calling user (threaded to metadata + memory retrieval).

      • provider: IMetadataProvider

        The request-scoped metadata provider.

      Returns Promise<RealtimeSessionParamsPrep>

      The prep result: Success + co-agent/resolution/effective-config/session-params, or Success: false.

    • Resolves the APP cascade layer JSON from Application.AgentSettings: the Realtime overrides (persona/disclosure/model) plus RelevantAgents mapped to the union-accumulated allowed-agent set, all translated into the canonical {"realtime":{…}} shape via BuildAppRealtimeOverridesJson.

      One tiny by-ID read (served from the provider's RunView cache on repeat). Tolerant — any failure, a missing app, or an app with no AgentSettings returns null (no app layer), never throws.

      Parameters

      • applicationId: string

        The app the session runs in, or absent/blank for no app layer.

      • contextUser: UserInfo

        The calling user (RunView scope).

      • provider: IMetadataProvider

        The request-scoped metadata provider (reserved; RunView uses the default).

      Returns Promise<string>

      The canonical app-layer JSON string, or null.

    • Resolves the primary-scope pair a returning visitor's memory is filed under, mirroring the recap side (writeReturningVisitorRecap's resolveRecapScope):

      • linked visitor → (Conversation.LinkedEntityID, Conversation.LinkedRecordID)
      • linked anonymous → (the "MJ: Conversations" entity, Conversation.LastConversationID)
      • brand-new visitor → undefined (no scoped memory)

      Best-effort: any failure (no conversation id, load failure, entity not found) resolves to undefined so memory injection silently falls back to unscoped behavior.

      Parameters

      • conversationId: string

        the current session's conversation id, if any.

      • provider: IMetadataProvider

        the request/session metadata provider (multi-provider-safe — never global Metadata).

      • contextUser: UserInfo

        the calling user.

      Returns Promise<{ entityId: string; recordId: string }>

      the scope pair, or undefined when this isn't a returning visitor.

    • Resolves which agent a delegation should run, given an optional colleague name the model passed.

      • No name → the lead ExecuteRelayedToolInput.TargetAgentID (classic single-target).
      • A name matching an allowed colleague (by label, resolved agent Name, or id — case/whitespace insensitive) OR the lead's own name → that agent.
      • A name matching nothing in the union/lead → an error result naming the available colleagues, so the model self-corrects conversationally rather than silently delegating to the wrong agent.

      The allowed union is an affordance/UX filter; CanRun on the resolved agent (inside the agent run) remains the security boundary — both apply.

      Parameters

      • requestedAgent: string

        The colleague name the model named, or undefined.

      • input: ExecuteRelayedToolInput

        The relayed tool input (lead id + allowed union).

      Returns { Agent?: MJAIAgentEntityExtended; Error: string }

      { Agent } on success, or { Error } describing why no agent resolved.

    • Resolves the EFFECTIVE realtime configuration via the surface-agnostic precedence cascade: agent-TYPE DefaultConfiguration (base) < co-agent TypeConfiguration < target agent TypeConfiguration < (pre-authorized) runtime override — deep-merged per key and normalized. The target layer is what makes a voiced agent (Sage, Marketing Agent, …) carry its own voice/model regardless of host. Tolerant end-to-end: malformed layers contribute nothing and an unloaded metadata cache yields no type defaults. See plans/realtime/realtime-core-host-convergence.md.

      Parameters

      • coAgent: MJAIAgentEntityExtended

        The resolved co-agent.

      • OptionaloverridesJson: string

        The pre-authorized runtime override layer, when present.

      • OptionaltargetAgent: MJAIAgentEntityExtended

        The TARGET agent being voiced, when distinct from the co-agent — contributes the per-voiced-agent layer (above the co-agent, below the runtime override). Omit when there is none.

      • OptionalappSettingsJson: string

      Returns RealtimeCoAgentConfig

      The normalized effective configuration (possibly empty, never null).

    • Resolves the Realtime model + vendor driver + API key, mirroring BaseAgent's server-bridged resolution: highest-power active model of AIModelType Realtime; highest-priority active vendor whose DriverClass has a resolvable API key; instantiated via the ClassFactory.

      Overridable seam. Test subclasses override this to return a mock model so the service can be exercised without provider SDKs or DB metadata. Returns null (never throws) when any step can't be satisfied.

      Parameters

      Returns Promise<RealtimeModelResolution>

      The resolved model + identifiers, or null.

    • Selects the highest-priority active vendor for a model whose DriverClass has a resolvable API key. Mirrors BaseAgent.selectRealtimeVendor.

      Parameters

      • modelID: string

        The chosen model's id.

      Returns { APIName: string; DriverClass: string; VendorID: string }

      The vendor driver/api identifiers, or null when none has a usable key.

    • Wires a server long-lived (bridged) realtime session onto the SAME core machinery the client-direct path uses — so a LiveKit (or future Zoom/Teams) agent does real work and is tracked identically, with zero host-local re-implementation. This is the Phase 2 counterpart to PrepareClientSession: the browser relays tool calls back over GraphQL to ExecuteRelayedTool, whereas here the server holds the live IRealtimeSession and we wire its OnToolCall directly to the SAME ExecuteRelayedTool (so invoke-target-agent runs the target via AgentRunner, nests under the co-agent run, supports barge-in cancel + paused-run resume — all of it, for free).

      Responsibilities, in order:

      1. Create the co-agent observability run (+ prompt run + step) so the voice session shows up in the agent-run timeline and delegated runs nest under it (best-effort; a failure just omits the ids).
      2. Wire session.OnToolCallExecuteRelayedToolsession.SendToolResult.
      3. Guarantee finalize-once: wrap session.Close() and listen for an unexpected drop (OnClose), both routed through one idempotent finalizer. The bridge teardown calls Close(), so the run finalizes on graceful end; a dropped socket finalizes via OnClose.

      Parameters

      Returns Promise<BridgeRealtimeRuntime>

      A BridgeRealtimeRuntime the bridge holds for the session lifetime.