Member Junction
    Preparing search index...

    Host services handed to a BaseRealtimeChannelClient at BaseRealtimeChannelClient.Initialize.

    The context is the plugin's ONLY line back to the live session — channels never talk to RealtimeSessionService (or any host component) directly, which is what keeps them drop-in plugins. Every member is host-implemented:

    • the SESSION SERVICE supplies SendContextNote (perception feed into the live model), RequestSave (debounced state-of-record persistence onto the session's MJ: AI Agent Session Channels row) and AgentName;
    • the OVERLAY SHELL wires SetFocusMode (through the service's focus stream) so a channel surface can request the focus layout — main call column collapsed, surface panel filling the overlay, floating call pill riding on top.
    interface RealtimeChannelContext {
        AgentName: string;
        AgentSessionID: string | null;
        AgentVideo$?: Observable<MediaVideoSource | null>;
        AppContext$?: Observable<AppContextSnapshot | null>;
        Captures$?: Observable<RealtimeCaptureStates>;
        ChannelConfig?: Readonly<JSONObject>;
        Client?: BaseRealtimeClient | null;
        ConnectionState$?: Observable<RealtimeConnectionState>;
        Provider: IMetadataProvider | null;
        Resuming$?: Observable<boolean>;
        SessionEvents$?: Observable<RealtimeSessionStreamEvent>;
        VideoSources$?: Observable<readonly VideoSourceState[]>;
        DispatchContextAction?(
            request: RealtimeContextActionRequest,
        ): Promise<RealtimeContextActionResult>;
        ExecuteClientTool?(
            name: string,
            params: Record<string, unknown>,
        ): Promise<{ ErrorMessage?: string; Result?: unknown; Success: boolean }>;
        ExecuteServerAction<TResult>(
            query: string,
            variables: Record<string, JSONValue>,
        ): Promise<TResult | null>;
        IsTrackEstablished?(
            modality: string,
            direction: RealtimeTrackDirection,
        ): boolean;
        RequestSave(stateJson: string): void;
        RequestSpokenResponse?(instructions: string): void;
        SaveAsArtifact(name: string, contentJson: string): Promise<string | null>;
        SendContextNote(text: string): void;
        SendVideoFrame?(base64Image: string, mimeType?: string): void;
        SetFocusMode(on: boolean): void;
        StartCapture?(kind: RealtimeCaptureKind): Promise<RealtimeCaptureState>;
        StopCapture?(kind: RealtimeCaptureKind): void;
    }
    Index

    Properties

    AgentName: string

    Display name of the agent the live session fronts (e.g. "Sage"), fixed at session start.

    AgentSessionID: string | null

    The live MJ: AI Agent Sessions id this channel belongs to, or null before the session has minted / after it has torn down. A channel whose tools or surface drive a SERVER-SIDE resource (e.g. the Remote Browser channel's server-hosted browser) passes this as the agentSessionID argument to its own GraphQL resolvers via ExecuteServerAction. Most channels (whiteboard, shared doc) keep all state client-side and never read it.

    AgentVideo$?: Observable<MediaVideoSource | null>

    OPTIONAL — the agent's video while the model sends it, null otherwise (the runtime's AgentVideo$). A channel that shows the agent, one that sinks outbound video (BaseRealtimeChannelClient.GetSunkTracks), follows it here.

    AppContext$?: Observable<AppContextSnapshot | null>

    OPTIONAL — the live app-context stream (where the user is, what they see, and the available client-tool + agent manifest), pushed by the host (Explorer) at session start and on subsequent changes. The headless ClientContextChannel subscribes to this and streams deltas to the model via SendContextNote. Absent on hosts that don't supply app context (e.g. custom apps); channels must read it null-safely.

    Captures$?: Observable<RealtimeCaptureStates>

    OPTIONAL — the session's camera and screen share, now and on every change (the runtime's Captures$). A channel that fronts a capture (BaseRealtimeChannelClient.CaptureKind) follows it here.

    ChannelConfig?: Readonly<JSONObject>

    The resolved per-channel configuration for THIS channel: the host's defaults beneath the agent's and the app's channels.config[<key>] (see ResolveRealtimeChannelScope). Empty ({}) when none was configured. A channel reads its policy knobs from here (e.g. an identity channel's allowed email domains); it never reads the cascade itself.

    Treat it as untrusted, validated input: it is JSON authored by an operator, not typed code.

    Client?: BaseRealtimeClient | null

    OPTIONAL — the underlying BaseRealtimeClient driving the media and transport planes. Send video through a ChannelInboundVideoBridge over it, not through its own SendVideoFrame, which bypasses the session's VideoSourceArbiter.

    ConnectionState$?: Observable<RealtimeConnectionState>

    OPTIONAL — the call's state, now and on every change (the runtime's ConnectionState$): connecting, listening, speaking, thinking, an error, or closed. A channel that shows the agent follows the agent's turn here.

    Provider: IMetadataProvider | null

    The MJ metadata provider the live session runs on — the SAME IMetadataProvider instance that authenticates the session (NOT necessarily the global default). A channel whose surface renders MemberJunction-backed data (e.g. the Media channel streaming an MJ: Files record through mj-storage-media-player) threads THIS provider into its surface / GraphQL calls so it stays multi-provider safe. null only in degenerate/early states; channels fall back to the global default when absent.

    Resuming$?: Observable<boolean>

    OPTIONAL — whether the call is resuming on a new provider connection mid-call, now and on every change (the runtime's Resuming$). The agent's video can stop for a few seconds while it does, so a channel that shows the agent holds the last frame through the gap rather than showing that the video has stalled.

    SessionEvents$?: Observable<RealtimeSessionStreamEvent>

    OPTIONAL — the events the SERVER publishes to this session (identity verification today; app-defined types too), hot with no replay. A channel that reacts to them (an identity channel learning that the user verified) subscribes in BaseRealtimeChannelClient.OnInitialize and unsubscribes at BaseRealtimeChannelClient.Dispose. Absent on hosts whose session has no event transport.

    VideoSources$?: Observable<readonly VideoSourceState[]>

    OPTIONAL — the session's video sources, now and on every change (the runtime's VideoSources$): each source the video source arbiter knows, whether it is on (Enabled) and whether the model is being sent its frames (Active). On a model that takes one video stream, a capture that is on can still be one the model is not sent. A channel that fronts a capture finds its source by REALTIME_CAPTURE_SOURCE_IDS, and says the agent sees the capture only while that source is both.

    Methods

    • OPTIONAL — runs a CHANNEL-ADDRESSED ContextTool call: { action, params, target: { channel, instance? } }. The headless ClientContextChannel's proxy routes a call here when the model named a target; the host runtime resolves the channel, validates params against the verb's declared schema (a failure comes back as a structured, model-recoverable error — never a half-applied mutation), enforces InvokableBy, and opens an on-demand channel when the action is 'open'. Never throws.

      Absent on hosts that do not scope channels; the proxy then reports that channel addressing is unavailable instead of guessing.

      Parameters

      Returns Promise<RealtimeContextActionResult>

      A structured outcome the proxy serializes back to the model.

    • OPTIONAL — executes a host-registered surface CLIENT TOOL by name (the handlers the host wired from the active surface's SetAgentClientTools). The headless ClientContextChannel's ContextTool proxy routes the model's { action, params } here so a surface tool runs in the browser with no server round-trip. Best-effort and tolerant — resolves to a structured result (never throws); Success: false for an unknown tool or a thrown handler. Absent on hosts that register no client tools.

      Parameters

      • name: string

        The client-tool name (the model's action).

      • params: Record<string, unknown>

        The tool parameters (the model's params).

      Returns Promise<{ ErrorMessage?: string; Result?: unknown; Success: boolean }>

      A structured result the channel serializes back to the model.

    • Executes a CHANNEL-SPECIFIC GraphQL operation against the session's MJ server — the escape hatch for channels backed by a SERVER-SIDE resource that the generic RequestSave / SaveAsArtifact contract doesn't cover (e.g. the Remote Browser channel driving a server-hosted browser through its own ExecuteRemoteBrowserAction mutation + RemoteBrowserSnapshot query).

      The host runs the operation through the SAME provider the live session uses, so the request rides the authenticated session. Best-effort and tolerant: a transport or server error resolves to null (logged host-side) rather than throwing, so a channel can map the failure to a model-readable result string without try/catch.

      Type Parameters

      • TResult

        The expected shape of the GraphQL operation's data payload.

      Parameters

      • query: string

        The GraphQL query/mutation document.

      • variables: Record<string, JSONValue>

        The operation variables (all JSON-serializable).

      Returns Promise<TResult | null>

      The operation's data payload, or null on any failure / when no session is live.

    • Asks the host to persist stateJson as this channel's state of record. The host DEBOUNCES (a change burst becomes one save) and flushes any pending save at session teardown — the plugin just calls this on every state mutation and never schedules timers itself. Best-effort: persistence failures are logged host-side, never thrown.

      Parameters

      • stateJson: string

      Returns void

    • Asks the live model to SPEAK a response to the supplied instructions RIGHT NOW — the channel's "react to this" path, e.g. a widget submission the user expects an audible reaction to (SendContextNote deliberately never triggers speech). Rides the realtime client's spoken-update channel, so on some providers the spoken reply is narration-kind (ephemeral, not persisted as a caption). OPTIONAL member: older host contexts may not supply it — plugins must call it null-safely.

      Parameters

      • instructions: string

      Returns void

    • Persists a snapshot of the channel's state as a first-class versioned artifact (MJ: Artifacts + version, linked into conversation history when possible) — e.g. the whiteboard's "Save to artifacts". Distinct from RealtimeChannelContext.RequestSave, which maintains the session's rolling state of record. Best-effort: resolves to the created Artifact ID, or null on failure (logged host-side, never thrown). Works during the call AND right after it ends (the host retains the session id for late saves).

      Parameters

      • name: string
      • contentJson: string

      Returns Promise<string | null>

    • Feeds a background context note into the live realtime model (no spoken reply is requested) — the PERCEPTION direction of the channel: serialized state deltas flow here so the agent stays aware of what's on the surface. No-op when the session isn't live.

      Parameters

      • text: string

      Returns void

    • OPTIONAL — sends a visual frame into the live session's inbound video track, as this channel's source at the session's VideoSourceArbiter (the id its own frame bridge uses, ChannelVideoSourceID). The frame reaches the model only while the arbiter picks this channel, and the model is told when that changes; nothing is sent while the channel's exposure is below pixels. No-op when the session has not established an inbound video track or is not live.

      Parameters

      • base64Image: string

        The image data (base64-encoded JPEG/PNG).

      • OptionalmimeType: string

        The image MIME type (defaults to 'image/jpeg').

      Returns void

      Use a ChannelInboundVideoBridge (@memberjunction/ai-realtime-client), or BaseRealtimeChannelClient.EnableVisualPerception, which runs one for the channel: the bridge lists the source before its first frame, removes it when stopped, and says whether each frame was sent. The context's Client still bypasses the arbiter: a frame sent with its own SendVideoFrame overrides the source the arbiter picked, and the model is not told.

    • Requests (or releases) the FOCUS layout for this channel's surface: the overlay collapses the main call column and fills its stage with the surface, with a compact floating call pill keeping mute / thread / end reachable. Any channel may request it; the host tracks which channel holds focus and routes the pill's "exit" back to it via BaseRealtimeChannelClient.RequestFocusExit.

      Parameters

      • on: boolean

      Returns void