The resolved configuration; the defaults until Initialize reads the cascade.
ProtectedContextThe host context, available from Initialize until Dispose.
null outside that window — guard with ?. in any code that can run early/late.
ProtectedDataHow long after an action the data state is re-read; overridable for tests.
ReadonlyEngineThe components that are open.
ReadonlyEvents$Everything this channel streams out, as typed events: state changes, discrete user actions, frames pushed, opened/completed. Hosts subscribe (the embeddable element re-emits these as DOM events); the model is told about a change through a coalesced note instead — see RecordChange. Completes at Dispose.
ProtectedHandleHow long a verb waits for a component to initialize; overridable so tests need not sleep.
ReadonlyOutput$ProtectedPerceptionTuning for the perception coalescer (debounce window, max note size). Override to change it.
ProtectedSessionProtectedSessionMax time ResolveAgentSessionId waits for the session id to bind before giving up, and the poll interval it re-checks on. Protected so tests can shrink the wait; production keeps the defaults (the real mint race is sub-second, 8s is generous headroom).
ProtectedVisualThe shared frame bridge that carries frames to the model's inbound video track. Deliberately NOT named
videoBridge: channels that predate the pump (the Remote Browser, app channels) declare their own
private videoBridge, and a second private of the same name in the base class would stop them compiling.
StaticStrictWhether state snapshots are validated against the descriptor's noun schemas as notes are built —
a development aid that surfaces a channel whose state drifted from its own declaration. On by
default only when a Node-style environment reports a non-production NODE_ENV (tests, tooling);
a browser production build never pays for it. Hosts may set it explicitly.
The runtime capture this channel fronts ('camera' or 'screen'), or null. Such a channel is the capture's policy:
the runtime starts the capture only while the channel is in the session and the session's policy lets the agent see
pixels through it, and the user's "agent can see" choice for the channel decides whether the capture's frames reach
the model. Default null.
The channel definition name — MUST match the MJ: AI Agent Channels row's Name
(e.g. 'Whiteboard'). Used as the persistence key for SerializeState saves
and as the channel tab's stable key.
ProtectedCurrentThe id of the latest state change (0 before any).
How much of this channel the model may perceive without asking: the lowest of the channel's own ceiling
(GetDescriptor().MaxExposure), the server policy and the user's choice. A channel nobody has applied a
policy to simply has its ceiling, which is what every channel had before exposure policy existed.
'none': no state notes and no frames.'state': structured state notes; no frames.'pixels': state notes and frames.It governs what flows to the model UNPROMPTED (perception notes, the contents of opened / completed
notes, frames). It does not REDACT what a verb's result contains: a verb is all-or-nothing. A verb whose
result would show the model something above this level declares ReturnsChannelData on the verb, and
when this level is below it the verb is refused whole (exposure_restricted) and listed to the model as
unavailable; see RefuseVerbForExposure. A verb that does not declare it returns what the channel
says it returns, so a channel whose verbs reveal sensitive data must either declare it or lower its own
MaxExposure.
Why Exposure is below the channel's ceiling; empty when it is not.
The id state notes carry. A multi-instance channel's snapshot describes ALL its instances (keyed by their own ids,
c1, c2, ...), so the notes about the snapshot as a whole are addressed to all rather than to one of them.
Whether this channel shows the agent's video: it sinks outbound video (GetSunkTracks). The runtime marks it as used when the video arrives, and a host presents its surface as the agent (the call overlay puts it in the agent's place in the call rather than in the focus layout).
Where this channel's surface shows when a call starts, and where the user may move it: from the channel's registry
row (UIConfig.Placement and UIConfig.AllowedPlacements). A channel without a row, or whose row says nothing,
starts on its tab and can go anywhere.
Whether the user may share this channel's surface on its own: a host that offers it lists the surface, named by
TabTitle with TabIcon, under "This panel" in its Share menu while the surface is on screen, and the
share then shows only that surface. Default false, so a channel opts in with this getter. A surface that shows the
agent its own call (the camera, the screen share, the agent's video) never should.
OPTIONAL accent color for the channel's tab (a CSS color string, e.g. an hsl() /
token). When a plugin supplies one, the overlay paints the tab's dot + active underline
with it; when omitted (the default null), the overlay derives a stable, deterministic
color from the ChannelName so every channel still reads as a distinct, colored
surface. A channel only overrides this to enforce a specific brand accent.
Font Awesome icon class for the channel's tab (e.g. 'fa-solid fa-chalkboard'). Defaults to a generic puzzle-piece.
Label for the channel's tab on the overlay's surface panel (e.g. 'Whiteboard'). Defaults to ChannelName.
The shared name prefix of every tool this channel exposes (e.g. 'Whiteboard_').
The host registers ONE local-execution route per plugin: tool calls whose name starts
with this prefix go to ApplyAgentTool instead of the server relay.
Default '' — a v2 channel with NO native tools (it is reached only through the ContextTool
proxy). The host registers no route for an empty prefix: an empty prefix would match every
tool name and hijack the server relay.
Asks to be offered a delegated run's artifacts when the operator turned on auto-open, or when one of them is already open here (so a revised version replaces it). Whether an artifact is a component is only known once it is loaded, so with auto-open on the channel is mounted for any delegated artifact and ignores the ones that are not.
Executes ONE agent tool call locally (the ACTION direction) and returns the result
JSON string fed back to the model as the tool_response. Called for every tool whose
name starts with ToolNamePrefix. Must work both WITH a bound surface (apply +
UI garnish) and WITHOUT one (apply to the state engine directly — the surface may not
exist yet, e.g. it has not been shown). Should not throw: return a
{ success: false, error } payload so the model can narrate the failure (the host
additionally wraps anything thrown).
This is the LEGACY entry point. Channels written against the v2 contract override ApplyVerb instead and inherit this method: the default maps the tool name back to its verb, calls ApplyVerb as the agent, and serializes the structured result. Channels written against v1 override THIS and inherit ApplyVerb, which adapts it. A channel must override at least one of the two — overriding neither is reported as a failure result, never a recursion.
Applies exposure policy: the server's decision and the user's own choice. The runtime calls this when it mounts the channel and again whenever the user changes their choice.
Withdrawing exposure takes effect immediately: a pending perception note is dropped, a pending frame is dropped, and the channel's video source stops being forwarded. Restoring it re-baselines (the model's picture of the channel is stale, so the next note is a full snapshot) and sends a fresh frame. When the call is live the model is told about the change, with the reason, so it never assumes it can still see what it can't.
Policy — the server-decided exposure; User — the user's choice (omit for none);
Reasons — why the policy lowered it, from the server's limits.
Sets SurfacePlacement. The runtime calls it with the registry row's placement when it builds the channel, before the channel mounts, as it applies exposure policy with ApplyExposure.
Runs one verb. The dispatcher has already checked the verb against the (merged) descriptor; the exact check against the addressed component's own schema happens here.
OptionalinstanceId: stringHands the dynamically created surface the engine it renders and the hooks it reports through.
ProtectedCaptureA picture of the component the user is looking at, as base64 JPEG, or null when there is none to send: exposure is
below pixels, the session has no inbound video track, nothing is on screen, or the host's rasterizer produced nothing
(it reports why, once, itself). Never throws.
The base channel's visual pump has already checked exposure and the track before it asks; they are checked again so a frame is never rasterized (the costly part) for a model that may not see it, whoever asks.
The channel finished and is handing something back (a submitted form, a chosen option, a
finished game). Emits on Output$ and Events$ and tells the model with a
completed note. The output is validated against the descriptor's Output schema when
StrictStateChecks is on; a mismatch is logged, never swallowed into silence or thrown.
What the channel hands back.
OptionalinstanceId: stringThe instance that completed, for a multi-instance channel (default: the primary instance).
ProtectedConfirmAn AGENT action succeeded and changed the channel's picture: push exactly one confirmation frame now and tell the model not to narrate its own change. Call only on SUCCESS — confirming a failed action would assert an edit landed that did not. A no-op until EnableVisualPerception ran.
Tears the plugin down at session end: release the surface binding, unsubscribe
state-engine subscriptions, then drop the context. Subclasses overriding this MUST
call super.Dispose(). Any final state save has already been flushed by the host
(the debounced RealtimeChannelContext.RequestSave pipeline) before disposal.
ProtectedEmitEmits a typed event on Events$ (NOT to the model — use RecordChange for state the model should perceive, or send an explicit note for a discrete event it must hear about).
The event name (declare it in the descriptor's Events).
Optionalpayload: JSONObjectThe event payload.
OptionalchangeId: numberThe change id the event relates to, when it relates to one.
OptionalinstanceId: stringThe instance the event is about, for a multi-instance channel (default: the primary instance).
ProtectedEnableOpts this channel into VISUAL perception: its surface can be shown to a video-capable model as frames. Call it from OnInitialize with the channel's frame source.
The channel then needs only to say when its picture changed — NotifyVisualChange after a
user edit, ConfirmVisualChange after a successful agent edit — and the base class does the
rest: change-driven (no idle heartbeat), paced to the negotiated cadence, a trailing settle so the
model ends on the final state of a burst, dedupe of identical frames, and a single confirmation
frame (with a "do not narrate" note) after an agent edit. Every frame is tagged with the change id
of the state it was captured at (frame_pushed events), so state and pixels can be cross-checked.
Also makes GetSourcedTracks report the inbound video track, which is what gets it negotiated. Idempotent: calling it again replaces the provider and options.
Renders the channel's current picture as a base64 JPEG (or null).
Optionaloptions: VisualPerceptionOptionsProtectedEnsureReturns the shared frame bridge, creating it if needed and (only when the channel opted into the
legacy poller) starting its fixed-rate poll once the video track is up. null when no context is bound.
The descriptor verb a native tool call runs, or undefined when the tool is not one of the channel's verbs
(the runtime uses it to apply RefuseVerbForExposure on the native-tool route as well as the ContextTool one).
The native tool name the model called.
ProtectedFlushSends any pending coalesced perception note now (e.g. before a surface is torn down).
The channel's self-description. The verbs and events are the built-ins plus those of every component that is open right now, so the contract grows as components are opened and the dispatcher validates against it.
The channel's FIRST-RUN INTRO content, or null when the channel offers no onboarding.
The overlay shows this once per channel per user — the first time the user opens this
channel's surface tab — and remembers "seen" via the user's settings (NOT localStorage),
so it never re-appears on later sessions or other devices.
Default: null (no intro). The base Voice/text channel has no plugin at all, so it never
shows an intro; an interactive channel with a surface worth explaining (whiteboard, remote
browser, …) overrides this to return its ChannelOnboardingDetails. A plugin that
doesn't override it simply shows nothing — onboarding is strictly opt-in.
Media tracks this client channel can SOURCE — samples flowing into the model.
Default [].
The open components, keyed by instance id (see the components noun). A pure read.
Media tracks this client channel can SINK — samples flowing from the model OUT.
Default [].
The session requests them when it connects, so the driver establishes the ones the model supports. A channel that sinks outbound video shows the agent's video: it follows RealtimeChannelContext.AgentVideo$, and counts as used once the video arrives, so the host shows its surface.
The Angular component the overlay creates dynamically as this channel's tab pane, or null
for a server-only channel that renders no MJ surface (its surface, if any, lives on the
external platform — e.g. a bridge-contributed native whiteboard or meeting-controls channel).
When this returns null, the overlay renders NO tab for the channel and never calls
BindSurface/UnbindSurface — but the channel's tools (GetToolDefinitions /
ApplyAgentTool) and perception (RealtimeChannelContext.SendContextNote) still run.
A created surface instance is handed straight back via BindSurface; the host treats it as
opaque.
Default: null (server-only). A channel with a rendered surface overrides this to return its
component type.
The channel has no native tools: every verb is reached through the ContextTool proxy, because the verbs depend on what is open.
Whether this channel has a rendered MJ surface (GetSurfaceComponent returns non-null).
The overlay uses this to decide whether to register a surface tab; server-only channels are
false. Override only if surface availability must be decided WITHOUT constructing the type
(the default calls GetSurfaceComponent once).
Binds the host context and invokes the OnInitialize hook. Called exactly once per session, right after ClassFactory instantiation and before any tool call or surface bind.
ProtectedNotifyThe channel's picture changed because of a USER action (or a scene replacement): push a frame, respecting the cadence. Resolves when the decision has been carried out; never rejects. A no-op until EnableVisualPerception ran and the session has an inbound video track.
Called by the surface when something happens inside a rendered component: an event it emits, a state change, or the end of its initialization. Becomes a change in the state the model perceives and a typed event for observers.
Takes a delegated run's artifacts: an artifact that is already open moves to its newer version (or, when the configuration says not to swap, the agent is told one exists); with auto-open on, a component that is not open is opened.
ProtectedOnSubclass hook invoked from Initialize once Context is bound — wire
state-engine subscriptions (e.g. state change → Context.RequestSave(...)) here.
Default: no-op.
ProtectedOnLoads the artifact and opens it as a new instance (or, when that artifact is already open, brings the existing instance forward and moves it to the requested version). Refusals are structured results, never exceptions.
Subclass hook invoked once the realtime session is connected and live (the client driver
is created, connected, and media tracks negotiated), or when the channel is opened in a live
session. Channels that establish media bridges (e.g. video streaming) can start them here when
Context.Client is available.
Default: a channel with visual perception adds the inbound video track it sources when the call
has none (it was opened mid-call) and brings its frame bridge up; any other channel does nothing.
An override that keeps visual perception calls super.OnSessionStarted().
Called by the host when the bound surface moves: to the stage (it fills the call), to its tab, or out of sight because the user hid it. A channel can adapt its own chrome, such as a "Move to stage" button that has nothing to do once the surface is on the stage. Called right after BindSurface with the current placement, then on every move, and not after UnbindSurface. Default: no-op.
Called by the host when the bound surface comes into sight or goes out of it (the panel collapsed or hidden, another tab active, the call minimized). The surface stays bound and keeps its state; a channel can pause work nobody sees, such as a screenshot poll, and resume it when the surface is shown again. The host calls it right after BindSurface with the surface's current visibility, then on every change, and not after UnbindSurface. Default: no-op, so a surface keeps running out of sight.
Opens the channel — mounts it for use and seeds it with inputs. Called by the runtime when the
agent (or user) opens an on-demand channel, after the channel has been initialized.
Validates inputs against the descriptor's Inputs schema (a violation is a structured
'invalid_params' failure, never a half-opened channel), runs the OnOpen hook, then tells
the model the channel is open and what it currently holds (opened note with a state snapshot) —
and takes that snapshot as the baseline later changes are described against.
Optionalinputs: JSONObjectWhat the agent asked to seed the channel with.
ProtectedRecordRecords that the channel's state changed. Call this on EVERY mutation, whoever caused it.
It (1) assigns the change a monotonic id, (2) emits a state_changed event on Events$
immediately (observers see every change), and (3) — unless Perceive is false — schedules a
COALESCED structured note to the model: a burst of N changes inside the debounce window produces
ONE [channel:<Key>#<instance>] state_changed {"changeId":…,"changes":N,"delta":{…}} note carrying
what differs from what the model was last told (the first note is a full snapshot). Notes that
would be too large degrade to a list of changed paths.
Optionaloptions: { Author?: RealtimeChannelActor | "system"; Perceive?: boolean }Author — who made the change (carried on the event); Perceive — whether the
model should be told (default true; a channel that already feeds the model its own perception
— like the Whiteboard's scene deltas — passes false).
The change's id.
Whether the agent may be given this verb's result at the channel's current exposure. A verb that declares
ReturnsChannelData is refused whole when exposure is below that level, because its result would hand the agent
what the user or a zero-data-retention policy held back from it. Policy, not redaction: the result is never filtered.
Applies to the AGENT (the dispatcher and the native-tool route call it); a user acting through the surface is not restricted.
The verb the agent is calling.
The sentence to give the agent, or null when the call may proceed.
The focus pill's "exit" affordance, routed by the overlay to the channel that holds
focus. Implementations should leave focus mode through their OWN surface (so surface
toggles stay in sync), ultimately emitting Context.SetFocusMode(false). The overlay
defensively clears its layout flag as well, so a no-op default is safe.
ProtectedResolveResolves the live RealtimeChannelContext.AgentSessionID, briefly WAITING for it when it
isn't bound yet rather than giving up instantly. AgentSessionID is a live getter over the
session service's current id: it reads null in the window BEFORE the session mints (the
realtime model can fire a tool call the very first beat it connects, before mintSession
resolves) and again AFTER teardown. Server-backed tool paths (e.g. the Remote Browser channel's
browser_* tools) call this instead of reading Context?.AgentSessionID synchronously, so a tool
invoked a beat early WAITS for the session to come live — defense-in-depth against the
"session id missing" race — instead of returning a hard failure to the model.
Returns the id as soon as it's non-null (the common path resolves immediately, no delay), or
null if it's still unbound after SessionIdWaitTimeoutMs — or the channel was
Disposed in the meantime (Context goes null, so we stop waiting on a torn-down session).
ProtectedResolveMaps a verb to the native tool name it is reachable as (inverse of ResolveVerbForTool).
ProtectedResolveMaps a native tool name back to its verb: the descriptor's NativeToolName match first, then the
name minus ToolNamePrefix. Protected so a channel with an unusual naming scheme can
override it.
Live-only: a resumed session never restores open components (see SerializeState). Always false.
Open components are LIVE-ONLY: there is no state of record to persist, so nothing is saved with the session
(null, which the host never writes) and RestoreState refuses what it is offered. A resumed session starts
with no component open; the agent opens them again. What the model perceives right now is GetState.
Replaces where artifacts are loaded from (a host with its own store, or a test). Without it the channel loads through the session's provider and the signed-in user's access.
Called by the host when the surface component is being destroyed (the channel left the session / the overlay was torn down). Drop the instance reference and unsubscribe any output subscriptions — after this, ApplyAgentTool runs in its no-surface mode. Default: no-op.
Checks the current GetState snapshot against the descriptor's noun schemas and returns every violation (empty when valid). Useful in a channel's own tests; the runtime also runs it on every perception note when StrictStateChecks is on.
The INTERACTIVE COMPONENT channel: shows the user any interactive component artifact (a dashboard, a report, a form, a chart) next to the voice conversation, and lets the agent operate it.
It is a MULTI-INSTANCE channel (several components may be open at once, each addressed by
target.instance) and its contract is DERIVED from what it hosts: the actions an agent can take on an instance are that component's owncustomMethodsand the standard methods its spec says it supports; its state is the component's data state; its events are the spec'sevents; the inputs for opening it are the spec'sproperties. A component nobody wrote channel code for is therefore operable from its spec alone. SeeDeriveComponentContract.Built-in:
open { artifactId | artifactVersionId, inputs? }(the dispatcher's reserved action),show_version,close.What the model perceives
Per instance: which artifact version is open, what the user last did in it (its events), and its data state, bounded to a handful of rows. All of it is gated by exposure policy (
min(channel, agent config, zero-data-retention, user)). Pixels flow only if the host registered a frame capturer (ChannelFrameCapture). Agent-INVOKED verbs return what the component returns; exposure gates what is volunteered to the model, not what it asked for.Access
Artifacts are loaded through the signed-in user's own session and read permission (see
RunViewComponentArtifactSource). The agent cannot open anything the user cannot.Lifetime
Open components are live-only: they are not persisted with the session and are not restored on resume.