Protected ReadonlyComponentThe existing component to show as the channel's surface. The overlay creates it; the adapter binds to it.
ProtectedContextThe host context, available from Initialize until Dispose.
null outside that window — guard with ?. in any code that can run early/late.
Protected ReadonlyDescriptorThe channel's self-description: what it is, what it holds (nouns), what it accepts (verbs with parameter schemas and
InvokableBy), what it emits (events), how it opens (inputs) and what it hands back (output). Pure data, and the only
thing an agent has to go on, so write Instructions and every description for the MODEL.
ReadonlyEvents$Everything this channel streams out, as typed events: state changes, discrete user actions, frames pushed, opened/completed. Hosts subscribe (the embeddable element re-emits these as DOM events); the model is told about a change through a coalesced note instead — see RecordChange. Completes at Dispose.
ReadonlyOutput$ProtectedPerceptionTuning for the perception coalescer (debounce window, max note size). Override to change it.
ProtectedSessionProtectedSessionMax time ResolveAgentSessionId waits for the session id to bind before giving up, and the poll interval it re-checks on. Protected so tests can shrink the wait; production keeps the defaults (the real mint race is sub-second, 8s is generous headroom).
ProtectedSurfaceHow long a verb waits for the surface to bind before failing with 'surface_unavailable', in ms. Override to change it.
ProtectedVerbHow long ApplySurfaceVerb may run before the call fails with 'verb_timeout' and the queue is released, in ms
(15 s by default; the surface bind wait is separate, see SurfaceBindTimeoutMs). Override it for a component whose
verbs are legitimately slow; a value that is not a positive number turns the limit off, which is only safe for a verb that
is guaranteed to settle. See the file comment for what happens to a verb that settles after the timeout.
ProtectedVisualThe shared frame bridge that carries frames to the model's inbound video track. Deliberately NOT named
videoBridge: channels that predate the pump (the Remote Browser, app channels) declare their own
private videoBridge, and a second private of the same name in the base class would stop them compiling.
StaticStrictWhether state snapshots are validated against the descriptor's noun schemas as notes are built —
a development aid that surfaces a channel whose state drifted from its own declaration. On by
default only when a Node-style environment reports a non-production NODE_ENV (tests, tooling);
a browser production build never pays for it. Hosts may set it explicitly.
The runtime capture this channel fronts ('camera' or 'screen'), or null. Such a channel is the capture's policy:
the runtime starts the capture only while the channel is in the session and the session's policy lets the agent see
pixels through it, and the user's "agent can see" choice for the channel decides whether the capture's frames reach
the model. Default null.
The channel definition name — MUST match the MJ: AI Agent Channels row's Name
(e.g. 'Whiteboard'). Used as the persistence key for SerializeState saves
and as the channel tab's stable key.
ProtectedCurrentThe id of the latest state change (0 before any).
How much of this channel the model may perceive without asking: the lowest of the channel's own ceiling
(GetDescriptor().MaxExposure), the server policy and the user's choice. A channel nobody has applied a
policy to simply has its ceiling, which is what every channel had before exposure policy existed.
'none': no state notes and no frames.'state': structured state notes; no frames.'pixels': state notes and frames.It governs what flows to the model UNPROMPTED (perception notes, the contents of opened / completed
notes, frames). It does not REDACT what a verb's result contains: a verb is all-or-nothing. A verb whose
result would show the model something above this level declares ReturnsChannelData on the verb, and
when this level is below it the verb is refused whole (exposure_restricted) and listed to the model as
unavailable; see RefuseVerbForExposure. A verb that does not declare it returns what the channel
says it returns, so a channel whose verbs reveal sensitive data must either declare it or lower its own
MaxExposure.
Why Exposure is below the channel's ceiling; empty when it is not.
The channel's stable primary-instance id. A single-instance channel has exactly one; a
RealtimeChannelDescriptor.MultiInstance channel owns further instance ids itself and
routes by the instanceId argument of ApplyVerb. Appears in every structured note
([channel:<Key>#<instance>] …) and every RealtimeChannelEvent.
Whether this channel shows the agent's video: it sinks outbound video (GetSunkTracks). The runtime marks it as used when the video arrives, and a host presents its surface as the agent (the call overlay puts it in the agent's place in the call rather than in the focus layout).
ProtectedSurfaceThe bound component, or null while no surface is up.
Where this channel's surface shows when a call starts, and where the user may move it: from the channel's registry
row (UIConfig.Placement and UIConfig.AllowedPlacements). A channel without a row, or whose row says nothing,
starts on its tab and can go anywhere.
Whether the user may share this channel's surface on its own: a host that offers it lists the surface, named by
TabTitle with TabIcon, under "This panel" in its Share menu while the surface is on screen, and the
share then shows only that surface. Default false, so a channel opts in with this getter. A surface that shows the
agent its own call (the camera, the screen share, the agent's video) never should.
OPTIONAL accent color for the channel's tab (a CSS color string, e.g. an hsl() /
token). When a plugin supplies one, the overlay paints the tab's dot + active underline
with it; when omitted (the default null), the overlay derives a stable, deterministic
color from the ChannelName so every channel still reads as a distinct, colored
surface. A channel only overrides this to enforce a specific brand accent.
Font Awesome icon class for the channel's tab (e.g. 'fa-solid fa-chalkboard'). Defaults to a generic puzzle-piece.
Label for the channel's tab on the overlay's surface panel (e.g. 'Whiteboard'). Defaults to ChannelName.
The shared name prefix of every tool this channel exposes (e.g. 'Whiteboard_').
The host registers ONE local-execution route per plugin: tool calls whose name starts
with this prefix go to ApplyAgentTool instead of the server relay.
Default '' — a v2 channel with NO native tools (it is reached only through the ContextTool
proxy). The host registers no route for an empty prefix: an empty prefix would match every
tool name and hijack the server relay.
Whether this channel wants to be offered the artifacts a delegated run just produced (the agent asked Skip or Sage to build something, and it came back as artifacts). A channel that hosts artifacts says yes for the ones it can show — and for an artifact it ALREADY shows, so a newer version replaces it in place.
Asked of every channel in the session, mounted or merely advertised. For an advertised channel it is asked before the channel has been initialized, so it must not depend on Context; the channel's resolved configuration is passed in for exactly that reason. Saying yes for an advertised channel mounts it, so only answer yes when you will do something with the artifacts.
Default: false — a channel that does not host artifacts is never bothered.
Executes ONE agent tool call locally (the ACTION direction) and returns the result
JSON string fed back to the model as the tool_response. Called for every tool whose
name starts with ToolNamePrefix. Must work both WITH a bound surface (apply +
UI garnish) and WITHOUT one (apply to the state engine directly — the surface may not
exist yet, e.g. it has not been shown). Should not throw: return a
{ success: false, error } payload so the model can narrate the failure (the host
additionally wraps anything thrown).
This is the LEGACY entry point. Channels written against the v2 contract override ApplyVerb instead and inherit this method: the default maps the tool name back to its verb, calls ApplyVerb as the agent, and serializes the structured result. Channels written against v1 override THIS and inherit ApplyVerb, which adapts it. A channel must override at least one of the two — overriding neither is reported as a failure result, never a recursion.
Applies exposure policy: the server's decision and the user's own choice. The runtime calls this when it mounts the channel and again whenever the user changes their choice.
Withdrawing exposure takes effect immediately: a pending perception note is dropped, a pending frame is dropped, and the channel's video source stops being forwarded. Restoring it re-baselines (the model's picture of the channel is stale, so the next note is a full snapshot) and sends a fresh frame. When the call is live the model is told about the change, with the reason, so it never assumes it can still see what it can't.
Policy — the server-decided exposure; User — the user's choice (omit for none);
Reasons — why the policy lowered it, from the server's limits.
Sets SurfacePlacement. The runtime calls it with the registry row's placement when it builds the channel, before the channel mounts, as it applies exposure policy with ApplyExposure.
ProtectedApplyWhat each verb does to the component. Parameters are already validated against the descriptor's schema.
Runs one verb. Verbs are serialized (a second call waits for the first), wait (bounded) for the surface, and run against the real component, so the outcome the agent hears is the outcome the user sees.
The overlay created the component: bind to it. Subscribes to its events and completion, restores the last known state, applies any seed inputs that arrived before it existed, and releases verbs that were waiting for it.
The channel finished and is handing something back (a submitted form, a chosen option, a
finished game). Emits on Output$ and Events$ and tells the model with a
completed note. The output is validated against the descriptor's Output schema when
StrictStateChecks is on; a mismatch is logged, never swallowed into silence or thrown.
What the channel hands back.
OptionalinstanceId: stringThe instance that completed, for a multi-instance channel (default: the primary instance).
ProtectedConfirmAn AGENT action succeeded and changed the channel's picture: push exactly one confirmation frame now and tell the model not to narrate its own change. Call only on SUCCESS — confirming a failed action would assert an edit landed that did not. A no-op until EnableVisualPerception ran.
Tears the plugin down at session end: release the surface binding, unsubscribe
state-engine subscriptions, then drop the context. Subclasses overriding this MUST
call super.Dispose(). Any final state save has already been flushed by the host
(the debounced RealtimeChannelContext.RequestSave pipeline) before disposal.
ProtectedEmitEmits a typed event on Events$ (NOT to the model — use RecordChange for state the model should perceive, or send an explicit note for a discrete event it must hear about).
The event name (declare it in the descriptor's Events).
Optionalpayload: JSONObjectThe event payload.
OptionalchangeId: numberThe change id the event relates to, when it relates to one.
OptionalinstanceId: stringThe instance the event is about, for a multi-instance channel (default: the primary instance).
ProtectedEnableOpts this channel into VISUAL perception: its surface can be shown to a video-capable model as frames. Call it from OnInitialize with the channel's frame source.
The channel then needs only to say when its picture changed — NotifyVisualChange after a
user edit, ConfirmVisualChange after a successful agent edit — and the base class does the
rest: change-driven (no idle heartbeat), paced to the negotiated cadence, a trailing settle so the
model ends on the final state of a burst, dedupe of identical frames, and a single confirmation
frame (with a "do not narrate" note) after an agent edit. Every frame is tagged with the change id
of the state it was captured at (frame_pushed events), so state and pixels can be cross-checked.
Also makes GetSourcedTracks report the inbound video track, which is what gets it negotiated. Idempotent: calling it again replaces the provider and options.
Renders the channel's current picture as a base64 JPEG (or null).
Optionaloptions: VisualPerceptionOptionsProtectedEnsureReturns the shared frame bridge, creating it if needed and (only when the channel opted into the
legacy poller) starting its fixed-rate poll once the video track is up. null when no context is bound.
The descriptor verb a native tool call runs, or undefined when the tool is not one of the channel's verbs
(the runtime uses it to apply RefuseVerbForExposure on the native-tool route as well as the ContextTool one).
The native tool name the model called.
ProtectedFlushSends any pending coalesced perception note now (e.g. before a surface is torn down).
The channel's SELF-DESCRIPTION — what it is, what it holds, what it accepts, how it behaves in a session. This is the contract an agent reads to operate a channel it has never seen, and what the runtime reads to scope, advertise and validate. Pure data; called often, so keep it cheap.
Default: a descriptor SYNTHESIZED from the v1 members (tools → verbs, intro → instructions),
stamped with the legacy contract version — so a channel that implements nothing new is still
describable and works exactly as before. A v2 channel overrides this with an authored descriptor.
(Build verbs from GetToolDefinitions with BuildToolBackedVerbs when they are the
native tools, so a subclass that adds a tool does not leave the descriptor lying.)
The channel's FIRST-RUN INTRO content, or null when the channel offers no onboarding.
The overlay shows this once per channel per user — the first time the user opens this
channel's surface tab — and remembers "seen" via the user's settings (NOT localStorage),
so it never re-appears on later sessions or other devices.
Default: null (no intro). The base Voice/text channel has no plugin at all, so it never
shows an intro; an interactive channel with a surface worth explaining (whiteboard, remote
browser, …) overrides this to return its ChannelOnboardingDetails. A plugin that
doesn't override it simply shows nothing — onboarding is strictly opt-in.
Media tracks this client channel can SOURCE — samples flowing into the model.
Default [].
The channel's state: the live component's, or the last known when no component is bound (so a note built while the panel is collapsed still describes the channel truthfully).
Media tracks this client channel can SINK — samples flowing from the model OUT.
Default [].
The session requests them when it connects, so the driver establishes the ones the model supports. A channel that sinks outbound video shows the agent's video: it follows RealtimeChannelContext.AgentVideo$, and counts as used once the video arrives, so the host shows its surface.
The Angular component the overlay creates dynamically as this channel's tab pane, or null
for a server-only channel that renders no MJ surface (its surface, if any, lives on the
external platform — e.g. a bridge-contributed native whiteboard or meeting-controls channel).
When this returns null, the overlay renders NO tab for the channel and never calls
BindSurface/UnbindSurface — but the channel's tools (GetToolDefinitions /
ApplyAgentTool) and perception (RealtimeChannelContext.SendContextNote) still run.
A created surface instance is handed straight back via BindSurface; the host treats it as
opaque.
Default: null (server-only). A channel with a rendered surface overrides this to return its
component type.
The channel has no native tools: its verbs are reached through the ContextTool proxy.
Whether this channel has a rendered MJ surface (GetSurfaceComponent returns non-null).
The overlay uses this to decide whether to register a surface tab; server-only channels are
false. Override only if surface availability must be decided WITHOUT constructing the type
(the default calls GetSurfaceComponent once).
Binds the host context and invokes the OnInitialize hook. Called exactly once per session, right after ClassFactory instantiation and before any tool call or surface bind.
ProtectedNotifyThe channel's picture changed because of a USER action (or a scene replacement): push a frame, respecting the cadence. Resolves when the decision has been carried out; never rejects. A no-op until EnableVisualPerception ran and the session has an inbound video track.
A delegated run produced artifacts this channel said it wants (AcceptsDelegationArtifacts). The channel is mounted and initialized by now. Never throw into the runtime: handle your own failures and report them (the runtime logs anything that escapes, but cannot recover the work).
Default: nothing.
ProtectedOnOpts into pictures when the subclass can name the component's element and the host registered a rasterizer. Subclasses
that override this MUST call super.OnInitialize().
ProtectedOnSeeds the surface with the open inputs: now if the component is bound, otherwise when it binds (there is nothing to report back but success, so there is nothing to wait for).
Subclass hook invoked once the realtime session is connected and live (the client driver
is created, connected, and media tracks negotiated), or when the channel is opened in a live
session. Channels that establish media bridges (e.g. video streaming) can start them here when
Context.Client is available.
Default: a channel with visual perception adds the inbound video track it sources when the call
has none (it was opened mid-call) and brings its frame bridge up; any other channel does nothing.
An override that keeps visual perception calls super.OnSessionStarted().
ProtectedOnA board created again (the panel was collapsed and expanded, or the session resumed) is put back as it was.
ProtectedOnopen may say who plays first.
Called by the host when the bound surface moves: to the stage (it fills the call), to its tab, or out of sight because the user hid it. A channel can adapt its own chrome, such as a "Move to stage" button that has nothing to do once the surface is on the stage. Called right after BindSurface with the current placement, then on every move, and not after UnbindSurface. Default: no-op.
Called by the host when the bound surface comes into sight or goes out of it (the panel collapsed or hidden, another tab active, the call minimized). The surface stays bound and keeps its state; a channel can pause work nobody sees, such as a screenshot poll, and resume it when the surface is shown again. The host calls it right after BindSurface with the surface's current visibility, then on every change, and not after UnbindSurface. Default: no-op, so a surface keeps running out of sight.
Opens the channel — mounts it for use and seeds it with inputs. Called by the runtime when the
agent (or user) opens an on-demand channel, after the channel has been initialized.
Validates inputs against the descriptor's Inputs schema (a violation is a structured
'invalid_params' failure, never a half-opened channel), runs the OnOpen hook, then tells
the model the channel is open and what it currently holds (opened note with a state snapshot) —
and takes that snapshot as the baseline later changes are described against.
Optionalinputs: JSONObjectWhat the agent asked to seed the channel with.
ProtectedReadThe board as the model perceives it: one entry per noun in the descriptor.
ProtectedRecordRecords that the channel's state changed. Call this on EVERY mutation, whoever caused it.
It (1) assigns the change a monotonic id, (2) emits a state_changed event on Events$
immediately (observers see every change), and (3) — unless Perceive is false — schedules a
COALESCED structured note to the model: a burst of N changes inside the debounce window produces
ONE [channel:<Key>#<instance>] state_changed {"changeId":…,"changes":N,"delta":{…}} note carrying
what differs from what the model was last told (the first note is a full snapshot). Notes that
would be too large degrade to a list of changed paths.
Optionaloptions: { Author?: RealtimeChannelActor | "system"; Perceive?: boolean }Author — who made the change (carried on the event); Perceive — whether the
model should be told (default true; a channel that already feeds the model its own perception
— like the Whiteboard's scene deltas — passes false).
The change's id.
Whether the agent may be given this verb's result at the channel's current exposure. A verb that declares
ReturnsChannelData is refused whole when exposure is below that level, because its result would hand the agent
what the user or a zero-data-retention policy held back from it. Policy, not redaction: the result is never filtered.
Applies to the AGENT (the dispatcher and the native-tool route call it); a user acting through the surface is not restricted.
The verb the agent is calling.
The sentence to give the agent, or null when the call may proceed.
The focus pill's "exit" affordance, routed by the overlay to the channel that holds
focus. Implementations should leave focus mode through their OWN surface (so surface
toggles stay in sync), ultimately emitting Context.SetFocusMode(false). The overlay
defensively clears its layout flag as well, so a no-op default is safe.
ProtectedResolveResolves the live RealtimeChannelContext.AgentSessionID, briefly WAITING for it when it
isn't bound yet rather than giving up instantly. AgentSessionID is a live getter over the
session service's current id: it reads null in the window BEFORE the session mints (the
realtime model can fire a tool call the very first beat it connects, before mintSession
resolves) and again AFTER teardown. Server-backed tool paths (e.g. the Remote Browser channel's
browser_* tools) call this instead of reading Context?.AgentSessionID synchronously, so a tool
invoked a beat early WAITS for the session to come live — defense-in-depth against the
"session id missing" race — instead of returning a hard failure to the model.
Returns the id as soon as it's non-null (the common path resolves immediately, no delay), or
null if it's still unbound after SessionIdWaitTimeoutMs — or the channel was
Disposed in the meantime (Context goes null, so we stop waiting on a torn-down session).
ProtectedResolveMaps a verb to the native tool name it is reachable as (inverse of ResolveVerbForTool).
ProtectedResolveMaps a native tool name back to its verb: the descriptor's NativeToolName match first, then the
name minus ToolNamePrefix. Protected so a channel with an unusual naming scheme can
override it.
A saved state came back (a resumed session): hold it, and hand it to OnSurfaceBound when the component is
created. Returns false for anything that is not a JSON object.
The state of record: the last known state as JSON, or null before there is any.
ProtectedSurfaceA finished game is the channel's output.
ProtectedSurfaceWhat a picture of the channel is taken of (when the host registered a rasterizer and the user allows pixels).
ProtectedSurfaceThe component's outputs, as events. An output fired while the agent's play runs is attributed to the agent by the adapter.
The component is going away (the panel collapsed, or the session ended): keep its last state, let go of it.
Checks the current GetState snapshot against the descriptor's noun schemas and returns every violation (empty when valid). Useful in a channel's own tests; the runtime also runs it on every perception note when StrictStateChecks is on.
Tic-tac-toe as a realtime channel. See the file comment for the walkthrough.