Remote Browser Channel Guide

The Remote Browser channel lets a realtime agent drive a real, live browser while it talks — a sales agent running an interactive product demo, a support agent walking a user through a UI, a research agent showing what it found, or a trainer agent that demonstrates a task in a real app, then hands the user the wheel (“your turn — try X, Y, Z”) and watches + coaches.

It is an in-house MJ realtime channel (a sibling concept to a Realtime Bridge — see REALTIME_BRIDGES_GUIDE.md). Its viewport renders in an MJ console panel, or screen-shares into a Zoom/Teams meeting through a bridge’s ScreenOut track. Read this before touching anything under packages/AI/RemoteBrowser/ or before adding a browser backend.

Companion plan: plans/realtime/realtime-bridges-architecture.md §4d / §4d-i.

---

1. The one design idea — push everything generic, keep backends thin

Every browser backend (self-hosted Chrome and every browser-as-a-service) exposes the same primitive: a Chrome DevTools Protocol (CDP) endpoint. So the whole subsystem is built on one rule: do the browser work once, generically, in @memberjunction/computer-use; the Remote Browser layer only maps vocabulary and manages session lifecycle.

┌─────────────────────────────────────────────────────────────────────────────┐
│ @memberjunction/computer-use            GENERIC browser I/O + perception      │
│   PlaywrightBrowserAdapter: connectOverCDP, ExecuteAction (selector OR coord),│
│   CaptureScreenshot, StartScreencast, GetAccessibilitySnapshot, QueryElement, │
│   GetVisibleText, GetTitle, WaitForLoadState, MouseMove …                     │
└───────────────▲─────────────────────────────────────────────────────────────┘
                │ wraps + maps (one lossless mapper)
┌───────────────┴───────────────────────────────────────────────────────────────┐
│ @memberjunction/remote-browser-cdp      SHARED CDP kit                          │
│   CdpRemoteBrowserSession (implements IRemoteBrowserSession over the adapter),  │
│   mapRemoteBrowserAction / mapHumanInput, BaseCdpRemoteBrowserProvider,         │
│   ICdpSessionBackend                                                            │
└───────────────▲────────────────────────────────────────────────▲──────────────┘
                │ extends (AcquireSession + 3 hooks)              │ contracts
┌───────────────┴──────────────────────┐  ┌──────────────────────┴──────────────┐
│ 5 backend drivers (thin)             │  │ @memberjunction/remote-browser-base  │
│  remote-browser-selfhost             │  │  BaseRemoteBrowserProvider,           │
│  remote-browser-browserbase          │  │  RemoteBrowserEngineBase (registry),  │
│  remote-browser-steel                │  │  IRemoteBrowserSession, action/input  │
│  remote-browser-browserless          │  │  types, control modes/strategies,     │
│  remote-browser-hyperbrowser         │  │  IRemoteBrowserProviderFeatures        │
└──────────────────────────────────────┘  └──────────────────────────────────────┘
                │ resolved by ClassFactory (DriverClass)
┌───────────────┴───────────────────────────────────────────────────────────────┐
│ @memberjunction/remote-browser-server   RemoteBrowserEngine (composition over   │
│   RemoteBrowserEngineBase) + RemoteBrowserChannel (BaseRealtimeChannelServer):  │
│   tool vocabulary, perception, control arbiter, viewport→ScreenOut             │
└────────────────────────────────────────────────────────────────────────────────┘

Because the heavy lifting is generic, each of the 5 backends is ~one method + a service seam. Adding the sixth is trivial.

---

2. The registry table & capability gating

A backend is a row in AIRemoteBrowserProvider (MJ: AI Remote Browser Providers, migration V202606161000), mirroring AIBridgeProvider:

Column	Meaning
Name	Display name (e.g. Browserbase)
ProviderType	SelfHost (MJ orchestrates a Chrome container) or Service (a BaaS CDP endpoint)
DriverClass	ClassFactory key → the BaseRemoteBrowserProvider subclass
Status	Active / Disabled
SupportedFeatures	Strongly-typed JSON (IRemoteBrowserProviderFeatures, bound via JSONType → the generated SupportedFeaturesObject accessor)
DefaultControlMode	AgentOnly / ViewOnly / Collaborative (channel default; runtime-overridable)
ConfigSchema / Configuration	optional JSON Schema + backend config (no inline secrets)

IRemoteBrowserProviderFeatures (the capability matrix — omitted flag = unsupported):

• Control substrate: RawCdpControl (the universal CDP path), NativeAIControl (the backend’s own AI harness, e.g. Browserbase Stagehand).
• Viewing & collaboration: LiveView (a hosted live-view URL), HumanTakeover (route human input), ScreenStreaming (continuous viewport frames → ScreenOut).
• Operational: Stealth, ProxyEgress, SessionRecording, PersistentContext, MultiTab, FileDownloads, CaptchaSolving.

Two-layer gating (defense in depth), identical to bridges: the engine checks the flag before calling a capability-gated session method; the session/driver also re-checks and throws RemoteBrowserCapabilityNotSupportedError if a flag lied or a caller bypassed the gate.

---

3. Control modes vs control strategies (don’t conflate them)

These are orthogonal:

• Control MODE — who may drive (a channel/runtime policy):
  • AgentOnly — only the agent drives (a sales demo that never hands over).
  • ViewOnly — the agent drives; humans watch the live view but cannot take over.
  • Collaborative — a human can grab the wheel (the trainer agent: demonstrate, then “your turn”).
  • Lives as DefaultControlMode on the provider; the RemoteBrowserChannel overrides it per-channel and at runtime. A mode is only valid if the backend supports its prerequisites (HumanTakeover for Collaborative; LiveView for ViewOnly/Collaborative) — enforced by isControlModeSupported.
• Control STRATEGY — how the agent decides what to click (resolved by resolveControlStrategy):
  • ComputerUse (default, universal) — MJ’s own perception→action loop over CDP. Right for a realtime co-agent that is already a powerful brain emitting tool calls while it talks.
  • NativeAI (capability-gated by NativeAIControl) — delegate a high-level intent to the backend’s own harness (Stagehand Act, Hyperbrowser agent). An accelerator for heavy/robust automation; it runs its own model loop, so it’s offered, not the default for the talking agent.
  • The strategy is resolved and invoked by the goal-driven path (AchieveGoal → dispatchRemoteBrowserGoal) — see §9.

---

4. How to add a new browser backend

The whole driver is: subclass BaseCdpRemoteBrowserProvider, implement one method + three hooks, register, and seed a row.

Step 1 — Subclass & register

import { RegisterClass } from '@memberjunction/global';
import { BaseRemoteBrowserProvider, RemoteBrowserProviderContext } from '@memberjunction/remote-browser-base';
import { BaseCdpRemoteBrowserProvider, AcquiredCdpSession } from '@memberjunction/remote-browser-cdp';

@RegisterClass(BaseRemoteBrowserProvider, 'AcmeRemoteBrowser')
export class AcmeRemoteBrowser extends BaseCdpRemoteBrowserProvider {
    protected async AcquireSession(ctx: RemoteBrowserProviderContext): Promise<AcquiredCdpSession> {
        const s = await this.client().CreateSession(/* … */);   // your service SDK (injectable seam)
        return {
            CdpEndpoint: s.CdpEndpoint,                          // attach point for computer-use
            Backend: {
                GetLiveViewUrl: async () => s.LiveViewUrl,       // or throw if no LiveView
                InvokeNativeAIControl: (intent) => this.client().Act(s.Id, intent), // or throw
                Release: async () => this.client().EndSession(s.Id),
            },
        };
    }
}

That’s it. Connect/Disconnect, attaching PlaywrightBrowserAdapter over CDP, action mapping, screencast, human-takeover input, perception, and capability gating are all inherited from CdpRemoteBrowserSession.

Step 2 — Injectable service seam + Fake (so tests need no network)

Mirror the existing backends: a SetClientFactory(factory) static + a default factory that throws "bind the real Acme client" until production binds the real SDK. Tests inject a Fake client.

Step 3 — Seed the registry row

Add a row to metadata/ai-remote-browser-providers/.ai-remote-browser-providers.json with Name, ProviderType, DriverClass (AcmeRemoteBrowser), DefaultControlMode, and a SupportedFeatures JSON string reflecting exactly what your ICdpSessionBackend actually supports (don’t claim NativeAIControl if InvokeNativeAIControl throws). Push via mj sync.

---

5. The five shipped backends

Backend	Type	DriverClass	Distinctive capability
Self-Hosted Chrome	SelfHost	SelfHostRemoteBrowser	MJ-orchestrated Chrome container (IChromeContainerRunner seam); LiveView via an MJ-hosted viewer backed by the inherited screencast; no provider AI harness
Browserbase	Service	BrowserbaseRemoteBrowser	First-party Stagehand native-AI (Act); richest feature set
Steel	Service	SteelRemoteBrowser	Full session viewer + stealth/proxy/recording; no first-party AI harness
Browserless	Service	BrowserlessRemoteBrowser	Lean CDP-as-a-service, ViewOnly default; no takeover/native-AI
Hyperbrowser	Service	HyperbrowserRemoteBrowser	Agentic native-AI (RunAgentTask)

All are Active in the seed; like the bridge drivers, each binds its real provider SDK / container runner via a SetClientFactory / SetContainerRunnerFactory seam before production use.

---

6. What @memberjunction/computer-use provides (the generic engine)

The Remote Browser work deliberately enriched computer-use rather than hand-rolling browser control per backend (the “push to the generic level” principle). Additive capabilities now available to every computer-use consumer:

• Selector-aware actions — Click/Type/Scroll/Wait take an optional Selector; when set the adapter targets the matched element (robust DOM), else the existing coordinate/delta/duration path.
• Screencast — StartScreencast(onFrame)/StopScreencast() via CDP Page.startScreencast (monotonic frames + ack) — the source for the channel’s ScreenOut track.
• Tab audio capture — StartAudioCapture(onChunk)/StopAudioCapture() taps the playing <video>/<audio> element in-page (captureStream() + a webm-opus MediaRecorder) and emits monotonic AudioCaptureChunks — the source for the Remote Browser channel’s tab-audio stream (see §8d). No-op default on adapters that can’t capture.
• MouseMove action — completes the input vocabulary for hover / human-takeover.
• Perception — GetAccessibilitySnapshot() (token-efficient ARIA tree), QueryElement(selector) ({Exists,Visible,Text,BoundingBox}), GetVisibleText(), GetTitle(), WaitForLoadState().

The lossless mapRemoteBrowserAction (in the CDP kit) is the single place Base actions become computer-use actions — propagating both selector and coordinate paths so none of the adapter’s robustness is lost.

---

7. The server layer — engine & channel

@memberjunction/remote-browser-server:

• RemoteBrowserEngine (server-only BaseSingleton, composes over RemoteBrowserEngineBase.Instance exactly as AIBridgeEngine composes over AIBridgeEngineBase): resolves the provider row + driver, builds the RemoteBrowserProviderContext (features from SupportedFeaturesObject, control mode = override ?? DefaultControlMode, validated), starts/tracks live sessions, runs the control arbiter (agent⇄human handoff honoring the mode) and a janitor, and exposes the PipeScreencastToTrack seam the bridge ScreenOut track consumes.
• RemoteBrowserChannel extends BaseRealtimeChannelServer: the agent’s browser tool vocabulary (OpenUrl/Click/Type/Key/Scroll/Back/Forward/Wait/Screenshot/GetPageText), ExecuteServerTool mapping each to a RemoteBrowserAction over the live session (routing high-level intents to InvokeNativeAIControl when the strategy resolves to NativeAI), and the perception feed that lets the agent see the page it drives.

---

8. The realtime client surface — live view, agent perception, human takeover

The Remote Browser channel renders in the realtime overlay’s Browser tab (RemoteBrowserSurfaceComponent). Three layers sit on top of the action vocabulary, each capability-gated so a backend opts in purely by advertising a flag — no per-provider client code.

8a. Live view — CDP screencast (push), screenshot poll (fallback)

The surface shows the live page two ways, picked by the ScreenStreaming capability:

• Screencast (default for capable backends) — on BindSurface the channel calls the StartRemoteBrowserScreencast mutation; the server runs IRemoteBrowserSession.StartScreencast and publishes each RemoteBrowserScreencastFrame on the existing PUSH_STATUS_UPDATES_TOPIC for the user’s session (the same push channel as delegation progress — no new WS infra). The conversations service’s push subscription routes those frames to the channel → the surface paints them on a <canvas> (one reused Image + requestAnimationFrame, drop-old coalescing, outside the Angular zone). Stopped on UnbindSurface/Dispose. (LiveKit/WebRTC video is the planned efficiency upgrade behind the same surface.)
• Screenshot poll (fallback) — the original RemoteBrowserSnapshot query every 700ms, used only when StartRemoteBrowserScreencast reports Streaming: false (backend lacks ScreenStreaming).

8b. Agent perception — the visual interpreter (for non-omnimodal realtime models)

Realtime voice models (GPT Realtime, Grok Voice) are audio/text only — they cannot see the screenshot. So beyond the URL/Detail perception fed back after each action, the agent gets two vision-query tools (distinct from the navigate/click action path) that let it kinda see:

• browser_DescribePage — a concise text description of what’s currently visible.
• browser_LocateElement(description) — find a UI element by visual description (works on messy sites that lack good a11y/DOM structure) and get its approximate pixel centroid (x, y) so the agent can then browser_Click it.

Both relay to the InterpretRemoteBrowserPage server mutation, which CaptureScreenshots the viewport and runs the “Remote Browser Visual Interpreter” AI Prompt (pinned to a fast vision model, Gemini 3.1 Flash-Lite, via its MJ: AI Prompt Models association — tunable in metadata, not code) over the image, returning strict JSON { description, elements: [{ label, x, y, confidence }] }. This is the bridge to visual sites until realtime models become omnimodal; it complements (doesn’t replace) a11y/GetVisibleText perception.

8c. Human takeover (Collaborative mode)

The session resolves a control mode (AgentOnly / ViewOnly / Collaborative) and the engine runs a floor arbiter (agent⇄human). In Collaborative, the surface captures pointer/keyboard events on the live view, maps display coordinates → viewport coordinates, and relays them as RemoteBrowserHumanInput (pointer-move / pointer-click / key) → IRemoteBrowserSession.RouteHumanInput (capability-gated on HumanTakeover). See RelayRemoteBrowserHumanInput.

8d. Tab audio — browser → user (so a co-agent’s video/audio demo is HEARD)

The screencast shows the page; audio streaming plays its sound — when the co-agent opens a YouTube clip or any audio/video site, the user hears it. It is the audio sibling of the screencast, built one-for-one the same way:

	Screencast (video)	Audio
Base type	RemoteBrowserScreencastFrame	RemoteBrowserAudioChunk (DataBase64 + Codec/SampleRate/Channels/SequenceNumber)
Session contract	StartScreencast/StopScreencast	StartAudioStream/StopAudioStream
Server mutations	Start/StopRemoteBrowserScreencast	Start/StopRemoteBrowserAudioStream
Push envelope	type:'RemoteBrowserScreencastFrame'	type:'RemoteBrowserAudioChunk' (same PUSH_STATUS_UPDATES_TOPIC, scoped by sessionId)
Client render	<canvas> paint	RemoteBrowserAudioPlayer (MediaSource + audio/webm;codecs=opus → hidden <audio>)

v1 is browser → user only. The contract is shaped to allow user → browser (a virtual mic) later, but that is not built.

Capability gating is by BACKEND IMPLEMENTATION, not a metadata flag. Unlike ScreenStreaming (a generated IRemoteBrowserProviderFeatures flag), audio gates on whether the driver’s ICdpSessionBackend provides the optional StartAudioCapture(adapter, onChunk) hook. If it’s absent, CdpRemoteBrowserSession.StartAudioStream throws RemoteBrowserCapabilityNotSupportedError; the server catches it and reports Streaming: false (exactly like a non-streaming backend), and the client simply plays nothing. (Fast-follow, documented but not in v1: add an AudioStreaming? flag to the IRemoteBrowserProviderFeatures JSONType for UI discoverability — needs migration → CodeGen → KNOWN_REMOTE_BROWSER_FEATURE_KEYS → metadata seed.)

Self-host capture mechanism (the concrete first backend) — in-page, headless-safe, no OS audio device, no extension. Generic in @memberjunction/computer-use (PlaywrightBrowserAdapter.StartAudioCapture), so every CDP backend can reuse it: a capture agent is injected via Playwright page.exposeBinding('__mjAudioChunk', …)

• page.addInitScript(…). It watches the DOM for <video>/<audio> elements (initial scan + MutationObserver
• play/pause listeners), taps the first one that is actively playing with element.captureStream(), and feeds its audio tracks to an in-page MediaRecorder({ mimeType: 'audio/webm;codecs=opus' }) at a ~250ms timeslice. Each dataavailable blob is base64-encoded and posted back through the binding → mapped to an AudioCaptureChunk → RemoteBrowserAudioChunk. The agent restarts the recorder when the active element swaps / plays / pauses.

Documented limitations: only audio routed through a media ELEMENT is captured (covers YouTube and most video/audio sites). Pure Web-Audio-API sound (some games/apps) and DRM/EME media (where captureStream() is blocked) are NOT captured. The documented future path for full-fidelity / DRM audio is a server-side virtual audio sink (PulseAudio null-sink / macOS BlackHole + ffmpeg) feeding 'opus'/'pcm16' chunks through the same contract. As with the screencast, a binary/WebRTC transport is the future efficiency upgrade over base64-on-PUSH_STATUS (shared with the screencast path).

Speaker toggle. The surface shows a speaker on/off button in the live-view bar whenever audio is streaming (default ON — the realtime call is the activating user gesture, so autoplay is allowed). Toggling it mutes/unmutes the RemoteBrowserAudioPlayer. Audio starts on BindSurface (alongside the screencast) and stops on UnbindSurface/Dispose.

---

9. Goal-driven browser control — set a goal, not granular clicks

§7’s tool vocabulary (browser_Click/browser_Type/…) lets a realtime agent drive the page one action at a time. That is the right granularity when the talking agent already knows exactly what to click — but for a multi-step chore (“log into this site and download the latest invoice”) it forces the realtime model to perceive, plan, and click step by step, which is slow and brittle on messy sites.

Goal-driven control hands a high-level goal to MJ’s computer-use loop, which plans and executes it autonomously against the same live browser the human is watching — using its own, independently-selected vision/action model (typically stronger at perception+planning than an audio realtime model). The realtime agent stays in its lane: conversation, intent, and narration.

9a. The strategy switch (now actually wired)

§3’s control strategy (ComputerUse vs NativeAI) was, until this feature, resolved but never invoked. The new entry points fill that gap:

browser_AchieveGoal (realtime tool)
  → ExecuteRemoteBrowserGoal (GraphQL mutation, @memberjunction/server)
    → RemoteBrowserEngine.AchieveGoal(agentSessionID, goal, opts)   [lazy-starts the session]
      → dispatchRemoteBrowserGoal(session, features, goal, opts)    [pure, testable strategy switch]
        ├─ 'ComputerUse' → IRemoteBrowserSession.RunComputerUseGoal(goal, options)   [the default, universal path]
        └─ 'NativeAI'    → IRemoteBrowserSession.InvokeNativeAIControl(intent)        [backend harness, capability-gated]

resolveControlStrategy(features, preferredStrategy) picks the lane: NativeAI only when the backend advertises NativeAIControl and the caller didn’t pin ComputerUse; otherwise the universal ComputerUse loop. The whole switch is the pure function dispatchRemoteBrowserGoal(), unit-tested with a fake session — the engine just lazy-starts the browser and calls it.

9b. ComputerUse runs on the session’s OWN adapter (no second browser)

CdpRemoteBrowserSession.RunComputerUseGoal drives computer-use against the same PlaywrightBrowserAdapter instance the human watches (the one already attached over the session’s CDP endpoint) — so the goal loop and the live view/screencast are the same page. The engine is injected behind the ComputerUseGoalRun seam (CdpRemoteBrowserSession.SetGoalEngineFactory): tests bind a fake (no browser, no LLM), production binds the MJ engine (§9e). Per-step progress flows out via OnProgress (a voice session narrates it); barge-in flows in via Signal → the engine’s cooperative Stop().

9c. Model-blind credentials — the model emits the LABEL, never the VALUE

Secrets ride a separate channel from the prompt. The goal run carries a Context object; the goal and the controller’s actions reference values by label — e.g. “log in using {{creds.username}} / {{creds.password}}”. The real value is substituted at the CDP keystroke boundary and nowhere else:

• wrapAdapterWithContext(adapter, context) (in @memberjunction/remote-browser-cdp) returns a proxy adapter. On ExecuteAction, it resolves {{label}} tokens to real values in a CLONE of the action (resolveActionTemplates) and passes the clone to the real adapter.
• The original action — the one the computer-use engine recorded in its step history (and that shows up in logs/transcripts) — keeps the {{label}}. So neither the realtime model nor the computer-use controller model ever holds the secret value; redaction falls out for free.

This mirrors the MJ agents context-variable pattern (resolveValueFromContext / getValueFromPath in @memberjunction/ai-agents), re-implemented minimally in the CDP package so it needs no agents dependency. Use it for credentials and any other sensitive run-scoped data.

9d. Collaborative pause — the human grabbing the wheel stops the goal

In Collaborative mode (§3, §8c), a granted human takeover must not race the autonomous loop on the shared browser. RemoteBrowserEngine.AchieveGoal registers an AbortController per session (chained to the caller’s Signal); a successful GrantControl(sessionId, 'Human') — and EndSession — abort it, so the computer-use loop pauses cooperatively via Stop(). An Agent grant is unconditional and never pauses the agent’s own goal.

9e. Controller/judge prompt + model selection (the default flip)

The default ProgressComputerUseEngine (base computer-use) has no model selection, so it can’t run a goal unsupervised. At server startup BindRemoteBrowserGoalEngine() (in @memberjunction/server, agentSessions/remoteBrowserGoalEngine.ts) binds the seam to MJProgressComputerUseEngine extends MJComputerUseEngine — which routes controller/judge prompts through AIPromptRunner, persists step media, forwards progress, and runs them as the session’s user (the acting ContextUser flows base → cdp → engine).

The default flip (golden source of truth): when the caller pins neither a prompt nor a model, MJComputerUseEngine.Run now defaults the controller + judge to the stored Computer Use - Controller / Computer Use - Judge metadata prompts (constants DEFAULT_CONTROLLER_PROMPT_NAME / DEFAULT_JUDGE_PROMPT_NAME). Those prompts carry both the prompt text and the model selection — by default Gemini 3.1 Flash-Lite → Gemini 3.5 Flash → Claude Haiku 4.5 → GPT 5.5 (the prompt’s MJ: AI Prompt Models rows, highest Priority first, each on two vendors for failover). So the goal loop runs through the full MJ prompt stack with metadata-configured models, not a code prompt. Swap the model by editing the prompt’s bindings in metadata — no code change.

Resolution order: explicit ControllerPromptRef/ControllerModel (caller override) → the stored default prompt (the flip) → autoSelectControllerModel() (legacy fallback: highest-PowerRank LLM advertising Image input, via pickHighestPowerVisionLLM, then plain highest-power LLM). The default lookup is non-throwing, so a missing stored prompt (e.g. a standalone/no-metadata caller) degrades cleanly to auto-selection and never hard-fails.

Single source of truth for the prompt text: the behavioral core of the controller (the Available Actions catalog + Response Format/Rules) and judge prompts lives once in metadata/prompts/templates/computer-use/_includes/*.md, pulled into the Layer-2 metadata templates via the push-time {@include} directive and generated into the Layer-1 (@memberjunction/computer-use) standalone fallback by its prebuild (scripts/generate-prompt-parts.mjs → prompt-parts.generated.ts). A drift-guard test (prompt-single-source.test.ts) asserts both layers carry the same shared blocks.

9f. Run-step observability — one “Browser goal” step, prompt sub-steps nested under it

A single goal fans out into many prompt runs (a controller call per perceive-act step, plus judge calls). Rather than flatten those under the realtime co-agent run, the goal is modeled as one parent step with child sub-steps — exactly the ParentID shape BaseAgent uses for loop iterations:

co-agent run (realtime)
 └─ ▸ Browser goal: "log into portal…"      MJ: AI Agent Run Steps · StepType 'Tool'   (parent)
      ├─ Prompt: Controller   step 'Prompt', TargetID = prompt def, TargetLogID = its AIPromptRun
      ├─ Prompt: Controller
      └─ Prompt: Judge

• The realtime layer (ExecuteRemoteBrowserGoal resolver) reads the session’s coAgentRunID from AIAgentSession.Config_, creates the parent Tool step (beginBrowserGoalStep), threads AgentRunID + the parent step id into AchieveGoal, and finalizes the step from the result.
• MJComputerUseEngine (via AgentRunStepTracker) creates a child Prompt step per controller/judge prompt under that parent, stamping TargetLogID with the produced AIPromptRun id.
• The whole thing is best-effort: no co-agent run (or any failure) just means the goal runs unlinked — it never aborts the goal.
• Fire-and-forget, off the hot loop: the per-prompt INSERT/UPDATE step writes are queued (not awaited) via AgentRunStepSaveQueue and flushed once when the goal completes — so an N-iteration goal never pays N synchronous DB round-trips on its critical path (which matters on slower databases). This is the exact pattern BaseAgent uses for its own steps.
• Single source of truth: both the step field semantics (initAgentRunStep / finalizeAgentRunStep) AND the fire-and-forget save orchestration (AgentRunStepSaveQueue) live once in @memberjunction/ai-core-plus; BaseAgent and the Computer Use tracker both delegate to them rather than duplicating the create/finalize/persist logic.

---

10. Reference map

Package	Role
@memberjunction/computer-use	Generic browser I/O + perception (CDP, selectors, screencast, tab-audio capture, a11y)
@memberjunction/remote-browser-base	Universal contracts + RemoteBrowserEngineBase registry cache; goal types (RunComputerUseGoalOptions, RemoteBrowserGoalResult)
@memberjunction/remote-browser-cdp	Shared CDP session kit (the DRY heart) + lossless mapper; RunComputerUseGoal, the ComputerUseGoalRun seam, model-blind context-injection
@memberjunction/remote-browser-{selfhost,browserbase,steel,browserless,hyperbrowser}	The 5 backends
@memberjunction/remote-browser-server	RemoteBrowserEngine (incl. AchieveGoal + pure dispatchRemoteBrowserGoal) + RemoteBrowserChannel
@memberjunction/computer-use-engine	MJComputerUseEngine — defaults controller/judge to the stored Computer Use - Controller/- Judge prompts (§9e), with pickHighestPowerVisionLLM auto-selection as the fallback; AgentRunStepTracker (child prompt sub-steps)
@memberjunction/ai-core-plus	Shared single-source-of-truth step helpers initAgentRunStep / finalizeAgentRunStep (used by BaseAgent AND the Computer Use tracker)
@memberjunction/server	ExecuteRemoteBrowserGoal mutation + BindRemoteBrowserGoalEngine (binds MJProgressComputerUseEngine); beginBrowserGoalStep/finalizeBrowserGoalStep (parent goal step)
migration V202606161000	AIRemoteBrowserProvider registry table
metadata/ai-remote-browser-providers/	The 5 backend seed rows

Goal-driven design doc: plans/realtime/computer-use-remote-browser-blend.md.

See also: REALTIME_BRIDGES_GUIDE.md (the sibling bridge subsystem — the Remote Browser screen-shares into meetings through a bridge’s ScreenOut track).