# Realtime Bridges Guide

How MemberJunction connects its **one** realtime agent engine to the outside world — Zoom/Teams/
Slack/Meet/Webex/Discord **meetings** and Twilio/Vonage/RingCentral **telephony** — through a single,
pluggable **media-transport seam**.

There is exactly **one** realtime agent engine (the `Realtime` agent type, the co-agent resolution
chain, tools, narration, channels, transcript relay, the `AIAgentSession` lifecycle — all documented in
the [Real-Time Co-Agents Guide](REALTIME_CO_AGENTS_GUIDE.md)). A **bridge** is a pluggable media
transport that fills that engine's long-deferred *server-bridged* seam from an external endpoint. We do
not build a second realtime stack; a bridge is just a driver that supplies the media plane.

> **Nothing in this architecture is audio-specific.** Audio is the first track lit up because the
> realtime models are there today, but the seam carries typed media tracks (audio / video / screen,
> each inbound *and* outbound). When realtime models gain full-duplex video, the same bridges already
> carry the video tracks with zero re-architecture.

**Companion documents:**

- [`/plans/realtime/realtime-bridges-architecture.md`](../plans/realtime/realtime-bridges-architecture.md)
  — the full architecture (mermaid, ERD, all phases, channels, multi-party, turn-taking). This guide
  is the developer-facing distillation of what has **shipped**; the plan retains the full rationale and
  the planned/deferred tracks.
- [`/guides/REALTIME_CO_AGENTS_GUIDE.md`](REALTIME_CO_AGENTS_GUIDE.md) — the realtime engine the bridge
  plugs into (the `Realtime` agent type, `IRealtimeSession`, channels, sessions, the janitor pattern
  bridges copy).
- [`packages/AI/BridgeBase/README.md`](../packages/AI/BridgeBase/README.md) — the base package
  (`@memberjunction/ai-bridge-base`).
- [`packages/AI/Bridge/README.md`](../packages/AI/Bridge/README.md) — the server package
  (`@memberjunction/ai-bridge-server`).

---

## Table of Contents

1. [What a Bridge Is](#1-what-a-bridge-is)
2. [Architecture](#2-architecture)
3. [The Capability Model](#3-the-capability-model)
4. [How to Add a New Bridge Driver](#4-how-to-add-a-new-bridge-driver)
5. [The Transport Seam](#5-the-transport-seam)
6. [Turn-Taking](#6-turn-taking)
7. [Entity Invariants](#7-entity-invariants)
8. [Join Methods & Agent Identity (Planned)](#8-join-methods--agent-identity-planned)
9. [Multi-party (LiveKit + multiple agents)](#9-multi-party-livekit--multiple-agents)
10. [Roadmap / Phase Status](#10-roadmap--phase-status)

---

## 1. What a Bridge Is

A bridge is a **media transport + (planned) channel contributor** that connects the realtime agent
engine to an external endpoint. Concretely it is a driver class (`LoopbackBridge` today; `ZoomBridge`,
`TwilioBridge`, … as drivers land) that:

- **carries bidirectional media** — audio, video, screen, full duplex; nothing is audio-specific; and
- (planned, Phase 2) **contributes channels** — the platform's native surfaces (hand-raise, roster,
  whiteboard) routed through MJ's existing channel plane.

The single biggest design win is **reuse**. The realtime session contract `IRealtimeSession` (in
`@memberjunction/ai`) is already media-agnostic — its `SendInput(chunk)` is *what the agent hears* and
`OnOutput(handler)` is *what the agent says* — but it had **no client-facing pipe**. A bridge **is**
that pipe.

> **🚨 A bridge is transport + host UX tools ONLY — it does NOT build the agent.** Identity (the agent
> speaks first-person AS the target), the model/voice **precedence cascade**, the base prompt + memory, the
> `invoke-target-agent` delegation, and session/run tracking all come from the **one** core producer,
> `RealtimeClientSessionService.PrepareRealtimeSessionParams`, which the bridge *consumes* (see
> `BaseAgent.StartBridgeRealtimeSession`). A bridge that re-implements any of that is drift — exactly what
> the convergence in [`plans/realtime/realtime-core-host-convergence.md`](../plans/realtime/realtime-core-host-convergence.md)
> removed (and what `ai-agents/src/__tests__/realtime-convergence-drift.test.ts` guards). What a bridge DOES
> own: media transport (audio/video/screen) and any **host-specific** UX tools, injected via
> `RealtimeSessionRunnerDeps.ExtraTools`/`ExecuteTool` — never baked into core. So the agent is the *same
> agent* (identity, voice, delegation, tracking) whether you reach it in native chat, LiveKit, or Zoom/Teams;
> only how it's heard and what surfaces it can touch differ. See the boundary table in the
> [Co-Agents Guide §3](REALTIME_CO_AGENTS_GUIDE.md#-single-source-of-truth-one-prep-the-precedence-cascade-the-corehost-boundary).

### The transport seam

```
external endpoint                 the bridge driver                the realtime session
(Zoom / Twilio / …)               (BaseRealtimeBridge)             (IRealtimeSession)
        │   inbound media   ──▶   OnMedia(frame)         ──▶   SendInput(chunk)   "the agent hears"
        │   outbound media  ◀──   SendMedia(track,frame) ◀──   OnOutput(chunk)    "the agent says"
```

The engine wires those two hooks once, generically (see [§5](#5-the-transport-seam)). Meetings and
telephony are two families of the **same** abstraction — and because the session *is* the existing
`AIAgentSession`, a bridged session reviews in the existing console with zero new persistence code.

### No new "session" entity

The session is the existing `AIAgentSession`. A bridge is an **attachment** to it (the
`AIAgentSessionBridge` row), exactly parallel to how a channel attaches to a session. One session
record, one lifecycle, one transcript path — whether the media plane is a browser, a Zoom room, or a
phone line.

### A bridge is also a channel contributor (forward reference)

Beyond media, a bridge is *also* a contributor of **channels** — the platform's native tool +
signaling vocabulary (hand-raise, roster, native whiteboard) routed through MJ's existing channel
plane. That server-side channel plane is **planned (Phase 2)** and is not in the shipped code; see
[§10](#10-roadmap--phase-status). This guide documents the media transport + capability + turn-taking +
entity layers that **have** shipped (Phase 0/1).

---

## 2. Architecture

### The engine pair — `AIBridgeEngineBase` / `AIBridgeEngine`

The bridge plane mirrors `AIEngineBase` / `AIEngine` exactly.

```mermaid
graph TD
    subgraph Meta["Metadata (CodeGen entities, seeded via mj-sync)"]
        P["MJ: AI Bridge Providers<br/>registry + SupportedFeatures"]
        ID["MJ: AI Bridge Agent Identities<br/>agent's address / phone #"]
        PC["MJ: AI Bridge Provider Channels<br/>provider → channel junction"]
        SB["MJ: AI Agent Session Bridges<br/>session ↔ transport binding"]
        PART["MJ: AI Agent Session Bridge Participants<br/>who's on the call"]
    end
    subgraph Base["@memberjunction/ai-bridge-base (metadata only, no execution)"]
        EB["AIBridgeEngineBase<br/>(BaseEngine) — caches providers,<br/>identities, provider-channels"]
        BBP["BaseRealtimeBridge (abstract)<br/>media contract + capability-gated methods"]
        TT["TurnTakingPolicy<br/>passive / active / hybrid"]
    end
    subgraph Server["@memberjunction/ai-bridge-server (coordination + execution)"]
        ENG["AIBridgeEngine<br/>composes the base — lifecycle,<br/>host affinity, janitor, transport seam"]
        LB["LoopbackBridge<br/>(in-memory test driver)"]
    end
    RS["IRealtimeSession<br/>(@memberjunction/ai — injected, never constructed)"]

    Meta --> EB
    EB -.composed by.-> ENG
    BBP -.implemented by.-> LB
    TT --> ENG
    ENG <-->|"transport seam<br/>SendInput / OnOutput"| RS
    ENG -->|"ClassFactory.CreateInstance(BaseRealtimeBridge, DriverClass)"| LB

    style Base fill:#1f3a5f,stroke:#264f7a,color:#fff
    style Server fill:#5a3d7a,stroke:#7c5295,color:#fff
    style Meta fill:#2d5016,stroke:#1a5c3a,color:#fff
```

**`AIBridgeEngineBase`** (`@memberjunction/ai-bridge-base`) — a `BaseEngine` subclass that caches the
three bridge reference datasets and exposes synchronous resolution helpers over them. **No execution.**
Because the datasets are small reference tables, every lookup filters the in-memory array client-side
(no `RunView`s). It is `@RegisterForStartup()` and reactive via the inherited `BaseEngine` event
integration. Key helpers:

| Helper | Purpose |
|---|---|
| `ProviderByName(name)` / `ProviderByDriverClass(driverClass)` | Resolve a provider row (case-insensitive, trim-tolerant). |
| `IdentityByValue(value, providerId?)` | Route an inbound invite/call (addressed to a mailbox/DID) to the owning agent. Active identities only. |
| `IdentitiesForAgent(agentId, providerId?)` | The agent's active identities (which surfaces it is reachable on). |
| `ChannelsForProvider(id)` / `DefaultChannelsForProvider(id)` | The channels a provider contributes / auto-attaches, ordered by `Sequence`. |

**`AIBridgeEngine`** (`@memberjunction/ai-bridge-server`) — the server tier that **composes** the base
(it does **not** extend it) and adds all coordination + execution: the transport seam, the bot-session
lifecycle state machine, host affinity + the janitor, participant tracking, and turn-taking
integration.

#### Why composition, not inheritance

`AIBridgeEngineBase` is a `BaseEngine` singleton keyed by class name. If `AIBridgeEngine` *inherited*
from it, the startup manager would instantiate **two** separate engines (the base AND the subclass),
each loading and event-subscribing its own copy of the same caches. By **composing** —
`AIBridgeEngine` reads through `protected get Base() => AIBridgeEngineBase.Instance` and delegates every
metadata accessor — there is exactly **one** `BaseEngine` cache, warmed once. This is the identical
pattern `AIEngine` uses over `AIEngineBase`. From the architecture's source:

```typescript
@RegisterForStartup({ deferred: true, deferredDelay: 15000, description: 'Server-side AI Bridge Engine' })
export class AIBridgeEngine extends BaseSingleton<AIBridgeEngine> implements IStartupSink {
    /** The single underlying base instance whose BaseEngine cache holds the registry. */
    protected get Base(): AIBridgeEngineBase {
        return AIBridgeEngineBase.Instance;
    }
    public async HandleStartup(contextUser?, provider?) { await this.Config(false, contextUser, provider); }
    public async Config(forceRefresh?, contextUser?, provider?) { await this.Base.Config(forceRefresh, contextUser, provider); }
    // ...every metadata read delegates: ProviderByName, IdentityByValue, ChannelsForProvider, …
}
```

### The `BaseRealtimeBridge` driver family

`BaseRealtimeBridge` is the abstract driver — a **sibling to `BaseRealtimeModel`** in the realtime
stack. Where a realtime *model* driver wraps a speech-to-speech provider, a bridge *driver* wraps a
media transport. Both self-register with the MJ `ClassFactory` and are resolved by metadata key. The
base class is abstract and unregistered; the engine resolves a concrete driver via
`MJGlobal.Instance.ClassFactory.CreateInstance(BaseRealtimeBridge, provider.DriverClass)`. See
[§4](#4-how-to-add-a-new-bridge-driver).

### The five entities

All five are CodeGen-generated, named with the `MJ:` prefix, and seeded via mj-sync metadata (not SQL
INSERTs). The migration created the tables; CodeGen produced the entity classes in
`@memberjunction/core-entities`.

```mermaid
erDiagram
    AIAgent ||--o{ AIBridgeAgentIdentity : "reachable as"
    AIBridgeProvider ||--o{ AIBridgeAgentIdentity : "on platform"
    AIBridgeProvider ||--o{ AIAgentSessionBridge : "transports via"
    AIBridgeProvider ||--o{ AIBridgeProviderChannel : "contributes"
    AIAgentChannel ||--o{ AIBridgeProviderChannel : "as channel"
    AIAgentSession ||--o| AIAgentSessionBridge : "bridged by"
    AIAgentSessionBridge ||--o{ AIAgentSessionBridgeParticipant : "has"
    User ||--o{ AIAgentSessionBridgeParticipant : "is (if matched)"
```

| Entity (name) | Generated class | Role |
|---|---|---|
| `MJ: AI Bridge Providers` | `MJAIBridgeProviderEntity` | The platform registry. Holds `Name`, `DriverClass` (the `ClassFactory` key), `Status`, and `SupportedFeatures` (the typed JSON capability blob, accessed via `SupportedFeaturesObject`). |
| `MJ: AI Bridge Agent Identities` | `MJAIBridgeAgentIdentityEntity` | Which agents are reachable where — `IdentityType` (`Email` / `PhoneNumber` / `AccountID`) + `IdentityValue`, per agent, per provider. |
| `MJ: AI Bridge Provider Channels` | `MJAIBridgeProviderChannelEntity` | The provider→`AIAgentChannel` junction (`IsDefault`, `Sequence`) — which channels a provider contributes (consumed by the planned Phase 2 channel plane). |
| `MJ: AI Agent Session Bridges` | `MJAIAgentSessionBridgeEntity` | The transport attachment to an `AIAgentSession`. The lifecycle state machine (`Status`), direction, join method, turn mode, platform handles, `HostInstanceID`. |
| `MJ: AI Agent Session Bridge Participants` | `MJAIAgentSessionBridgeParticipantEntity` | One row per participant on the call (the diarization map), including the agent bot itself (`IsAgent = 1`). |

Telephony reuses these **same five tables** with no schema change: `BridgeType = 'Telephony'`,
`IdentityType = 'PhoneNumber'`, and a `SupportedFeatures` blob with `{ DTMF, OutboundDial, CallTransfer }`
on. That is the proof the model is genuinely unified.

---

## 3. The Capability Model

Every bridge is **100% generic and capability-gated**. A platform's capabilities are declared in
`MJ: AI Bridge Providers.SupportedFeatures` — a single `NVARCHAR(MAX)` JSON column strongly typed via
the `IBridgeProviderFeatures` interface (MJ's JSONType system; CodeGen emits the typed
`SupportedFeaturesObject` accessor). One JSON column instead of ~16 BIT columns keeps the table simple
and lets new features be added without a schema migration — just extend the interface.

### `IBridgeProviderFeatures`

The shipped interface (`metadata/entities/JSONType-interfaces/IBridgeProviderFeatures.ts`) holds
**transport/media** concerns only. All flags are optional — an omitted flag means **not supported**.

```typescript
export interface IBridgeProviderFeatures {
    // Join methods
    OnDemandJoin?: boolean; ScheduledJoin?: boolean; InviteJoin?: boolean;
    NativeInvite?: boolean; InboundRouting?: boolean; OutboundDial?: boolean;
    // Media tracks (directional — nothing here is audio-specific)
    AudioIn?: boolean;  AudioOut?: boolean;
    VideoIn?: boolean;  VideoOut?: boolean;
    ScreenIn?: boolean; ScreenOut?: boolean;
    // Signals & telephony
    SpeakerDiarization?: boolean; DTMF?: boolean; CallTransfer?: boolean; Recording?: boolean;
}
```

> Interactive surfaces (hand-raise, in-meeting chat, native whiteboard) are deliberately **not**
> features here — they are **channels** the bridge contributes (planned Phase 2). This interface is
> transport/media capabilities only.

In bridge code, `IBridgeProviderFeatures` is available as a documented type alias from
`@memberjunction/ai-bridge-base` (it aliases the generated `MJAIBridgeProviderEntity_IBridgeProviderFeatures`
so drivers needn't import the verbose generated name; the canonical source remains
`@memberjunction/core-entities`).

### The two-layer gating rule (defense in depth)

```mermaid
flowchart TD
    A["Engine wants SendDTMF()"] --> B{"provider.SupportedFeaturesObject.DTMF?"}
    B -->|"false / omitted (metadata)"| C["skip — never call the driver"]
    B -->|"true"| D["driver.SendDTMF()"]
    D --> E{"driver overrode it?"}
    E -->|"no — base rejects with<br/>BridgeCapabilityNotSupportedError"| F["loud failure: the flag lied"]
    E -->|"yes"| G["RequireFeature('DTMF') re-asserts, then sends"]

    style C fill:#2d5016,stroke:#1a5c3a,color:#fff
    style F fill:#7a1f1f,stroke:#a02a2a,color:#fff
    style G fill:#2d5016,stroke:#1a5c3a,color:#fff
```

1. **The engine checks the flag first** and never calls a driver method whose feature is off. For
   example, participant tracking is gated before the driver is ever touched:
   ```typescript
   private wireParticipantTracking(active: ActiveBridgeSession): void {
       if (active.Provider.SupportedFeaturesObject?.SpeakerDiarization !== true) {
           return; // provider metadata says no roster — never call the driver.
       }
       try { active.Bridge.OnParticipantChange(/* … */); }
       catch (err) { /* driver didn't implement it despite the flag — log + continue */ }
   }
   ```
2. **The driver's virtual methods throw by default** (`BridgeCapabilityNotSupportedError`). This is the
   second layer: a driver that has not overridden a method — or a metadata flag that *lied* — fails
   loudly rather than silently degrading. An overriding driver re-asserts the flag at the top of its
   method via the protected `RequireFeature(flag)` helper, so even a direct (engine-bypassing) caller
   cannot run an action the metadata says is off.

`BridgeCapabilityNotSupportedError` carries both `FeatureName` and `ProviderName`, so the loud failure
points at exactly which capability flag lied (or which driver method is missing) for which platform.

---

## 4. How to Add a New Bridge Driver

This is the most important section. Adding a platform is a **minimal subclass** — the base + engine
carry all the generic weight (session wiring, frame normalization, turn-taking, participant bookkeeping,
lifecycle, the janitor). You implement only the irreducibly platform-specific primitives.

### Step 1 — Subclass `BaseRealtimeBridge` and register it

```typescript
import { RegisterClass } from '@memberjunction/global';
import { BaseRealtimeBridge } from '@memberjunction/ai-bridge-base';

@RegisterClass(BaseRealtimeBridge, 'ZoomBridge')   // ← the DriverClass key the provider row resolves
export class ZoomBridge extends BaseRealtimeBridge {
    // ...
}
```

The `'ZoomBridge'` string is the `ClassFactory` key. A `MJ: AI Bridge Providers` row whose
`DriverClass = 'ZoomBridge'` resolves to this class. (Add a tree-shaking-prevention loader and call it
from your package entry point, exactly as `LoadLoopbackBridge` does, so bundlers don't eliminate the
dynamically-registered class.)

### Step 2 — Implement the four abstract methods (every bridge MUST)

```typescript
public async Connect(ctx: RealtimeBridgeContext): Promise<BridgeConnectResult> {
    this.applyContext(ctx);                       // capture features + provider name FIRST
    // … join the meeting / place or accept the call …
    return { BotParticipantId, ExternalConnectionId };   // engine persists these onto the bridge row
}

public async Disconnect(reason: BridgeDisconnectReason): Promise<void> {
    // leave / hang up cleanly. `reason` lets you shortcut a graceful handshake on 'Error'/'Shutdown'.
}

public SendMedia(track: BridgeMediaTrackKind, frame: BridgeMediaFrame): void {
    // outbound: the agent's voice/video/screen INTO the meeting (fed from IRealtimeSession.OnOutput).
    // Audio is just one track — `frame.Track` selects which.
}

public OnMedia(handler: (frame: BridgeMediaFrame) => void): void {
    // inbound: what the agent HEARS/SEES (routed to IRealtimeSession.SendInput).
    // When the platform diarizes, stamp `frame.SpeakerLabel` so turn-taking can attribute speech.
}
```

`applyContext(ctx)` (a protected helper) captures the provider's `SupportedFeatures` and name onto the
instance so `RequireFeature` and capability-error messages work. **Call it first in `Connect`.**

### Step 3 — Override only the capability-gated virtuals your platform supports

Each optional method throws `BridgeCapabilityNotSupportedError` by default. Override the ones your
platform offers, and **re-assert the flag** at the top with `RequireFeature`:

| Method | Gating flag | Family |
|---|---|---|
| `GetParticipants()` / `OnParticipantChange(handler)` | `SpeakerDiarization` | roster + diarization mapping |
| `SendDTMF(digits)` / `OnDTMF(handler)` | `DTMF` | telephony |
| `TransferCall(target)` | `CallTransfer` | telephony |
| `StartRecording()` | `Recording` | recording (subject to consent handling upstream) |

Video/screen tracks need no extra methods — they ride the same `SendMedia` / `OnMedia` calls, gated by
the directional flags (`VideoIn/Out`, `ScreenIn/Out`).

### Step 4 — Set the matching `SupportedFeatures` flags in the provider seed

The driver advertises what it overrode; the provider's `SupportedFeatures` JSON **must match** — and is
validated server-side ([§7](#7-entity-invariants)). Seed the provider row (with its capability blob)
via mj-sync metadata, never a SQL INSERT.

### Worked minimal example — `LoopbackBridge`

The shipped `LoopbackBridge` (`@memberjunction/ai-bridge-server`) is the canonical minimal driver. It
proves the transport seam round-trips media with **no platform**: every outbound frame is echoed
straight back inbound (re-stamped `audio-out → audio-in`), and it overrides the diarization-gated roster
methods to return a single synthetic agent participant.

```typescript
@RegisterClass(BaseRealtimeBridge, LOOPBACK_BRIDGE_DRIVER_CLASS)   // 'LoopbackBridge'
export class LoopbackBridge extends BaseRealtimeBridge {
    public async Connect(ctx: RealtimeBridgeContext): Promise<BridgeConnectResult> {
        this.applyContext(ctx);
        this.connected = true;
        this.participantHandler?.([LoopbackBridge.AGENT_PARTICIPANT]);   // emit initial roster
        return { BotParticipantId: 'loopback-agent', ExternalConnectionId: `loopback:${ctx.Address || 'in-memory'}` };
    }

    public SendMedia(track: BridgeMediaTrackKind, frame: BridgeMediaFrame): void {
        if (!this.connected) return;                                     // disconnected → drop
        this.Sent.push(frame);
        this.mediaHandler?.({ ...frame, Track: this.toInboundTrack(track) });   // echo outbound → inbound
    }

    public OnMedia(handler: (frame: BridgeMediaFrame) => void): void { this.mediaHandler = handler; }

    public override async GetParticipants(): Promise<BridgeParticipantInfo[]> {
        this.RequireFeature('SpeakerDiarization');                       // defense-in-depth re-assert
        return [LoopbackBridge.AGENT_PARTICIPANT];
    }

    public override OnParticipantChange(handler: (p: BridgeParticipantInfo[]) => void): void {
        this.RequireFeature('SpeakerDiarization');
        this.participantHandler = handler;
    }
    // SendDTMF / TransferCall / StartRecording stay un-overridden → still throw NotSupported.
}
```

Telephony/recording features stay off, so those virtual base methods keep throwing — the loopback
driver documents the capability-gating contract by example. A real driver (e.g. `ZoomBridge`, **planned
Phase 3**) lives in its own package or the server package and overrides the diarization + (for
telephony, planned Phase 6) DTMF/transfer methods, translating between the platform's media SDK and
`BridgeMediaFrame`.

### Media-track translation & diarization

A real driver translates the platform's media SDK frames to/from `BridgeMediaFrame`:

- **Payload:** populate exactly one of `Bytes` (raw `ArrayBuffer`, the fast in-process path) **or**
  `Base64` (for transports that can only carry text). The engine's seam unwraps either.
- **Track + direction:** stamp `Track` (`audio-in` … `screen-out`). Direction is from the agent's POV:
  `*-in` flows from the endpoint to the agent; `*-out` from the agent to the endpoint.
- **Diarization:** when the platform labels speakers, stamp `SpeakerLabel` on inbound frames and emit
  `BridgeParticipantInfo` rosters via `OnParticipantChange`. The engine upserts those to
  `AIAgentSessionBridgeParticipant` rows and turn-taking uses the labels.

---

## 5. The Transport Seam

The transport seam is the single piece of code that completes the realtime engine's deferred
server-bridged media transport — the "`SendInput`/`OnOutput` have no client-facing pipe" gap. The
bridge **is** that pipe. It is wired in `AIBridgeEngine.StartBridgeSession` → `wireTransportSeam`.

### Injection — the engine never constructs the model

The realtime session is **injected** into `StartBridgeSession`, never constructed by the engine. This
keeps the only coupling the seam itself (to the `IRealtimeSession` *interface*), keeps the model's
lifecycle owned by the agent/session layer above, and makes the whole engine unit-testable with a mock
session + the `LoopbackBridge`.

```typescript
export interface StartBridgeSessionParams {
    AgentSessionID: string;              // the existing AIAgentSession this bridge attaches to
    Provider: MJAIBridgeProviderEntity;  // DriverClass resolves the driver
    RealtimeSession: IRealtimeSession;   // ← INJECTED — the engine wires its media plane, never builds it
    Address: string;                     // join URL/ID (meetings) or phone number (telephony)
    Direction?: 'Inbound' | 'Outbound';  // default 'Outbound'
    JoinMethod?: /* … */;                // default 'OnDemand'
    TurnMode?: BridgeTurnMode;           // default 'Passive'
    TurnMatcher?: IAddressedMatcher; TurnScorer?: IWorthSayingScorer; TurnTuning?: /* … */;
    ContextUser?: UserInfo;
    MetadataProvider?: IMetadataProvider;
}
```

### Wiring the two directions

```typescript
private wireTransportSeam(active: ActiveBridgeSession): void {
    const { Bridge, RealtimeSession } = active;

    // Inbound: endpoint media → the agent HEARS it.
    Bridge.OnMedia((frame: BridgeMediaFrame) => {
        const chunk = this.frameToArrayBuffer(frame);   // unwrap Bytes (fast) or decode Base64
        if (chunk) RealtimeSession.SendInput(chunk);
    });

    // Outbound: the agent SPEAKS → into the meeting/call.
    RealtimeSession.OnOutput((chunk: ArrayBuffer) => {
        const track: BridgeMediaTrackKind = 'audio-out';
        Bridge.SendMedia(track, this.arrayBufferToFrame(chunk, track));
    });
}
```

Audio is the first track lit up, so the outbound direction is wired as `audio-out`; video/screen ride
the same `SendMedia` method once the model emits them. The seam's `frameToArrayBuffer` prefers the
binary `Bytes` payload and decodes `Base64` only when that's all the frame carries (environment-agnostic
— Node `Buffer` server-side, `atob` fallback).

### Lifecycle state machine

`StartBridgeSession` drives the `AIAgentSessionBridge.Status` state machine and stamps `HostInstanceID`
for node affinity:

```mermaid
stateDiagram-v2
    [*] --> Pending: createBridgeRow (HostInstanceID stamped)
    Pending --> Connecting: transitionStatus + driver resolved via ClassFactory
    Connecting --> Connected: driver.Connect succeeds → handles persisted, seam wired
    Connecting --> Failed: resolution or Connect throws → failBridgeRow
    Connected --> Disconnected: StopBridgeSession (driver.Disconnect + CloseReason)
    Failed --> [*]
    Disconnected --> [*]
```

The flow, from the source:

1. `createBridgeRow` — insert the `AIAgentSessionBridge` as `Pending`, stamped with this host.
2. `resolveDriver` — `ClassFactory.CreateInstance(BaseRealtimeBridge, provider.DriverClass)`. (It first
   checks `GetRegistration` exists, because `CreateInstance` falls back to the abstract base when no
   driver is registered — returning an unusable instance otherwise.)
3. `transitionStatus → 'Connecting'`, build the `RealtimeBridgeContext`, call `driver.Connect`.
4. On success: persist `ExternalConnectionID` + `BotParticipantID`, stamp `ConnectedAt`,
   `transitionStatus → 'Connected'`, then `wireTransportSeam` + `wireParticipantTracking` +
   `wireTurnTaking`, and register the live `ActiveBridgeSession` in-memory.
5. On failure: `failBridgeRow` stamps `Failed` / `CloseReason = 'Error'` / `DisconnectedAt`, and
   rethrows.

`StopBridgeSession(id, reason, …)` disconnects the driver (tolerant of teardown errors) and transitions
the row to `Disconnected` with the `CloseReason`. Both start and stop are **idempotent** — stopping a
session this process no longer holds still reconciles the durable row (`markBridgeDisconnected` treats
an already-terminal row as success).

### `HostInstanceID` + the janitor

Every bridge row is stamped with `HostInstanceID` (from an **injected** `IHostInstanceIdentity` — the
host app supplies the real one; `DefaultHostInstanceIdentity` is the standalone fallback). A crash or
redeploy vaporizes the in-memory driver sockets but leaves durable rows reading `Connected` forever.
`ReconcileOrphans(contextUser, provider)` force-closes `Connecting`/`Connected` bridges left by a
**prior boot of this host** (matching hostname prefix, differing instance id) with
`CloseReason = 'Janitor'` — mirroring the realtime `SessionJanitor`. The **scheduling** (run-once-at-boot
+ periodic sweep) is intentionally a host-provided hook, so this engine package carries no timer/IO of
its own; the host's janitor calls `ReconcileOrphans`.

---

## 5b. The Channel Plane — a bridge contributes channels

A bridge is not only a media pipe; it is also a **channel contributor** (§4b of the architecture).
`AIBridgeEngine` wires the Phase 2 server-side channel plane (`RealtimeChannelServerHost` +
`MeetingControlsChannelServer`, both in `@memberjunction/ai-agents`) to a bridged session **generically**
— without the bridge-server engine depending on the heavy agents package.

### The additive seam — `IBridgeChannelHost` + `GetMeetingControlsEventSource`

Two additive pieces make this decoupled and back-compatible:

- **`StartBridgeSessionParams.ChannelHost?: IBridgeChannelHost`** — an optional, session-scoped view of
  the channel plane (`StartSessionChannels` / `GetSessionServerTools` / `ExecuteSessionServerTool` /
  `CloseSessionChannels`), declared in `@memberjunction/ai-bridge-base`. The **runner-constructing layer**
  (which already depends on `@memberjunction/ai-agents`) supplies a thin adapter binding these to
  `RealtimeChannelServerHost`. When omitted, an `IRealtimeSession`-only caller works exactly as before —
  no channel wiring.
- **`BaseRealtimeBridge.GetMeetingControlsEventSource(): IBridgeMeetingControlsEventSource | null`** — a
  driver returns a facilitator event source (roster / hand-raise / speaking / mute) when it has one
  (default `null`). This is **not** a throwing capability gate — most drivers legitimately contribute
  none. `ZoomBridge` returns one when `SpeakerDiarization` is on; the `LoopbackBridge` exposes a simple
  one for tests.

### What `wireChannelPlane` does (in `StartBridgeSession`)

```typescript
private async wireChannelPlane(active: ActiveBridgeSession): Promise<void> {
    const host = active.ChannelHost;
    if (!host) return;                                       // back-compat: no channel plane
    const meetingControls = active.Bridge.GetMeetingControlsEventSource();
    const sink = active.RealtimeSession.SendContextNote      // perception sink (provider-gated)
        ? (t: string) => active.RealtimeSession.SendContextNote?.(t) : undefined;

    await host.StartSessionChannels(active.AgentSessionID, meetingControls, sink);
    active.ServerChannelTools = host.GetSessionServerTools(active.AgentSessionID);   // → runner.ServerChannelTools
    active.ExecuteServerChannelTool = (name, args) =>                                // → runner.ExecuteServerChannelTool
        host.ExecuteSessionServerTool(active.AgentSessionID, name, args);
}
```

The contributed `ServerChannelTools` + `ExecuteServerChannelTool` are surfaced on the returned
`ActiveBridgeSession`, so the runner-constructing layer registers them on `RealtimeSessionRunner`. The
realtime session's `SendContextNote` is fed into the channel context so **channel perception** (a hand
went up, who is speaking, time remaining) reaches the model. Channel teardown runs on every stop path via
`closeChannelPlane`. The whole wiring is **failure-tolerant**: a channel-host error is logged and never
breaks the live media plane.

The Zoom driver realizes this end-to-end: `ZoomMeetingControlsEventSource` adapts the `IZoomMeetingSdk`
roster / hand-raise / speaking / mute into the channel's event source, so an agent on a Zoom call gains
the facilitator tool vocabulary (`MeetingControls_RaiseHand`, `_CallOnParticipant`, `_MuteParticipant`,
`_SetTimer`) + perception with zero platform code in the channel.

---

## 6. Turn-Taking

Turn-taking is **generic and platform-agnostic** — it operates on the diarized transcript stream the
bridge provides, lives in the base package, and is identical for Zoom or Twilio. The agent always
*hears* (inbound media always reaches the model so it has context); the policy gates only *generation*
— whether the agent **speaks**, **posts to chat**, or stays **silent**. All three modes ship together.

`TurnTakingPolicy` is **pure**: every external dependency (the addressed-matcher, the "worth saying"
scorer, and the clock) is **injected**, so there is no real time and no platform code in it — fully
unit-testable and deterministic.

| Mode | Behavior |
|---|---|
| **`Passive`** (default) | Generate ONLY when the agent is **addressed** (name/mention detection via the injected `IAddressedMatcher`; `RegexAddressedMatcher` is the shipped default). Two passive agents in one room never loop — neither speaks unless called on. |
| **`Active`** | A "worth saying" scorer that fires **only in silence windows** (never barging over a live speaker), throttled so it can't dominate. Speaks when `score ≥ threshold` AND the silence window elapsed AND not throttled. |
| **`Hybrid`** | Passive voice PLUS a chat hand-raise — speak when addressed, but post to chat (the social-cost-free "raise hand") when it has something to add and chat is available. Degrades to plain passive when chat is unavailable. |

```typescript
const policy = new TurnTakingPolicy({
    Mode: 'Passive',
    Matcher: new RegexAddressedMatcher(['Sage', 'the assistant']),
    // Active/Hybrid tuning (all optional): Scorer, SilenceWindowMs (1500), ThrottleMs (15000),
    // ScoreThreshold (0.7), Now (Date.now — injectable for deterministic tests).
});

const decision = policy.EvaluateTurn({
    Segment: { Text: 'Hey Sage, what do you think?', IsAgent: false, EndMs: Date.now() },
    HumanSpeaking: false,   // Active never fires while a human speaks
    ChatAvailable: false,   // Hybrid degrades to passive when false
});
// → { Action: 'Speak', Reason: 'Agent was addressed (passive).' }
```

`EvaluateTurn` returns `{ Action: 'Speak' | 'PostToChat' | 'Silent', Reason }`. The `Reason` string is
for observability and tests. Agent-spoken segments (`IsAgent: true`) are never treated as addressing the
agent — that avoids self-trigger loops.

### How the engine feeds the policy

`AIBridgeEngine.wireTurnTaking` subscribes the realtime session's transcript stream and feeds only
**completed human turns** through the policy, then acts on the decision:

```typescript
private wireTurnTaking(active: ActiveBridgeSession): void {
    active.RealtimeSession.OnTranscript((t: RealtimeTranscript) => {
        if (t.Role !== 'user' || !t.IsFinal) return;   // ignore the agent's own speech + partials
        const segment: TurnTranscriptSegment = { Text: t.Text, IsAgent: false, EndMs: Date.now() };
        const decision = active.TurnPolicy.EvaluateTurn({ Segment: segment });
        this.applyTurnDecision(active, decision.Action);
    });
}
```

`applyTurnDecision` acts behind the injected `IRealtimeSession`: `Speak` requests a spoken update (via
`RequestSpokenUpdate` when the provider supports it; otherwise the inbound media already reached the
model and it responds on its own turn). `PostToChat` is a documented hook for the **Phase 2** bridge-chat
channel (the wiring is complete; a chat-capable channel consumes it once the server-side channel plane
lands). `Silent` does nothing.

---

## 7. Entity Invariants

Each of the five entities has a server-side `*EntityServer` subclass (in `MJCoreEntitiesServer`) that
enforces cross-field / cross-record invariants the DB can't express, via `ValidateAsync`
(`DefaultSkipAsyncValidation` overridden to `false`). Each invariant is a **pure, separately-exported,
unit-tested helper**, and each gates on the fields it reads (so unrelated edits don't re-pay the cost or
resurface a pre-existing quirk on an untouched field).

| Entity server | Invariants enforced |
|---|---|
| `MJAIBridgeProviderEntityServer` | (1) **`SupportedFeatures` is well-formed** — parses as a JSON object whose every key is a known `IBridgeProviderFeatures` flag and every value is a boolean. Typos (`OutbondDial`) and non-booleans (`"true"`, `1`) are rejected with precise messages (`ValidateSupportedFeaturesJson`). (2) **`DriverClass` is non-empty** — it's the `ClassFactory` key; a blank value silently fails resolution (`ValidateDriverClass`). Both are intra-row → no DB round trips. |
| `MJAIBridgeAgentIdentityEntityServer` | (1) **`(ProviderID, IdentityValue)` is unique** (case-insensitive, cross-record `RunView` self-excluded on update) — two agents reachable at the same address on the same provider would make inbound routing ambiguous. (2) **`IdentityValue` shape matches `IdentityType`** — `Email` looks like an email, `PhoneNumber` is E.164-ish digits; `AccountID` is opaque (`ValidateIdentityValueFormat`). |
| `MJAIBridgeProviderChannelEntityServer` | **`(ProviderID, ChannelID)` is unique** — defense-in-depth + friendly message over the DB `UQ_AIBridgeProviderChannel` constraint (turns a raw SQL unique violation into a clear, field-attributed validation message). |
| `MJAIAgentSessionBridgeEntityServer` | Cross-field coherence the DB CHECKs can't express: (1) **Outbound needs a target** — `Direction='Outbound'` requires an `Address` or `ExternalConnectionID`. (2) **`Connected` ⇒ `ConnectedAt` set; terminal ⇒ `DisconnectedAt` set.** (3) **`CloseReason` only on terminal status** (a reason on an active bridge is a hard failure; a terminal bridge *missing* a reason is a **warning**, so the janitor/error paths aren't blocked from persisting). All intra-row → no DB round trips. |
| `MJAIAgentSessionBridgeParticipantEntityServer` | **At most one `IsAgent = 1` participant per bridge** — a bridge is one agent's connection into one room (multiple agents = multiple bridges, [§9](#9-multi-party-planned)); two bot participants would confuse diarization and the self-echo gate. The DB can't express a filtered-unique index, so a cross-record `RunView` (gated on dirty, self-excluded) enforces it. |

---

## 8. Join Methods & Agent Identity

There are several ways an agent gets onto an endpoint, each gated by a `SupportedFeatures` join flag and
recorded in `AIAgentSessionBridge.JoinMethod`:

- **On-demand** (`OnDemandJoin`) — a user pastes a join URL and sends the agent. The shipped default
  `JoinMethod` is `'OnDemand'`.
- **Scheduled** (`ScheduledJoin`) — a `Scheduled` bridge fires at the meeting start time (see the
  `ScheduledBridgeRunner` below).
- **Invite / Calendar** (`InviteJoin`) — *the headline UX, now built (Phase 4).* The agent gets a real
  mailbox/calendar identity **in the customer's own tenant** (`AIBridgeAgentIdentity` with
  `IdentityType = 'Email'`). Organizers invite `sage@customer.com` like a person; the `CalendarWatcher`
  matches the invite to the identity and creates a `Scheduled` bridge that joins at start. The same
  identity model serves **inbound telephony** (a call to the agent's DID routes to it via
  `IdentityByValue`).
- **Native marketplace inclusion** (`NativeInvite`) and **in-meeting command** — planned, per-platform.

Because the tenant-resident identity generalizes to "agent presence," the same `AIBridgeAgentIdentity`
row is the seam for future **email** and **calendar** surfaces, and existing tenant governance
(retention, DLP, eDiscovery, offboarding) applies to the agent automatically.

### 8a. Invite & Calendar Joins (built — `@memberjunction/ai-bridge-server`)

The "invite the agent like a person" pipeline has four collaborating pieces, all **injectable seams** so
the whole flow is unit-testable with no network/DB:

```mermaid
flowchart LR
    CAL["Calendar (Graph / Google)"] -->|ICalendarSource| W["CalendarWatcher.Sweep()"]
    W -->|new invite, agent is attendee| B["Scheduled AIAgentSessionBridge<br/>JoinMethod=Invite · Status=Scheduled<br/>ScheduledStartTime=start · Address=joinUrl"]
    B -->|ScheduledStartTime <= now| R["ScheduledBridgeRunner.RunDueBridges()"]
    R -->|host builds IRealtimeSession| E["AIBridgeEngine.StartBridgeSession()"]
    PROV["IAgentIdentityProvisioner"] -.provisions.-> ID["AIBridgeAgentIdentity (tenant mailbox)"]
    ID -.watched by.-> W

    style B fill:#2d5016,stroke:#1a5c3a,color:#fff
    style E fill:#5a3d7a,stroke:#7c5295,color:#fff
```

1. **`CalendarWatcher`** (`calendar-watcher.ts`) — for each active `IdentityType='Email'` identity, polls
   its calendar via an injected **`ICalendarSource`**, and for each new invite where the identity is an
   attendee creates a `Scheduled` `AIAgentSessionBridge` (+ a linked `Idle` `AIAgentSession`). It is
   **idempotent** (dedupes on the invite's `ExternalEventID`, stamped onto `ExternalConnectionID`),
   **tolerant** (a failing identity/source is logged and the sweep continues), and exposes only
   `Sweep()` — **no timer; the host schedules the polling loop** (matching the engine janitor). It
   ignores non-attendee invites, invites with no join URL, past-start invites, and invites whose join URL
   resolves to no active provider.

2. **`ICalendarSource`** (`calendar-source.ts`) — the **per-provider calendar-API seam**:
   `ListUpcomingInvites(identityValue, sinceCursor)` returning normalized
   `{ ExternalEventID, JoinUrl, StartTime, Attendees, Organizer }`. Ships with **`GraphCalendarSource`**
   and **`GoogleCalendarSource`** stubs that throw `CalendarSourceNotBoundError` until production binds
   the real Microsoft Graph (`/me/events` / app-level `calendarView` delta) and Google Calendar
   (`events.list` + `syncToken`) APIs. **The real-API binding is a documented seam** (like the bridge SDK
   seams) — see the `TODO(production)` in each adapter.

3. **`ResolveProviderFromJoinUrl(url, providers)`** (`join-url-resolver.ts`) — a **pure** helper mapping a
   join URL to the matching active `AIBridgeProvider` by hostname (`zoom.us` → Zoom,
   `teams.microsoft.com` → Teams, `meet.google.com` → Google Meet, Webex, Slack, Discord, RingCentral,
   …). Uses a strict **domain-suffix** test, so `zoom.us.attacker.com` and `evil.com/zoom.us/...` do **not**
   match. `ClassifyJoinUrl(url)` returns just the platform; both return `null` for unknown/malformed input.

4. **`ScheduledBridgeRunner`** (`scheduled-bridge-runner.ts`) — a documented host hook (like the janitor's
   `ReconcileOrphans`): `RunDueBridges()` loads `Scheduled` bridges with `ScheduledStartTime <= now` and
   calls `AIBridgeEngine.StartBridgeSession` for each. The **`IRealtimeSession` construction stays the
   host's job** — the runner asks a host-supplied `ScheduledBridgeSessionFactory` to build the start
   params, keeping the transport seam clean exactly like `StartBridgeSession`. The engine owns the
   `Scheduled → Connecting → Connected/Failed` transition, so a started/failed bridge drops off the next
   pass automatically (idempotent); a factory that returns `null` leaves the bridge `Scheduled` for later.

5. **`IAgentIdentityProvisioner`** (`identity-provisioner.ts`) — a thin documented seam to provision a
   tenant mailbox/calendar (or telephony DID) identity for an agent: `Provision(request)` returns the
   concrete `{ IdentityValue, Configuration }`, which `ApplyProvisionResultToIdentity` maps onto a
   `MJAIBridgeAgentIdentityEntity` (the caller saves it). `StubAgentIdentityProvisioner` throws
   `IdentityProvisionerNotBoundError` until production wires Microsoft Graph admin / Google Workspace
   Admin SDK / a telephony carrier API. The schema already holds the result — this is the seam + the
   `TODO(production)`, deliberately **not** over-built.

**Host wiring (production):** schedule `CalendarWatcher.Sweep()` and `ScheduledBridgeRunner.RunDueBridges()`
on the host's timer/scheduler, bind a real `ICalendarSource` + `IAgentIdentityProvisioner`, and supply a
`ScheduledBridgeSessionFactory` that constructs the `IRealtimeSession` + `RealtimeSessionRunner` for a due
bridge. Everything below the host is platform-free and tested with mocks.

---

## 9. Multi-party (LiveKit + multiple agents)

*Emergent property of the bridge — **shipped (Phase 7 core)**.* Multi-party is **not** a separate build.
The conferencing platform (or the MJ-native LiveKit room) *is* the shared room — it already does the SFU,
the mixing, the multi-party media plane.

- **Multiple humans?** The platform already provides it — MJ builds nothing.
- **Multiple agents?** Each agent is an **independent bridge connection** (its own bot, its own
  `AIAgentSession`) into the *same* room. Two agents joining one room literally hear each other — each
  one's voice is part of "everyone else" in the other's inbound mix. **No transcript-relay hack, no mixer
  in MJ — the room already mixes.**

### LiveKit — the MJ-native room (`@memberjunction/ai-bridge-livekit`)

Every other bridge connects **out** to a 3rd-party platform. **LiveKit is the opposite: a self-hosted
WebRTC SFU that MJ runs itself** — the "MJ-native room" for an MJ-owned multi-party experience (e.g.
embedded in Explorer). The architecture treats a Zoom meeting and a LiveKit room **identically** — both
are multi-party media transports — so LiveKit is "*another bridge, not a special build.*"

- `LiveKitBridge` (`@RegisterClass(BaseRealtimeBridge, 'LiveKitBridge')`) is an ordinary
  `BaseRealtimeBridge` driver behind an injectable `ILiveKitRoomSdk` seam (production binds
  `livekit-server-sdk` + a room client; tests inject `FakeLiveKitRoomSdk` — **24 tests, no network**).
- `connect(roomUrl, token)` joins the room as a bot participant with an MJ-minted token. LiveKit does the
  **full** media set — audio/video/screen in+out — and **per-participant audio tracks**, so diarization
  is native (no extra mixer). Chat rides the LiveKit **data channel** (`sendDataMessage`). It contributes
  a Meeting Controls facilitator channel exactly like Zoom.
- Capabilities: on-demand join, A/V/screen in+out, diarization, data-channel chat, room-admin mute. No
  scheduled/invite/telephony — those gated base methods throw `BridgeCapabilityNotSupportedError`.

> **Running LiveKit: local dev vs production (Cloud).** The LiveKit SFU runs *alongside* MJAPI — MJAPI
> only mints join tokens (`LIVEKIT_URL` / `LIVEKIT_API_KEY` / `LIVEKIT_API_SECRET`), it never proxies
> media. For **local dev**, run a throwaway server — Docker for browser-only testing, or native
> `livekit-server --config docker/livekit/livekit.yaml` for the agent-bot path on macOS (Docker NAT
> breaks the bot's WebRTC media there). For **production / real-world deployments**, use
> [**LiveKit Cloud**](https://cloud.livekit.io) — drop its `wss://…livekit.cloud` URL + key + secret into
> `.env`, no server to run. The canonical how-to (both options, the macOS gotcha, env vars) lives in
> [`docker/livekit/README.md`](../docker/livekit/README.md).

### Inter-agent speaking discipline — `MultiAgentRoomCoordinator`

The one genuinely new problem is keeping multiple agents from **talking over each other or looping**. The
`MultiAgentRoomCoordinator` (`@memberjunction/ai-bridge-server`, pure + synchronous, **20 tests**) adds
**floor arbitration** on top of passive turn-taking:

- An agent calls `CanTakeFloor(roomId, agentSessionId)` before generating speech → `true` only if no
  *other* agent holds the floor. `TakeFloor` claims it, `ReleaseFloor` frees it, `IsFloorHolder` lets a
  bumped agent learn it must yield. **At most one agent speaks at a time across agents.**
- A designated **facilitator** agent (typically the one running the Meeting Controls channel) may
  **override** the floor (`FacilitatorOverride`) to cut in and call on a specific agent. Designate via
  `RegisterRoomParticipant(roomId, id, /*isFacilitator*/ true)` or `SetFacilitator(...)` at runtime.
- **Loop-safety = two guards.** Passive turn-taking ([§6](#6-turn-taking)) means an agent speaks only when
  *addressed by name*, so two passive agents never start a loop; the floor coordinator means even if both
  policies fired, only one speaks. The combination makes two agents in a room loop-safe **and**
  non-overlapping.
- Distinct rooms are fully isolated; single-agent rooms impose no contention.

It is wired through the engine's **additive** hooks — `AIBridgeEngine.RegisterRoomParticipant(roomId,
agentSessionId, isFacilitator?)` / `UnregisterRoomParticipant(...)` — keyed on the shared external room id
(the driver's `ExternalConnectionID`). **Every bridged session registers on connect and unregisters on
teardown**, so a single-agent room is a 1-member room whose floor is always free (no contention) and a 2+-
member room takes turns. The engine's `Speak` turn-decision (`applyTurnDecision`) now runs the floor
internally: `TakeFloor` → denied ⇒ stay silent; granted ⇒ `armFloorHold` (a `floorMaxHoldMs` safety
release) then trigger speech; the floor releases on the agent's own final assistant transcript, the safety
timer, or teardown.

### Meeting mode — "hear everything, speak when addressed"

The actual fix for agents reacting to *all* room audio (spiral / "picked up the last conversation"): in a
multi-agent room each agent's model **auto-response is disabled** and the **bridge becomes the sole speech
trigger**, gated by the addressing matcher. The lever is a host-neutral `Config.disableAutoResponse` flag
each driver translates to its own mechanism — **OpenAI** → `turn_detection.create_response=false` (keeps
VAD/transcription, suppresses the auto-reply); **Gemini** → `realtimeInputConfig.automaticActivityDetection.
disabled=true` + manual `activityStart`/`activityEnd` turns (it has no detect-but-don't-respond lever). The
`LiveKitAgentRoomCoordinator` detects the multi-agent transition and starts the 2nd+ agent in meeting mode
(`RegexAddressedMatcher` on its names + a meeting-discipline prompt clause). *Gemini's manual-activity turn
boundaries are unit-tested at the send-shape level but await live validation in a busy room.*

### Retroactive re-gating — capability-gated (`IRealtimeSession.Reconfigure`)

When a room becomes multi-agent, the agents **already in it** are switched into meeting mode too, *if their
provider can reconfigure a live socket*. This is gated by the realtime-session **capability surface**
(`RealtimeSessionCapabilities.CanReconfigureTurnMode`, see the Co-Agents guide §4): OpenAI reports `true`
and implements `Reconfigure({DisableAutoResponse:true})` via a partial `session.update` (+ a
`SendContextNote` so the agent knows it's now in a meeting); Gemini reports `false` and is left
conversational — no dead-method call. Engine: `AIBridgeEngine.ReconfigureSessionToMeeting(bridgeId,
matcher)`; coordinator re-gates each existing roster agent.

### Turn moderator — room-level "who speaks next" (engine seam)

Per-agent addressing (the `RegexAddressedMatcher` broadcast above) routes *direct address* but can't bring a
relevant-but-unaddressed agent in, can't let two agents have a productive exchange, and can't distinguish a
useful back-and-forth from an unproductive loop. The **turn moderator** is an optional engine seam that
replaces the matcher with a single room-level decision per turn. The *LLM specifics* (it's a prompt not an
agent run, the config cascade, the model, proactive vs addressed-only) live in the
[Co-Agents guide §4](REALTIME_CO_AGENTS_GUIDE.md#turn-moderator-multi-agent-turn-taking); this is the
engine-side mechanics.

- **The seam.** `AIBridgeEngine.SetTurnModerator(moderator)` injects a `TurnModerator` —
  `(ctx: TurnModeratorContext) => Promise<string[]>` returning the ordered `AgentSessionID`s to trigger
  (0+). It is analogous to `SetTranscriptSink` / `SetSessionRunFinalizer`: the engine stays
  framework-agnostic (it knows only the function signature + the `TurnModeratorContext` /
  `ModeratorRosterAgent` / `ModeratorLookbackTurn` shapes it exports); the `@memberjunction/ai-agents`
  `RealtimeTurnModerator` supplies the implementation, wired at MJServer schema-build time
  (`RealtimeBridgeResolver` calls `SetTurnModerator(RealtimeTurnModeratorDecision)`).
- **Broadcast → moderator → serialized queue.** In a multi-agent room `dispatchUserTurn` branches: when a
  moderator is set it routes the turn through `runRoomModerator` (one decision) instead of broadcasting to
  each agent's matcher. `runRoomModerator` builds the context from the live roster + diarized lookback, calls
  the moderator, filters the result to valid roster sessions, sets the room's ordered `roomSpeakerQueue`, and
  `drainSpeakerQueue` triggers speakers **one at a time through the floor** (`MultiAgentRoomCoordinator`) — a
  multi-agent route is heard as a clean sequence, never overlap. Per-room re-entrancy is guarded
  (`roomModeratorBusy`); the floor-release path drains the next queued speaker.
- **Lookback buffer.** The engine keeps a bounded per-room diarized lookback (`roomLookback`, capped at
  `ROOM_LOOKBACK_MAX_TURNS = 50`, each turn clipped) appended on every final transcript via `recordRoomTurn`.
  That buffer is the moderator's conversation window (the moderator narrows it further to its configured
  `contextWindowTurns`). The engine also tracks `roomConsecutiveAgentTurns` (reset on any human turn) for the
  optional agent-only-loop backstop.
- **Pre-stage on agent speech.** The moderator also runs on **agent** turns (`onAgentTurnCompleted`), not just
  human ones — powering both agent↔agent continuation and pre-staging: because it fires on the agent's FINAL
  TEXT while that agent's audio is *still playing out*, the next decision's latency is hidden behind playback.
- **Barge-in invalidation.** A human barge-in clears the room's staged speaker queue
  (`clearRoomModeratorState(roomKey, /*full*/ false)`) so a pre-staged agent→agent hand-off is discarded when
  the user changes the topic; teardown clears the full moderator state (lookback + queue + counter).
- **Matcher fallback.** When no moderator is injected, `dispatchUserTurn` keeps using the existing per-agent
  `RegexAddressedMatcher` broadcast — single-agent rooms and no-moderator deployments are entirely unaffected.

### Unified room transcript + per-speaker diarization

The engine elects **one scribe per room** (the first agent; re-elected on departure) and feeds its FINAL
transcripts to a registered `BridgeTranscriptSink`. The app-layer sink (`CreateBridgeRoomTranscriptSink`,
`@memberjunction/ai-agents`) writes **one `MJ: Conversations` of `Type='Meeting Room'` per room** (linked to
the **Meet** application by name, scoped out of normal chat) + **one `MJ: Conversation Details` per
utterance**. **Diarization**: the model transcript has no speaker label, so the engine tracks
`LastInboundSpeaker` from the inbound frame's `SpeakerLabel` (which the drivers populate; see §4 media-track
translation) and stamps it on each `User` line — so a heard line resolves to the actual participant (human
or another agent), not a generic "User". UserID resolution (vs. display name) is the remaining refinement.

**Surfacing**: the **Voice Transcripts** nav item in the AI Analytics dashboard
(`@memberjunction/ng-dashboards`, `AnalyticsRealtimeTranscriptsComponent` + `realtime-transcripts-data.ts`)
is a master-detail, diarized browser over these meeting transcripts.

### Session reaping — auto-leave + idle sweep + run finalization

Three same-process safeguards keep abandoned sessions from lingering `Connected` with a `Running` co-agent
run: (1) **auto-leave** — once a human has been seen and the roster drops to agents-only, an `emptyGraceMs`
timer (cancelled on re-join) stops the session; (2) **idle/TTL sweep** — `SweepStaleSessions` (self-scheduled
from `HandleStartup`) reaps sessions idle past `idleTtlMs` or over `maxDurationMs`, the same-process
complement to the prior-boot `ReconcileOrphans`; (3) **run finalization on every teardown path** — a
registered finalizer (called from `markBridgeDisconnected`) closes the co-agent run even on orphan/cross-host
reaps. All thresholds are overridable via `AIBridgeEngine.ConfigureSessionTimings({...})`.

### ⚠️ Single-node room state (a multi-node hardening constraint)

The room-scoped coordination state — the `MultiAgentRoomCoordinator` floor, the transcript **scribe
election**, and the **room roster** (membership for meeting-mode detection + re-gating) — all live
**in-memory per engine instance**, i.e. **per MJAPI node**. The design therefore assumes **all agents in a
given room land on the same host**. Single-node deployments are unaffected.

In a **multi-node** deployment behind a load balancer, two "add agent to room" requests can hit different
nodes, splitting the state: two scribes could both write the transcript, the floor wouldn't arbitrate
across hosts, and meeting-mode detection (which counts roster members on *this* node) could miss agents on
another node. Bridge rows already carry `HostInstanceID` (used by the prior-boot janitor), so the
hardening paths are: (a) **room-affinity routing** — pin all of a room's agents to one node, or (b)
**shared room state** — move the floor/scribe/roster to a shared store (DB or Redis). Until one of those
lands, treat single-node-per-room as a deployment invariant.

### Echo / self-audio

A bot must not hear its **own** output or it would react to itself. Conferencing platforms (and the
LiveKit SFU) **exclude a participant's own published audio from that participant's inbound mix**, so the
bridge driver never feeds the agent its own voice (`LiveKitBridge` documents this). Where a platform does
*not* exclude own-audio, the bridge driver must gate it before `OnMedia`. The `MultiAgentRoomCoordinator`
itself never touches media — it operates purely on floor state. The `BridgeParticipantInfo.IsAgent` flag
also lets multi-agent turn-taking exclude agents from "a human addressed me" detection.

---

## 10. Roadmap / Phase Status

This guide documents **shipped** code (Phase 0 and Phase 1 core). Everything else is planned — honestly
marked here and in the architecture plan.

| Phase | Deliverable | Status |
|---|---|---|
| **0 — Transport seam** | `BaseRealtimeBridge` media contract wired to `IRealtimeSession` (`OnMedia → SendInput`, `OnOutput → SendMedia`); `AIBridgeEngine` ↔ injected `IRealtimeSession`; `LoopbackBridge` proving an in-memory round-trip. | ✅ **Shipped** |
| **1 — Schema + engines** | The 5-entity migration → CodeGen; `AIBridgeEngineBase` (cache) + `AIBridgeEngine` (composition, lifecycle, host registry, janitor `ReconcileOrphans`); capability-gated `BaseRealtimeBridge` + `BridgeCapabilityNotSupportedError`; `TurnTakingPolicy`; the `*EntityServer` invariants. | ✅ **Shipped** (provider seed metadata + the `Realtime: Advanced Bridge Controls` authorization pending) |
| **2 — Server-side channel plane** | Dynamic tool-definition contribution into the runner's `ServerChannelTools` / `ExecuteServerChannelTool`, optional client surface, the `Meeting Controls` channel (roster / hand-raise queue / who's-speaking / timer → facilitator). The bridge ↔ channel-plane integration (`AIBridgeEngine` wires a session's channels generically via the optional `IBridgeChannelHost` + `BaseRealtimeBridge.GetMeetingControlsEventSource`) is shipped. | ✅ **Shipped** |
| **3 — Zoom meeting bridge** | `ZoomBridge` (`@memberjunction/ai-bridge-zoom`): on-demand + scheduled join, audio in/out, diarized participants, participant mute + chat, and a Meeting Controls facilitator surface — all behind an injectable `IZoomMeetingSdk` seam (real-SDK binding is a deployment TODO via `SetSdkFactory`). First real platform, built + unit-tested with no network. | ✅ **Shipped** (real Zoom SDK adapter binding pending) |
| **4 — Invite/calendar joins + agent identity** | `CalendarWatcher` ("invite the agent like a person") over an injectable `ICalendarSource` (Graph/Google stubs), `ResolveProviderFromJoinUrl`, `ScheduledBridgeRunner` (a janitor-style host hook starting due `Scheduled` bridges), and the `IAgentIdentityProvisioner` seam — all platform-free + unit-tested. Shared with inbound telephony. | ✅ **Shipped** (real calendar-API + provisioning admin-API binding pending — documented `TODO(production)` seams) |
| **5 — Teams → Meet → Webex → Slack → Discord** | One minimal subclass per platform on the proven base. | 🔜 Planned |
| **6 — Telephony: RingCentral → Twilio → Vonage** | `BaseTelephonyBridge` — outbound dial + inbound DID routing, DTMF, transfer. Same engine/session/turn-taking. | 🔜 Planned |
| **7 — Multi-party** | `LiveKitBridge` (`@memberjunction/ai-bridge-livekit`) — the self-hosted MJ-native room (full A/V/screen, native per-participant diarization, data-channel chat) behind an injectable `ILiveKitRoomSdk` seam; `MultiAgentRoomCoordinator` (`@memberjunction/ai-bridge-server`) — floor arbitration (one agent speaks at a time) + facilitator override, loop-safe with passive turn-taking; additive `AIBridgeEngine.RegisterRoomParticipant`/`UnregisterRoomParticipant` hooks. Built + unit-tested with no network. | ✅ **Shipped (core)** (real LiveKit SDK adapter binding pending; runner-layer floor wiring around generation pending) |
| **8 — Remote Browser channel** | `RemoteBrowserChannel` + `ContainerRunner` (any Playwright Docker image), screen-track out, control arbiter. | 🔜 Planned |
| **9 — Native marketplace inclusion** | Per-platform "add the agent" apps. | 🔜 Planned |
| **UI — Realtime dashboard + forms** | A `Realtime` section in the AI dashboard + custom `*Extended` forms + an AIAgent Realtime panel. | 🔜 Planned |

```mermaid
graph LR
    P0["0 · Transport seam ✅"] --> P1["1 · Schema + engines ✅"]
    P1 --> P2["2 · Channel plane"]
    P2 --> P3["3 · Zoom ✅"]
    P3 --> P4["4 · Invite/identity ✅"]
    P3 --> P5["5 · Teams·Meet·Webex·Slack·Discord"]
    P4 --> P6["6 · Telephony"]
    P5 --> P7["7 · Multi-party + LiveKit ✅"]
    P6 --> P7
    P3 --> P8["8 · Remote Browser"]

    style P0 fill:#2d5016,stroke:#1a5c3a,color:#fff
    style P1 fill:#2d5016,stroke:#1a5c3a,color:#fff
    style P2 fill:#2d5016,stroke:#1a5c3a,color:#fff
    style P3 fill:#2d5016,stroke:#1a5c3a,color:#fff
    style P4 fill:#2d5016,stroke:#1a5c3a,color:#fff
    style P7 fill:#2d5016,stroke:#1a5c3a,color:#fff
```

---

## Reference Map

| Concern | Where |
|---|---|
| Abstract driver, capability gating | `packages/AI/BridgeBase/src/base-realtime-bridge.ts` |
| Metadata cache engine | `packages/AI/BridgeBase/src/ai-bridge-engine-base.ts` |
| Media-track types | `packages/AI/BridgeBase/src/media-tracks.ts` |
| Turn-taking policy | `packages/AI/BridgeBase/src/turn-taking-policy.ts` |
| Capability error | `packages/AI/BridgeBase/src/capability-errors.ts` |
| Server engine + transport seam + channel-plane wiring + janitor | `packages/AI/Bridge/src/ai-bridge-engine.ts` |
| Bridge ↔ channel-plane seam (`IBridgeChannelHost`, `IBridgeMeetingControlsEventSource`) | `packages/AI/BridgeBase/src/channel-plane.ts` |
| Worked minimal driver example | `packages/AI/Bridge/src/loopback-bridge.ts` |
| Calendar watcher (invite → scheduled bridge) | `packages/AI/Bridge/src/calendar-watcher.ts` |
| Calendar-API seam + Graph/Google stubs | `packages/AI/Bridge/src/calendar-source.ts` |
| Join-URL → provider resolution (pure) | `packages/AI/Bridge/src/join-url-resolver.ts` |
| Scheduled-bridge runner (host hook) | `packages/AI/Bridge/src/scheduled-bridge-runner.ts` |
| Agent identity provisioning seam | `packages/AI/Bridge/src/identity-provisioner.ts` |
| Zoom driver (first real platform) | `packages/AI/Providers/BridgeZoom/src/zoom-bridge.ts`, `zoom-sdk.ts`, `zoom-meeting-controls.ts` |
| LiveKit driver (MJ-native room) | `packages/AI/RealtimeBridge/Providers/LiveKit/src/livekit-bridge.ts`, `livekit-sdk.ts`, `livekit-meeting-controls.ts` |
| Multi-agent floor arbitration (one speaker at a time + facilitator) | `packages/AI/Bridge/src/multi-agent-room-coordinator.ts` (engine hooks: `AIBridgeEngine.RegisterRoomParticipant` / `UnregisterRoomParticipant` / `RoomCoordinator`) |
| Turn-moderator engine seam (`SetTurnModerator`, lookback, serialized queue, pre-stage, barge-in invalidation) | `packages/AI/RealtimeBridge/Server/src/ai-bridge-engine.ts` |
| Turn-moderator LLM plugin (prompt-not-agent, config cascade, name→session mapping) | `packages/AI/Agents/src/realtime/realtime-turn-moderator.ts`, `realtime-coagent-config.ts` |
| Server channel host + Meeting Controls channel | `packages/AI/Agents/src/realtime/realtime-channel-server-host.ts`, `meeting-controls-channel-server.ts` |
| Capability interface | `metadata/entities/JSONType-interfaces/IBridgeProviderFeatures.ts` |
| Entity invariants | `packages/MJCoreEntitiesServer/src/custom/MJAIBridge*EntityServer.server.ts`, `MJAIAgentSessionBridge*EntityServer.server.ts` |
| Generated entities | `@memberjunction/core-entities` (`MJAIBridgeProviderEntity`, …) |
| Full architecture + roadmap | `plans/realtime/realtime-bridges-architecture.md` |