@memberjunction/ai-agents

Complete framework for building and executing AI agents in MemberJunction. Provides the BaseAgent execution engine, pluggable agent type system (Loop, Flow, Realtime and Harness agents), hierarchical sub-agent orchestration, action execution, memory management with notes and examples, payload management, conversation context with message lifecycle management, and reranker integration.

It also provides two capability layers any agent can opt into:

• Skills — reusable capability bundles (instructions + Actions + sub-agents) an agent activates mid-run via a progressive-disclosure catalog. Availability is gated per-agent (AIAgent.AcceptsSkills); self-activation additionally requires the double activation gate (AISkill.ActivationMode × AIAgent.SkillActivationMode, both defaulting to RequestedOnly so autonomous skill expansion is always a deliberate double opt-in — users can still /skill-request any available skill). Every activation is recorded with provenance of authority on AIAgentRunStep.Skills (activation type, gate values, agent-stated reason), and Prompt/Actions/Sub-Agent steps carry per-step skill attribution. Skills are shareable via the dedicated skill-permissions model and portable via SKILL.md import/export.
• Plan Mode — a per-request human-in-the-loop gate (ExecuteAgentParams.planMode) that makes an agent present a plan and get it approved before executing any Actions or Sub-Agents, built on the existing MJ: AI Agent Requests pause/resume flow.

See the Agent Skills & Plan Mode Guide for the full architecture, the three-layer gate, the SKILL.md format, and the plan-approval flow.

Architecture

graph TD
    subgraph "@memberjunction/ai-agents"
        BA["BaseAgent<br/>Core Execution Engine"]
        style BA fill:#2d8659,stroke:#1a5c3a,color:#fff

        AR["AgentRunner<br/>Orchestration Entry Point"]
        style AR fill:#2d8659,stroke:#1a5c3a,color:#fff

        subgraph "Agent Types"
            BAT["BaseAgentType"]
            style BAT fill:#7c5295,stroke:#563a6b,color:#fff
            LAT["LoopAgentType"]
            style LAT fill:#7c5295,stroke:#563a6b,color:#fff
            FAT["FlowAgentType"]
            style FAT fill:#7c5295,stroke:#563a6b,color:#fff
        end

        subgraph "Support Systems"
            PM["PayloadManager<br/>Data Flow Between Steps"]
            style PM fill:#b8762f,stroke:#8a5722,color:#fff
            PCA["PayloadChangeAnalyzer"]
            style PCA fill:#b8762f,stroke:#8a5722,color:#fff
            PFM["PayloadFeedbackManager"]
            style PFM fill:#b8762f,stroke:#8a5722,color:#fff
            ACI["AgentContextInjector<br/>Notes, Examples, Data Sources"]
            style ACI fill:#b8762f,stroke:#8a5722,color:#fff
            ADP["AgentDataPreloader<br/>Batch Metadata Loading"]
            style ADP fill:#b8762f,stroke:#8a5722,color:#fff
            MMA["MemoryManagerAgent<br/>Note/Example Management"]
            style MMA fill:#b8762f,stroke:#8a5722,color:#fff
        end
    end

    BA --> BAT
    BA --> PM
    BA --> ACI
    BA --> ADP
    AR --> BA

    subgraph Dependencies
        AIP["@memberjunction/ai-prompts<br/>AIPromptRunner"]
        style AIP fill:#2d6a9f,stroke:#1a4971,color:#fff

        AIE["@memberjunction/aiengine<br/>AIEngine"]
        style AIE fill:#2d6a9f,stroke:#1a4971,color:#fff

        ACT["@memberjunction/actions<br/>ActionEngineServer"]
        style ACT fill:#2d6a9f,stroke:#1a4971,color:#fff

        RR["@memberjunction/ai-reranker<br/>RerankerService"]
        style RR fill:#2d6a9f,stroke:#1a4971,color:#fff
    end

    AIP --> BA
    AIE --> BA
    ACT --> BA
    RR --> BA

Installation

npm install @memberjunction/ai-agents

Key Components

BaseAgent

The core execution engine that all agents use. Handles:

• Hierarchical prompt execution (agent type’s system prompt as parent, agent’s prompts as children)
• Action execution through the MJ Actions framework
• Client tool invocation for browser-side UI operations (navigate to records, switch tabs, show search results) via PubSub round-trip
• Sub-agent orchestration with full context propagation
• Conversation context management with automatic message compaction
• Memory retrieval (notes and examples) with optional reranking
• Payload data management across execution steps
• ForEach and While loop operations
• Comprehensive execution tracking (AIAgentRun, AIAgentRunStep records)

AgentRunner

High-level entry point for agent execution. Provides:

• Agent resolution by ID or entity reference
• Permission checking before execution
• Data preloading for performance
• Simplified execution interface

import { AgentRunner } from '@memberjunction/ai-agents';

const runner = new AgentRunner();
const result = await runner.ExecuteAgent({
    agentId: 'agent-uuid',
    conversationMessages: [{ role: 'user', content: 'Analyze Q3 sales trends' }],
    contextUser: currentUser,
    onProgress: (step) => console.log(`${step.step}: ${step.message}`)
});

Agent Type System

Agents execute using a pluggable type system. The type determines how the agent decides its next action after each LLM call.

BaseAgentType

Abstract base that all agent types extend. Defines the DetermineNextStep() interface that produces a BaseAgentNextStep decision:

Step	Description
Chat	Send a message back to the user
Actions	Execute one or more server-side actions
ClientTools	Invoke browser-side tools (navigate, switch tabs, show results)
SubAgents	Delegate to sub-agents
MoreInfo	Ask the user for additional information
Retry	Retry the current step (e.g., after validation failure)
End	Complete execution
ForEach	Iterate over a collection
While	Loop while a condition is true

LoopAgentType

Conversational agent that runs in a loop: prompt -> decide -> act -> repeat. Best for interactive, chat-based agents. The LLM decides the next step at each iteration by producing a structured JSON response.

Decision models

An agent can ask a fast decision model typed questions in six places on its own. One master switch governs all of them: the Loop prompt param decisionsEnabled, false by default. Unless it is true, the agent never asks a decision model on its own, whatever the other settings say: their docs stay out of the prompt and the runtime refuses the call. With it true, each use still needs its own setting, and each of those is off by default too:

Use	Its own setting
Inline decisions on a turn	includeDecisionsDocs: true
finishIf gates on an Actions or Sub-Agent step	finishIfMode: 'shadow' or 'on'
Decision discovery (suggest an agent before the first prompt)	decisionDiscovery: true
Payload change check	payloadFeedbackCheck: true
Catalog narrowing	maxActionsInPrompt / maxSubAgentsInPrompt > 0
Memory Manager note gate	run data.enableDecisionGate: true

Set it in an agent’s AgentTypePromptParams ({"decisionsEnabled": true, "finishIfMode": "shadow"}) or for one run in data.__agentTypePromptParams. Uses someone placed explicitly do not read it: a Flow or task-graph Decision step, the Run Decision action, and other direct callers of AgentDecisionService. See the Decision Models guide.

Output Artifact Lifecycle (artifactDirective) & _ARTIFACTS_ENABLED Gating

When an artifact-producing Loop agent finishes a turn (taskComplete: true), it controls whether its final payload creates a new artifact, versions an existing artifact in the conversation, or suppresses artifact creation via artifactDirective: { behavior: 'create-new' | 'version-source' | 'suppress', targetArtifactId?, name?, description? }:

• Default-on per-agent gating (_ARTIFACTS_ENABLED): BaseAgent.isArtifactSupportEnabled(agent) returns true unless AIAgent.ArtifactCreationMode is 'Never' or 'System Only'. The Loop Agent system prompt ({% if _ARTIFACTS_ENABLED != false %}) and <mj-runtime-state> automatically omit all artifactDirective instructions and the _CONVERSATION_ARTIFACTS catalog for non-artifact agents (such as Sage and Knowledge Agent), while enabling them by default for artifact-producing agents (Query Builder, SkillSmith, Codesmith, Research Agent, etc.).
• Full-conversation artifact targeting (targetArtifactId): AgentRunner loads the conversation’s prior output artifacts into _CONVERSATION_ARTIFACTS so the agent can version any earlier artifact by targetArtifactId (not just the immediately preceding turn’s artifact).
• Clean payload reset on create-new: When artifactDirective.behavior === 'create-new' and the current payload is still the unmodified inherited payload from the prior conversation turn (payload === params.payload, even after read-only action steps) or replaceElements is used, BaseAgent.executePromptStep applies payloadChangeRequest against a fresh {} object so fields from a prior artifact never bleed into the new artifact.

See the Artifact Tools & Output Lifecycle Guide.

Single-Node Task Graph Folding (foldSingleTaskStep) vs. Multi-Step Durable Workflows

When a Loop agent (such as Sage) emits nextStep.type: 'Tasks', LoopAgentType.foldSingleTaskStep inspects the taskGraph:

• Single-node task graphs (1 task of kind 'Agent' or 'Action'): Automatically fold inline into a standard in-process 'Sub-Agents' or 'Actions' step — even when taskGraph.continuation === 'reinvoke' is specified (terminateAfter: false so the parent Loop agent gets another turn to synthesize the specialist’s output). This avoids durable task-graph dispatch overhead for 1-step specialist delegations (e.g., Query Builder, Visualization Agent) and keeps live sub-agent streaming in the active chat turn.
• Multi-node task graphs (2+ tasks with dependsOn / payloadMapping or non-trivial scheduling): Dispatch as durable MJ: Tasks rows through @memberjunction/task-graph, broadcasting real-time step transitions via TaskGraphFrameBroadcaster and delivering synthesized continuation replies and artifacts back to the conversation via TaskGraphContinuationDeliverer.

FlowAgentType

Step-based agent that follows a predefined flow graph — steps with explicit conditional paths between them. Best for deterministic workflows where the execution path is known in advance.

A top-level Flow agent no longer walks its own graph. DetermineInitialStep compiles the agent’s AIAgentStep + AIAgentStepPath rows into a TaskGraphSpec and returns a Tasks step; BaseAgent.executeTasksStep submits it and detaches. From that point the workflow is Task rows owned by the durable dispatcher in @memberjunction/task-graph, with the same claiming, conditions, skip cascade, retry and failure semantics as any other graph — one traversal engine, one set of rules, one place a bug can be fixed.

Consequences worth knowing before you debug one:

• A dispatched run ends at submission. It returns a handle (“Started — 4 tasks running”), not a result, and parks as Paused until the dispatcher settles the graph. Per-step detail lives in Task rows, not in AIAgentRunStep, and cost arrives afterwards via TotalCostRollup (see the guide).
• A run that needs the result walks the flow in-process. A sub-agent run (one with parentRun) does this automatically, and a top-level caller opts in with agentTypeParams.executionMode: 'inRun' (FlowAgentExecuteParams). The run returns the flow’s final payload, as in 5.x, and startAtStep works. A dispatched run that reaches the walker by mistake is refused at its choke point.
• The mode is on the run. A completed Decision step named “Workflow runs in this run” or “Workflow runs on the task-graph dispatcher” records which engine ran it and why.
• A dispatched Sub-Agent node starts its own AIAgentRun, linked from Task.AgentRunID and parented to the submitting run. If that sub-agent is itself a Flow agent, it runs in-process and returns its result to the task.
• A Decision step routes the same way in both modes. Paths read its answers only through decisions.<key>.<question>; its stepResult.result is the payload, which carries no answers. A flow with a Decision step is compiled and validated before its first step in-run too, so an incomplete Choice fork, a decision read before it can answer, or one read through the payload is refused in both modes. An answer below its minConfidence stops an in-run flow (the dispatcher holds the fork, and Retry on the step asks again). A failed decision call is a failed step: the paths that read its answers are passed over, and a recovery path such as stepResult.Success === false is taken whatever its rank.
• Multiple starting steps are supported. A Flow agent with more than one active StartingStep: true step (for example, a workflow saved from a task graph with parallel wave-1 roots) compiles all active starting steps as root nodes (dependsOn: []) in dispatch mode, and walks all starting branches sequentially before entering any downstream join step in inRun mode.
• Answers live in the run, not the payload. A run started with startAtStep past a Decision step has no answers from it, so every path that reads that decision stops the run with the reason. Start at or before the Decision step to route on it.

See the Workflows and Task Graphs Guide for the full model: node kinds, exclusive groups, payload mappings, loops, failure semantics, observability, and a worked configuration example.

HarnessAgentType

Runs an external agent harness — Claude Code, Codex CLI, OpenCode, Gemini CLI, Pi — as the reasoning substrate for an MJ agent, while MJ keeps identity, permissions, governed data access, payload contracts, HITL, cost control and run-level audit. Lives in @memberjunction/ai-agent-harness.

It is the opposite design choice from RealtimeAgentType: where Realtime is session-driven and bypasses the loop entirely (IsSessionDriven), Harness stays inside the loop. HarnessAgentType extends LoopAgentType and inherits DetermineNextStep wholesale, because a harness turn is protocol-identical to a Loop prompt iteration — the harness ends each turn by emitting the same next-step JSON envelope a Loop model emits. HarnessAgentBase extends BaseAgent overrides exactly one method, executePrompt, substituting a harness turn for the prompt call.

That is why every existing guarantee still applies with no new enforcement code: next-step validation, per-action MaxExecutionsPerRun, skill gates, plan-mode blocking, PayloadManager ACLs, checkExecutionGuardrails, run-step recording. One authority channel, not two — which is also why the MCP loopback is read-only.

The 'HarnessAgentType' string is registered under both ClassFactory roots (BaseAgentType for the protocol, BaseAgent for the driver), so a single AIAgentType.DriverClass value selects both halves.

See the External Agent Harness Guide.

RealtimeAgentType

Session-driven agent type for live, low-latency, full-duplex interaction (voice today; the contract is modality-agnostic). Unlike Loop and Flow, a Realtime agent does not run an iterative reasoning loop — modern realtime stacks (GPT Realtime, Gemini Live, ElevenLabs Agents, AssemblyAI Voice Agent) own the listen-reason-speak loop themselves, so RealtimeAgentType wraps a long-lived BaseRealtimeModel session instead. The marker is its IsSessionDriven getter (always true): BaseAgent duck-types it and routes execution to a RealtimeSessionRunner rather than entering the loop. See Real-Time Co-Agents below.

graph LR
    subgraph "Loop Agent"
        L1["Prompt LLM"] --> L2["Parse Response"]
        L2 --> L3{Decision}
        L3 -->|"Actions"| L4["Execute Actions"]
        L4 --> L1
        L3 -->|"Chat"| L5["Reply to User"]
        L5 --> L1
        L3 -->|"End"| L6["Complete"]
    end

    subgraph "Flow Agent"
        F1["Step 1"] -->|"Path A"| F2["Step 2a"]
        F1 -->|"Path B"| F3["Step 2b"]
        F2 --> F4["Step 3"]
        F3 --> F4
        F4 --> F5["End"]
    end

    style L1 fill:#2d6a9f,stroke:#1a4971,color:#fff
    style L2 fill:#7c5295,stroke:#563a6b,color:#fff
    style L3 fill:#b8762f,stroke:#8a5722,color:#fff
    style L4 fill:#2d8659,stroke:#1a5c3a,color:#fff
    style L5 fill:#2d8659,stroke:#1a5c3a,color:#fff
    style L6 fill:#2d8659,stroke:#1a5c3a,color:#fff

    style F1 fill:#2d6a9f,stroke:#1a4971,color:#fff
    style F2 fill:#7c5295,stroke:#563a6b,color:#fff
    style F3 fill:#7c5295,stroke:#563a6b,color:#fff
    style F4 fill:#b8762f,stroke:#8a5722,color:#fff
    style F5 fill:#2d8659,stroke:#1a5c3a,color:#fff

Real-Time Co-Agents

Real-time is the live, low-latency complement to the async Loop/Flow paradigm — not a replacement for it. Loop and Flow agents stay asynchronous and untouched; what they cannot deliver is a natural spoken conversation, because the loop is intentionally long-running while realtime models own sub-second turn taking, VAD, and barge-in natively. The Realtime agent type bridges the two worlds: a realtime model holds the conversation, and real work is delegated back to the async agents you already have as ordinary AIAgentRuns.

The first shipped agent of this type is the Realtime Co-Agent (seeded in metadata/agents/): a generic companion that acts as the live voice for any target MJ agent. The target is a runtime parameter — agents gain voice by configuration, never by being rewritten. Which co-agent voices a given target resolves through a metadata chain at session start: runtime coAgentId parameter → AIAgent.DefaultCoAgentID → AIAgentType.DefaultCoAgentID → the seeded global Realtime Co-Agent.

Key pieces in this package:

• RealtimeAgentType (src/agent-types/realtime-agent-type.ts) — the registered agent type (@RegisterClass(BaseAgentType, "RealtimeAgentType")). Its IsSessionDriven getter makes BaseAgent.ExecuteAgent branch into executeRealtimeSession() instead of the loop; the loop-oriented abstract methods are implemented defensively and should never be reached.

• RealtimeSessionRunner (src/realtime/realtime-session-runner.ts) — dependency-injected orchestrator for the server-bridged topology: opens the IRealtimeSession, registers the stable tool set, persists transcript turns, checkpoints usage onto the single long-lived AIPromptRun on a debounced cadence (crash-safe — a janitor close finalizes from the last-persisted values), and aborts an in-flight delegated run on barge-in. It also narrates delegated work itself: each significant progress event is fed to the session’s optional SendContextNote, and a built-in pacing engine (first spoken update ~5 s into a burst, ≥8 s session-global spacing, deduped multi-message digests, cancellation on completion/barge-in) drives RequestSpokenUpdate — both feature-detected, since providers may omit either capability. Fully unit-testable against a mock realtime model.

• realtime-narration.ts (src/realtime/realtime-narration.ts) — the shared narration module both topologies use: ResolveNarrationInstructionsTemplate() resolves the DB-driven Realtime Co-Agent - Progress Narration prompt (with a deprecated fallback to the pre-rename Voice Co-Agent - Progress Narration seed name) from AIEngine’s cache, and BuildServerNarrationInstructions() substitutes its placeholders for the server-bridged runner (the browser host has its own richer builder that chains the model’s actual prior narrations — only the browser can observe those).

• RealtimeTurnModerator (src/realtime/realtime-turn-moderator.ts, exported as RealtimeTurnModeratorDecision) — the multi-agent turn moderator behind the bridge engine’s SetTurnModerator seam. In a multi-agent voice room it decides, once per turn, which agent(s) (0+, ordered) should speak next — routing direct address and relevance, letting a productive agent↔agent exchange run, and going quiet on unproductive ping-pong. It runs as a fast AIPromptRun (not an agent run — a stateless who-speaks classification) via AIPromptRunner, tied to the co-agent’s AIAgentRun for full observability, resolving its room-wide config from the Realtime agent type’s DefaultConfiguration (turnTaking.moderator in realtime-coagent-config.ts) and the per-agent mode from the target agent’s layer. The seeded "Realtime: Turn Moderator" prompt is bound to a small/fast model (GPT-OSS-120B on Cerebras, swappable in metadata) because it sits on the latency-critical path. Degrades to a name-contains safety net only when no prompt is configured; the engine itself falls back to the per-agent matcher broadcast when no moderator is wired. The engine-side mechanics (lookback, serialized floor queue, pre-staging, barge-in invalidation) and the config cascade are documented in the Real-Time Co-Agents Guide §4 and the Realtime Bridges Guide §9.

• RealtimeToolBroker (src/realtime/realtime-tool-broker.ts) — the topology-agnostic tool-execution path both topologies share, so a tool call produces byte-for-byte identical results whether the provider socket lives on the server or in the browser. Routes the stable invoke-target-agent tool (the co-agent’s one headline capability — the target rides inside the call, never as a per-target tool) to the delegation seam, everything else to the tool executor; owns the per-call AbortController; serializes failures as structured errors the model can narrate.

• RealtimeClientSessionService (src/realtime/realtime-client-session-service.ts) — the server half of the client-direct topology (the shipped audio path: the browser opens its own provider socket with a server-minted ephemeral token, while the server keeps prompt/tool authority and executes every relayed tool call). PrepareClientSession resolves the Realtime model, assembles the companion system prompt + memory context (including a resumed session’s prior-leg transcript when the caller threads one in), mints the ClientRealtimeSessionConfig, and creates the co-agent observability run (an AIAgentRun + linked AIPromptRun stamped with the co-agent’s AgentID, plus a single Prompt-type AIAgentRunStep whose TargetLogID points at the prompt run — finalized with a Success stamp by FinalizeCoAgentRun). ExecuteRelayedTool runs relayed calls through the shared broker — registering each in a per-session in-flight delegation registry so CancelInFlightDelegations(sessionId, callId?) (the engine behind the CancelRealtimeSessionTool mutation) can abort them on explicit user cancel — including resuming a paused (AwaitingFeedback) interactive run and creating artifacts from delegated-run payloads.

• RealtimeChannelServerHost (src/realtime/realtime-channel-server-host.ts) — the server half of the interactive-channel plugin registry (MJ: AI Agent Channels.ServerPluginClass; base class BaseRealtimeChannelServer in @memberjunction/ai, mirroring the browser’s BaseRealtimeChannelClient). A BaseSingleton host that resolves the ACTIVE registry rows at session mint into one fresh plugin instance per session (unregistered keys skip with a log, never fatal) and routes the durable lifecycle into them: session started (SessionManager.CreateSession), channel-state saves pre-persistence (a plugin may validate/normalize the persisted state of record — it can transform a save, never lose or block one), and session close from every provenance (explicit / janitor / shutdown / error), with a brief post-close linger so the client’s final state flush still routes through the plugin before disposal. Every hook is failure-isolated — a throwing plugin can never break a live call.

• WhiteboardChannelServer (src/realtime/whiteboard-channel-server.ts, key 'WhiteboardChannelServer') — the reference server channel plugin: guards the Whiteboard’s persisted state of record by validating each landed save parses as a JSON object (flagging corrupt payloads at save time instead of at the next resume) and canonicalizing valid payloads to compact JSON. LoadWhiteboardChannelServer() is invoked from MJServer’s agentSessions module.

• MediaChannelServer (src/realtime/media-channel-server.ts, key 'MediaChannelServer') — the server channel plugin for the Media surface. Like the Whiteboard it canonicalizes the channel’s persisted media state, and additionally resolves the agent’s media kit at session start: it implements IRealtimeChannelServerDataAware to receive the session contextUser+provider, reads a per-session mediaCollectionID override out of RealtimeChannelServerContext.AgentSessionConfig (the persisted AIAgentSession.Config_), and feeds the model a manifest via SendContextNote. The kit-resolution logic is the pure, unit-tested agent-media-library.ts (buildAgentMediaContextNote / resolveAgentMediaManifest / formatAgentMediaManifest); precedence is runtime override > AIAgent.DefaultMediaCollectionID > none. The agent then surfaces items with the existing Media_ShowMedia client tool — no new client tool. LoadMediaChannelServer() is invoked from MJServer’s agentSessions module. See the Real-Time Co-Agents Guide.

• AgentMemoryContextBuilder (src/agent-memory-context-builder.ts) — the memory/RAG injection orchestration extracted out of BaseAgent so the co-agent assembles the same notes/examples/context a loop agent does, with no duplicated logic.

• realtime-channel-policy.ts (BuildSessionChannelPolicy) — the server half of channel scoping for client-direct sessions, pure and unit-tested. At mint the browser reports its channel candidates (channelCandidatesJson); RealtimeClientSessionService matches each to its MJ: AI Agent Channels row from the server’s own registry (the client’s view is never trusted), folds the cascade’s realtime.channels section (agent layers plus Application.AgentSettings.Realtime.Channels) through ResolveRealtimeChannelScope, and returns the resolved RealtimeSessionClientPolicy (PrepareClientSession → ClientPolicy; the resolver returns it as ClientPolicyJson and persists the in-scope channel keys on the session). The tools the model is told about are narrowed to match: a vetoed channel’s native tools never reach the provider, an included 'opt-in' channel’s are added by name, an 'on-demand' channel declares none (it is opened through ContextTool), and tools no candidate owns (the host’s own) pass through. Server plugins start before the scope is known, so RealtimeChannelServerHost.PruneSessionChannels drops the excluded channels’ server halves afterwards. Absent candidates (a client that predates scoping, or a host that mints through its own proxy) ⇒ tools used exactly as declared and no policy.

• ClientOnlyChannelServer (src/realtime/client-only-channel-server.ts, key 'ClientOnlyChannelServer') — the generic server half for any channel that runs entirely in the browser. MJ: AI Agent Channels.ServerPluginClass is NOT NULL, so a client-only channel (an Open App’s form or game channel) sets its row’s ServerPluginClass to this key instead of writing an empty plugin. It takes its channel name from the registry row it was resolved for (BindChannelName, called by RealtimeChannelServerHost), contributes no server tools and does not interpret or rewrite saves. It does not decide scope (resolved before any plugin exists) and cannot stop a save (the host persists whatever the client submits when a plugin returns null), so persistence is controlled by the client plugin’s SerializeState(). LoadClientOnlyChannelServer() is invoked from MJServer’s agentSessions module.

• InteractiveComponentChannelServer (src/realtime/interactive-component-channel-server.ts, key 'InteractiveComponentChannelServer') — the deliberately thin server half of the Interactive Component channel (MJ: AI Agent Channels row InteractiveComponent), a ClientOnlyChannelServer subclass that keeps its existing registry key: no server tools, and the client plugin saves no state of record, because the channel’s actions are derived from the open component’s spec and run in the browser. LoadInteractiveComponentChannelServer() is invoked from MJServer’s agentSessions module.

• Channel exposure and zero data retention. BuildSessionChannelPolicy also resolves each channel’s exposure (min(channel ceiling, agent cap, zero-data-retention ceiling); the user’s own choice is applied by the browser). The agent cap is realtime.channels.config.<Key>.maxExposure; requireZeroDataRetentionFor: ['state' | 'pixels'] (in the realtime configuration cascade and in Application.AgentSettings.Realtime.Channels.RequireZeroDataRetentionFor) lowers exposure when the session model’s effective configuration (AIEngine.GetEffectiveModelConfiguration) does not declare Privacy.ZeroDataRetention: true; a failed model lookup fails closed. The resolved exposure and the reasons for any downgrade ride back in ClientPolicyJson. Also in the realtime session tuning section: session.unverifiedMaxSeconds / session.verifiedMaxSeconds (positive integers, carried through the cascade for the session-deadline logic; never forwarded to the provider driver).

• App client tools and the unified resolver. Application.AgentSettings.ClientTools is now actually read by the realtime path: resolved against the tool-definition catalog and layered, through ResolveClientTools (override → session → app → static), beneath the live surface’s manifest in the app-context snapshot the companion prompt renders. The agent’s static MJ: AI Agent Client Tools junction tools ride the returned policy (so the browser can describe them) but are deliberately not added to the voice prompt — voice prompts are token-sensitive and a tool with no browser handler is noise. The app’s AgentSettings column is parsed with ParseAgentSettings rather than the generated accessor.

Delegated runs link to the co-agent’s run via ParentRunID and share the session’s AgentSessionID, so the whole episode — session, co-agent run (with its prompt run + Timeline step), delegated runs, transcript (ConversationDetail rows), channel state, usage telemetry (checkpointed server-side, or relayed from the browser via RelayRealtimeUsage) — groups under one session record with full standard observability.

The browser-side driver layer lives in @memberjunction/ai-realtime-client; the model primitive (BaseRealtimeModel / IRealtimeSession) lives in @memberjunction/ai; the Angular UX (call overlay, interactive channels, the live whiteboard) lives in @memberjunction/ng-conversations. The flagship guide — concept, triple-registry plugin architecture, topologies, session lifecycle, channels, narration, security — is guides/REALTIME_CO_AGENTS_GUIDE.md.

PayloadManager

Manages data flow through agent execution:

• Stores key-value data accessible across all steps and sub-agents
• Supports typed payload changes requested by the LLM (newElements, updateElements, replaceElements, removeElements)
• Auto-corrects LLM output shape errors (autoCorrectStrayKeys), moving stray sibling keys into newElements/updateElements and automatically unwrapping common LLM wrapper aliases (setFields, setElements, createElements, addElements, modifyElements, patchElements, updates, fields, set, add) when the wrapper key does not exist on the payload and is not in allowedPaths
• Validates and applies changes through PayloadChangeAnalyzer
• Provides feedback to the LLM about successful/failed changes via PayloadFeedbackManager

const manager = new PayloadManager();
manager.Set('customerData', { name: 'Acme', revenue: 1000000 });
const data = manager.Get('customerData');

AgentContextInjector

Injects contextual information into agent prompts:

• Retrieves relevant notes via vector similarity search
• Retrieves relevant examples for few-shot learning
• Injects data source content
• Applies reranking when configured

AgentDataPreloader

Optimizes agent startup by batch-loading all required metadata in parallel:

• Agent entity with all relationships
• Actions and their parameters
• Sub-agent data
• Prompt configurations

📖 Memory is a big topic. For the complete architecture — the three-tier model, injection and scoping, the in-flight write pipeline, and the Memory Manager’s hardening/consolidation/decay phases — see the Agent Memory Guide. The sections below are component-level summaries.

MemoryWriteManager (In-Flight Memory Writes)

Lets agents commit durable, cross-run memories mid-run — the user says “I prefer bar charts” and the agent records it the moment it’s stated, via a memoryWrites field on the loop response (a sibling of scratchpad and artifactToolCalls, processed inline with zero turn cost). Gated per agent via AIAgent.AllowMemoryWrite (on by default — opt out for restricted or experimental agents); disabled agents never see the capability in their system prompt, and the turn loop independently rejects drift.

The agent-facing surface is intentionally trivial (“remember this”) — the framework enforces every guard:

• Type restriction — only descriptive Preference/Context memories in-flight; behavioral/Constraint memories require Memory Manager or human promotion (prompt-injection defense)
• Scope clamp — writes land ≤ Agent + User scope; hints can narrow but never broaden, and never global
• Near-duplicate guard (vector similarity, default 0.85) — a near-dup of a note written earlier in the same run is superseded in place (last write wins); a pre-existing note is reinforced (AccessCount bump) only when the request exactly restates it (normalized text equality). Any textual difference writes a new provisional note instead — a difference may be a correction (“never use pie charts” must not be absorbed into a stale “loves pie charts” note); true paraphrase duplicates are consolidated later by the Memory Manager’s hardening dedupe. Fails open if the vector service is unavailable.
• Per-run cap (default 5) and within-run idempotency hashing
• Provenance + TTL — notes land with Status='Provisional', AuthorType='Agent', source run/conversation stamped, and a 7-day ExpiresAt safety net

Provisional notes are immediately injectable into future runs — rendered first with (provisional) labels, per-note recorded dates ([Preference, 2026-06-10] …, so the policy’s most-recent-wins tiebreaker is resolvable), and recency-wins precedence (a just-stated preference beats stale vetted memory) — and are later hardened or pruned by MemoryManagerAgent’s hardening pass (below). Every write is recorded as a Tool run step, and outcomes are reported back to the agent in an expiring conversation message so it doesn’t re-emit. Writes execute sequentially by design: each save syncs the in-memory vector store that the next write’s dedupe check reads.

For the full design see plans/agent-inflight-memory-writes.md; end-to-end verification harness at scripts/memory-write-smoke.ts.

MemoryManagerAgent

Handles persistent memory operations for agents:

• Creating and updating agent notes
• Managing agent examples
• Scoped memory for multi-tenant deployments (UserScope support)
• Consolidation, decay, and protection-tier maintenance over the agent note pool (see below)
• Hardening pass over agent-authored provisional notes — runs unconditionally at the start of every cycle (before the consolidation-gated phases): LLM-dedupes each Status='Provisional' note against hardened notes (duplicates archived with ConsolidatedIntoNoteID lineage), and hardens survivors to Status='Active' with ExpiresAt=NULL so they join importance scoring, consolidation, contradiction detection, and decay in the same cycle

Session tuning & resilience (v5.49): the effective config’s realtime.session section (effortLevel — MJ-normalized, parallelToolCalls, mcpTools, inputTranscriptionModel, transcriptionLanguages) flows into the driver Config bag on both topologies via GetSessionTuningSettings. The session runner observes the chained cancellation signal (AbortSignal dep), performs bounded transport reconnects on fatal drops (MaxTransportReconnects, default 1), and accumulates per-modality usage detail (RealtimeUsage.Input/OutputTokenDetails) that the checkpoint persists for multi-channel cost attribution. See guides/REALTIME_CO_AGENTS_GUIDE.md § “Session tuning, cost attribution & resilience”.

Consolidation Pipeline

When invoked in maintenance mode, MemoryManagerAgent runs an end-to-end pipeline over the agent’s notes to keep the memory pool useful and bounded over time. The pipeline is a sequence of phases on the run, each emitting its own observability data.

• Clustering — groups semantically similar notes by cosine similarity. The clustering threshold is 0.60 (intentionally broad — the LLM is the final arbiter for ambiguous clusters during the consolidate phase).
• Cluster splitting — splitOversizedCluster breaks any cluster larger than 7 notes so the consolidation prompt stays focused.
• Drift prevention — maxConsolidationCount = 3 caps the number of times any one note can be folded into successor consolidations. When a cluster reaches the cap, anchored-mode drilling resolves the original sources via DerivedFromNoteIDs so consolidation operates on the underlying facts rather than re-summarizing summaries.
• Consolidate — the LLM generates a merged note for each cluster. The pipeline records DerivedFromNoteIDs, increments ConsolidationCount, and revokes source notes with ConsolidatedIntoNoteID back-links to preserve provenance.
• Verify Consolidation Output — post-check phase (verifyConsolidationOutput) confirms entity-attribute coverage hasn’t been lost in the merge; reports entitiesChecked and entitiesMissing.
• Contradiction detection — extracts entity-attribute-value triples across active notes, flags conflicting values, and resolves contradictions by revoking the older or lower-importance side.
• Ebbinghaus decay archival — applies a forgetting curve to the importance score over time; low-importance notes that haven’t been retrieved are archived. ProtectionTier modulates the decay rate.
• Protection tiers — Immutable / Protected / Standard / Ephemeral. Immutable notes are never modified; Protected notes are excluded from consolidation but still age normally; Ephemeral notes decay faster.
• Outlier auto-promotion — notes whose semantic uniqueness lands in the 95th percentile are auto-promoted to Protected so they aren’t swallowed by future consolidations.

The pipeline emits two new run-step types for observability:

• Process Consolidation Cluster — one child step per cluster, with clusterSize, noteIds, shouldConsolidate, consolidatedNoteId, sourceNotesArchived, verificationPassed, entitiesChecked, entitiesMissing.
• Verify Consolidation Output — phase-level step covering the post-consolidation verification pass.

Run-level payload fields added: scoreDistribution, entityTriplesExtracted, decayScoreDistribution, protectedPreserved, ephemeralAccelerated, and consolidation triggerType (one of forced / time / event / count).

Memory Cleanup Agent has been deprecated — its responsibilities are folded into MemoryManagerAgent’s pipeline.

For the full design (functional requirements, threshold rationale, decay curve, contradiction taxonomy), see specs/001-memory-consolidation/spec.md and the implementation in src/memory-manager-agent.ts.

Usage

Basic Agent Execution

import { AgentRunner } from '@memberjunction/ai-agents';
import { ExecuteAgentParams } from '@memberjunction/ai-core-plus';

const runner = new AgentRunner();
const result = await runner.ExecuteAgent({
    agentId: 'my-agent-id',
    conversationMessages: [
        { role: 'user', content: 'What are the top 5 customers by revenue?' }
    ],
    contextUser: currentUser
});

if (result.success) {
    console.log(result.outputMessages);
}

With Sub-Agent Orchestration

Sub-agents are automatically discovered from the agent’s relationships and invoked when the LLM requests delegation:

const result = await runner.ExecuteAgent({
    agentId: 'orchestrator-agent-id',
    conversationMessages: messages,
    contextUser: currentUser,
    onProgress: (step) => {
        // Track execution across agent hierarchy
        console.log(`[${step.agentName}] ${step.message}`);
    }
});

With Runtime Action Changes

const result = await runner.ExecuteAgent({
    agentId: 'my-agent-id',
    conversationMessages: messages,
    contextUser: currentUser,
    actionChanges: [
        { scope: 'global', mode: 'add', actionIds: ['crm-search-id'] },
        { scope: 'all-subagents', mode: 'remove', actionIds: ['delete-record-id'] }
    ]
});

With Memory Scope (Multi-Tenant)

Multi-tenant deployments can isolate the agent’s memory cohort (notes and examples) per request by passing scope fields on ExecuteAgentParams. The fields are top-level — there is no userScope wrapper:

const result = await runner.ExecuteAgent({
    agentId: 'my-agent-id',
    conversationMessages: messages,
    contextUser: currentUser,
    // Primary scope (indexed for fast filtering)
    PrimaryScopeEntityName: 'Organizations',
    PrimaryScopeRecordID: orgId,
    // Secondary scopes (arbitrary dimensions, validated against the agent's ScopeConfig)
    SecondaryScopes: { TeamID: teamId }
});

See docs/AGENT_MEMORY_SCOPING.md for the full model — built-in scopes, primary/secondary semantics, inheritance modes, and how scope propagates through sub-agent invocations.

For an Audience (Shared Conversations)

When an agent answers in a conversation several people share, set Audience so the run shows only what the asker and every other participant may see. Set it from the host’s own participant list, server-side; it is never read from data.

const result = await new AgentRunner().RunAgent({
    agent,
    conversationMessages: messages,
    contextUser: asker,
    Audience: { Mode: 'Intersection', UserIDs: otherParticipantIDs }   // MJ: Users IDs
});

The run fails before any prompt if the audience is malformed or names an ID no user has (the server’s UserCache is refreshed once first). Under an audience that adds a reader besides the caller: pre-execution RAG checks every reader’s scope permission and searches with SearchParams.Audience; only shared notes and examples are injected; data-source preloading and the previous turn’s tool results are skipped; task graphs, client tools, the caller’s app context, memory writes and realtime sessions are withheld or refused; no out-of-conversation request (MJ: AI Agent Requests) is raised, so the run is never resumed without the audience; and every action call carries RunActionParams.Audience, which the action engine refuses (AUDIENCE_UNSUPPORTED, locked out for the run under its own 'audience' breaker reason) for any action that does not declare SupportsAudience — today only Search and Scoped Search do. Sub-agents inherit the audience. Limits (no persisted audience to resume with; per-reader scope expansion; no Union mode) and the full table are in Searching for an audience.

With Message Lifecycle Management

const result = await runner.ExecuteAgent({
    agentId: 'my-agent-id',
    conversationMessages: messages,
    contextUser: currentUser,
    messageExpirationOverride: {
        expirationTurns: 3,
        expirationMode: 'Compact',
        compactMode: 'First N Chars',
        compactLength: 500
    },
    onMessageLifecycle: (event) => {
        console.log(`${event.type}: ${event.reason} (saved ${event.tokensSaved} tokens)`);
    }
});

With Post-Run Compaction, Scratchpad Pruning & Memory Extraction (compactAfterRun)

Programmatic callers and multi-run workflows can opt into automatic post-run cleanup once an agent run settles via ExecuteAgentParams.compactAfterRun. When enabled (true or an AgentPostRunCompactionConfig object), BaseAgent.finalizeAgentRun executes a failure-isolated post-run compaction pass after pending step writes and the run record are saved:

1. Conversation Message Compaction — When params.conversationMessages.length >= messageCountThreshold (default: 20), older volatile result messages (IsVolatileResultMessage: action-result, client-tool-result, tool-result, sub-agent-result, loop-result) in [1 .. length - preserveRecentCount - 1] are compacted via compactMessage() while preserving the initial user goal (index 0), ordinary user/assistant conversational turns, turns carrying toolCalls, and the most recent turns.
2. Scratchpad Pruning — When the loop scratchpad contains more notes or completed tasks than maxScratchpadNotes (default: 20), ScratchpadManager.PruneNotes() and ScratchpadManager.EnforceTaskLimit() prune the oldest low-priority note lines first (preserving [HIGH] / [IMPORTANT] / [PINNED] / ❗ / ⚠️ notes) and trim excess completed tasks.
3. Background Memory Extraction — When extractMemories !== false and agentRun.Success is true, triggers background memory extraction (triggerPostRunMemoryExtraction) to harvest durable learnings from the completed run without blocking the caller.

const result = await runner.ExecuteAgent({
    agentId: 'my-agent-id',
    conversationMessages: messages,
    contextUser: currentUser,
    compactAfterRun: {
        enabled: true,
        messageCountThreshold: 12,
        extractMemories: true,
        maxScratchpadNotes: 20
    }
});

Latency Optimizations & Concurrent Sub-Agent Execution

To drastically reduce execution latency, the framework includes the following optimizations:

1. In-Memory Query Embedding Caching

In-memory caching is integrated into AIEngine.EmbedText to bypass redundant local ONNX embedding inference, eliminating CPU-bound event-loop blockages.

• Eviction: Backed by MJLruCache from @memberjunction/global — a bounded LRU keeps hot queries warm across bursts (default capacity 5000 entries).
• Cache keys: ${modelID}|sha256(text) so a large text doesn’t pin its full string in the cache map.
• Promise dedup: Concurrent callers for the same (model, text) share a single in-flight inference promise — only one ONNX run happens even under fan-out load.
• Control: Callers can supply options { bypassCache: true } (re-calculates but caches the new result) or { noCache: true } (bypasses and does not store the result).
• Empty input: Empty/whitespace text short-circuits to null without invoking the embedding provider.
• Negative result eviction: Failed or empty-vector responses are evicted so subsequent calls retry rather than returning a stale failure.
• Reset: The cache can be manually cleared by calling AIEngine.Instance.ClearEmbeddingCache().

2. Parallel Data Source Preloading

AgentDataPreloader.PreloadAgentData resolves agent data sources concurrently with a bounded concurrency cap (default 10 in-flight at once) so a long list of sources can’t saturate the DB connection pool.

3. Non-blocking Observability DB Logging

Observability writes (AIAgentRunStep and AIPromptRun records) are non-blocking during step execution and settled at terminal boundaries:

• Per-entity and cross-step save chaining (AgentRunStepSaveQueue): Step creation (createStepEntity, createCompactionStep) and finalization/updates (finalizeStepEntity, updateCompactionStep, checkExecutionGuardrails, validateSuccessNextStep, handleFinalPayloadValidationFailure) enqueue INSERT and UPDATE operations onto _stepSaveQueue without blocking the agent loop. Updates on a step are chained behind its initial INSERT, and finalizeStepEntity preserves pre-finalize step fields (such as guardrailExceeded in OutputData and FinalPayloadValidationResult/FinalPayloadValidationMessages) across post-INSERT reloads. Child steps (ParentID) are chained behind their parent step’s INSERT.
• In-flight PayloadAtStartSourceStepID FK ordering (FK_AIAgentRunStep_PayloadAtStartSourceStep): findPayloadAtStartSource matches the latest non-Running step with PayloadAtEnd even when that source step’s initial INSERT is still in flight (!sourceStep.IsSaved), avoiding redundant multi-kilobyte PayloadAtStart clones on fast sequential steps. When the matched source step is not yet saved, createStepEntity chains the new step’s INSERT after [parentStepEntity, payloadAtStartSource] and attaches a beforeInsert safety fallback that restores inline PayloadAtStart (and clears PayloadAtStartSourceStepID = null) if the source step’s INSERT failed.
• Complete terminal settlement: _stepSaveQueue.Flush() and _promptRunner.FlushPendingSaves() are awaited across all terminal paths — normal completion (finalizeAgentRun), failure (createFailureResult), cancellation (createCancelledResult), and post-turn conversation compaction (recordCompactionRunStep) — before the final agentRun.Save() executes, ensuring no orphaned or Running steps/prompt runs remain in Flight when ExecuteAgent() resolves. Save failures are logged via LogError and folded into agentRun.ErrorMessage for visibility.

4. Concurrent Sub-Agent Execution

Loop agents can request multiple sub-agents to run in parallel by returning a subAgents array in their nextStep decision:

• Bounded fan-out: Sub-agents dispatch with a configurable concurrency cap (default 5) so a misbehaving LLM can’t spawn unbounded parallel runs.
• Deterministic transcript: Delegation messages and progress events are pushed synchronously in the source order of the subAgents array — completion order doesn’t reorder the conversation.
• Payload isolation: Each child sub-agent receives a deep-cloned copy of its input payload, so in-flight mutations by one sibling can’t be observed by another.
• State Merging: Aggregates all media/file outputs and sequentially merges child/related sub-agent payloads back into the parent state via the PayloadManager to avoid race conditions.
• Per-sibling audit trail: Each parallel sub-agent’s AIAgentRunStep records its own contribution as PayloadAtEnd rather than the cumulative merged state, so forensic logs distinguish each sibling.
• Termination semantics: Matches the single sub-agent path — if any dispatched child requested terminateAfter: true, the parent terminates regardless of whether that child succeeded. The parent’s step is reported as Failed when any child failed, Success when terminating cleanly, and Retry otherwise.
• Context Preservation: Appends an aggregated delegation and completion log to the parent conversation context once all parallel steps complete.

Loop Response Schema

{
  "taskComplete": false,
  "nextStep": {
    "type": "Sub-Agent",
    "subAgents": [
      { "name": "DbAgent", "message": "Search the database", "terminateAfter": false },
      { "name": "WebAgent", "message": "Gather details from web search", "terminateAfter": false }
    ]
  }
}

5. Cache-Served Memory-Manager Maintenance Reads

The Memory Manager runs on a schedule (every ~15 minutes). AIEngineBase already holds the entire MJ: AI Agent Notes and MJ: AI Agent Examples pools in memory — loaded unfiltered as entity objects and kept current via BaseEntity save/delete events — so the maintenance phases serve their candidate scans from that cache (AIEngine.Instance.AgentNotes / AgentExamples) instead of issuing per-cycle RunView/RunViews round-trips.

• Eliminated round-trips: the consolidation event-trigger count, the orphan-prune candidate scan (loadPruneCandidateNotes), the TTL-expiry scan (loadExpiredItems), and the decay-candidate scan (loadDecayCandidates) all filter/sort/cap the in-memory arrays — removing the “Entity Already in Engine” redundancy-telemetry warnings these calls used to emit.
• Read-only projection, never aliasing: cache scans project to plain candidate rows ({ ID, ... }); they never hand a cached BaseEntity instance into a mutate-and-save path. The actual archive/decay/harden writes still re-Load() a fresh, owned entity via GetEntityObject, so the shared cache instances are never mutated in place.
• Single source of truth for status: note status filtering reuses IsInjectableNoteStatus() from @memberjunction/core-entities (Active + Provisional), the same predicate the read-path injection/scoping queries use, so the maintenance set can never drift from the injectable set.
• Intentional exceptions (still hit the DB): the hardening pass loads entity_object provisional notes specifically to mutate them, so it reads fresh, owned entities; the resolved-source-chain loader keeps a targeted RunView only for note IDs the cache misses; and MJ: AI Agent Runs reads stay as queries because runs are transactional and not cached by AIEngineBase.

6. Trailing Runtime State (prompt-cache layout)

The Loop agent’s volatile per-iteration state — current date/time, Scratchpad State and Payload — is never rendered in the system prompt. preparePromptParams builds it into a <mj-runtime-state> fragment (RuntimeStateFragmentBuilder) appended as the final user message of each request, and the system prompt carries a static ## Runtime State pointer. Provider prompt caching is a prefix match over tools → system → messages, so a byte-stable system prompt lets the entire history cache incrementally: measured on Sage, 0% → 78–90% cached on Anthropic and OpenAI, 21% → 91% on xAI, 30–65% → 66–96% on Gemini and Cerebras, with prompt cost per million tokens down 37–80%.

• Per-provider retention — providers with a byte-prefix cache (OpenAI, xAI) need prior fragments retained and the new one appended; block-cache providers (Anthropic, Gemini, Cerebras) get the previous fragment replaced. The choice is the model catalog’s ModelConfiguration.LLM.PrefixPromptCache flag, set under the vendor row’s Configuration.ModelDefaults and inherited by every model it serves unless a model-vendor row overrides it (BaseAgent.resolvePrefixPromptCache → AIEngine.GetEffectiveModelConfiguration, cascade Model Types < Models < Vendors’ Configuration.ModelDefaults < Model Vendors), decided once per run and frozen. No provider names live in code.
• Escaping at send time — the fragment tag literals are escaped in every non-system message of the outgoing copy of the history, so no tool result, user turn or tool-call argument can pose as framework state; the stored history is never mutated.
• Delivery gate — the fragment is only sent when the system prompt template carries the pointer; a template still embedding the old blocks (unsynced database) or one with neither (Flow, custom prompts) gets none. Markers: VOLATILE_TEMPLATE_MARKERS, overridable via the protected volatileTemplateMarkers getter.
• Specialization placement — specializationPlacement: auto relocates a child prompt into the fragment (<mj-agent-specialization>) only when its unrendered template references a volatile placeholder, handing the runner the pre-rendered text via AIPromptParams.PreRenderedChildTemplates.
• Anthropic breakpoint — BaseLLM.splitTrailingVolatileState lets AnthropicLLM place its cache_control breakpoint on the last real history message and send the fragment uncached.

All placeholder names, tag literals and headings are in src/constants.ts. Full mechanism: Agent Prompt Caching Guide; numbers: Performance and Cost Briefing.

Cross-Turn Conversation Compaction & History Retrieval

Long conversations no longer grow the agent’s context without bound. Two cooperating layers (design: plans/agent-conversation-compaction.md) activate only when a run carries ExecuteAgentParams.conversationId — programmatic runs, sub-agents, and tests without a conversation are untouched:

Tier A — durable summary baseline. ConversationCompactionManager watches the assembled window against the effective budget (AIAgent.ContextWindowMaxTokens → AIAgentType default → the selected model’s MaxInputTokens, clamped to the model; trigger/target percents inherit the same way). When the window crosses the trigger, one summary prompt (Conversation Summary, seeded metadata) folds the prior summary + raw delta only (recursive pattern) into a new summary persisted on the boundary row’s ConversationDetail.SummaryOfEarlierConversation, with SummaryPromptRunID linking the producing AIPromptRun. Every subsequent run then assembles [summary, boundary row, ...tail] via the shared ConversationEngine.AssembleContextWindow fold over rows from the single-sourced ConversationEngine.LoadWindowRowsFresh per-request loader. BaseAgent fires the pass post-turn (fire-and-forget after the run row saves — the caller’s completion event never waits on the summary LLM) for settled root runs — Completed OR AwaitingFeedback, since a Chat final step is the normal per-turn ending for conversational agents — with a pre-turn synchronous fallback when a run starts already over an explicitly configured budget. The pass loads its window fresh per request (RunView + the shared ConversationEngine.AssembleContextWindow fold — never the engine’s process-global cache), excludes the in-flight agent-response placeholder row (ExcludeDetailIds), and the boundary selector never picks the newest raw row — together eliminating the write race against the resolver. Fired/failed passes record a StepType='Compaction' run step whose TargetLogID is the same AIPromptRun the boundary row points at — written as a single pre-finalized INSERT (the pass is already over when the step is recorded, so createStepEntity’s completed option stamps the terminal state before the queued INSERT and no follow-up UPDATE is issued; __mj_UpdatedAt equals __mj_CreatedAt on these rows).

Tier B (existing, unchanged) — the in-turn message expiration/compaction and context-recovery escalation keep managing pressure within a run, now starting from the smaller windowed set.

Retrieval tools (ConversationToolManager, the conversationToolCalls inline response field — zero turn cost, mirroring artifactToolCalls): the summary is deliberately lossy; the full history stays addressable by the persisted per-conversation Sequence. getMessageBySequence, getMessagesByRange (max 50 messages / 32k chars), searchConversation (keyword/regex + role/range filters, snippets), and summarizeRange — a recursive sub-call that re-summarizes any ≤500-message slice through a task-specific lens on a cheap model (Summarize Conversation Range seeded prompt), its AIPromptRun linked via the Tool step’s TargetLogID. Results arrive as one-shot conversation messages with the standard compact-after-3-turns lifecycle. Guardrails: at most MAX_CONVERSATION_TOOL_CALLS_PER_TURN (8) tool calls execute per response (excess calls come back as skipped results the model can re-request), and searchConversation’s regex mode is ReDoS-guarded (256-char pattern cap, 20k-char per-message haystack slice, 2s total scan budget with partial-results warnings). Message edits after summarization are flagged via OriginalMessageChanged (server-side detection in MJConversationDetailEntityServer); summaries are never regenerated — agents page in exact rows instead.

Prior-turn tool-result carry-forward. Because each turn rebuilds its messages from the conversation window, a tool result paged in on turn N would be gone on turn N+1. injectPriorTurnToolResults re-injects the same agent’s immediately previous settled root run’s successful read-tool results (eligibility decided by the toolFamily stamped in each Tool step’s OutputData) as one transient, compactable message — one-turn memory by construction, scoped by AgentID so parallel agents in one conversation never inherit each other’s results. The per-turn prior-run lookup is served by PriorTurnToolResultCache (a BaseSingleton wrapping MJLruCache, keyed by conversation + agent): the settling root run publishes its own Tool-step projections from memory at finalize — an empty array for tool-free runs, so the common case costs zero DB queries per turn — and the loader falls back to the original RunView pair only on a cache miss. Both loaders share one entity-typed predicate (BaseAgent.carryForwardPredicate + settledRunStatuses) so the SQL filter and the in-memory projection cannot drift; freshness semantics and the multi-node staleness bound are documented on the cache class.

Action Failure Circuit Breaker

executeActionsStep now reports an action’s real outcome (ActionResult.Success, result code, message) — previously a failed action was reported to the model as a success and agents retried unconfigured tools until the run timed out. On top of that, ExecuteSingleAction applies a run-scoped breaker before dispatching, checked fatal → identical-arguments → budget:

Rule	Trigger	Effect
Fatal lockout	One credential / service-configuration failure (isFatalActionError; parameter-level “not configured” messages are deliberately non-fatal)	Action disabled for the run, any arguments
Identical arguments	IDENTICAL_FAILURE_THRESHOLD (2) failures with the same normalized arguments	Calls with those arguments blocked; other arguments dispatch
Attempt budget	ACTION_FAILURE_BUDGET (5) consecutive failures, any arguments	Action disabled for the run

A success clears both counters. A blocked call returns a CircuitBreakerActionResult (carrying Reason) in ~0 ms without reaching the action engine, and the model receives a [CRITICAL/ACTION_UNAVAILABLE | REPEATED_IDENTICAL_CALL | ATTEMPTS_EXHAUSTED] directive as a user message (a [WARNING/ACTION_FAILURE] with an “attempt N of 5” counter otherwise). Argument identity is normalizeActionParams (three protected layers: outer → per-entry → recursive value). The pipeline registry and the ForEach / While operators pass { skipCircuitBreaker: true } and bypass every rule, leaving the history untouched. Evidence and thresholds: Performance and Cost Briefing §4.

Root-Tree Cumulative Budget Guards (AgentTreeBudget)

Per-agent entity guardrails (MaxIterationsPerRun, MaxTokensPerRun, MaxCostPerRun, MaxTimePerRun) bound each AIAgentRun in isolation, which cannot stop multiplicative fan-out when a parent agent delegates to sequential or parallel sub-agents (for example, a depth-3 tree where each agent stays under a 10-step limit can still execute hundreds of steps across the tree).

BaseAgent enforces root-tree cumulative budget guards via a single shared AgentTreeBudget instance initialized at the root run and passed by object reference (params.treeBudget) to every child, related, parallel, and realtime-delegated sub-agent run:

Ceiling on ExecuteAgentParams / AgentTreeBudget	Shared Counter	Breach Message
maxTreeSteps	totalSteps (incremented on every createStepEntity)	Agent tree step budget exceeded (${totalSteps}/${maxTreeSteps} total steps across tree)
maxTreeTokens	totalTokens (accumulated after every prompt, decision, and compaction run)	Agent tree token budget exceeded (${totalTokens}/${maxTreeTokens} total tokens across tree)
maxTreeCost	totalCost (accumulated USD cost across all prompt runs in the tree)	Agent tree cost budget exceeded ($${totalCost.toFixed(4)}/$${maxTreeCost.toFixed(4)} across tree)
maxTreeDurationMs	Date.now() - startedAtMs (wall-clock elapsed time since root run started)	Agent tree duration budget exceeded (${elapsedMs}ms/${maxTreeDurationMs}ms across tree)

When any ceiling is reached:

1. budget.breachedReason is stamped on the shared AgentTreeBudget instance.
2. hasExceededAgentRunGuardrails() and executeNextStep() immediately return a terminal { step: 'Failed', terminate: true } decision with the breach reason.
3. Any parent or parallel sibling sharing the AgentTreeBudget observes breachedReason as soon as its in-flight step settles and halts the tree cleanly.

const result = await runner.RunAgent({
    agent: orchestratorAgent,
    conversationMessages: messages,
    contextUser: currentUser,
    maxTreeSteps: 25,
    maxTreeTokens: 100_000,
    maxTreeCost: 1.50,
    maxTreeDurationMs: 120_000
});

Documentation

Detailed guides are available in the docs/ directory:

Guide	Description
Actions Guide	Action discovery, execution, result lifecycle, expiration/compaction, context recovery
Artifact Tools & Output Lifecycle Guide	Inbound artifact routing (Inline vs ToolsOnly), tool libraries, output artifactDirective lifecycle (create-new, version-source, suppress), _ARTIFACTS_ENABLED prompt gating, full-conversation artifact targeting, and ArtifactType.ExtractRules
Decision Models	The decisionsEnabled master switch, then each automatic use: inline decisions, finishIf gates, discovery, the payload change check, catalog narrowing, the Memory Manager note gate
Prompt-Cache Performance and Cost Briefing	Before/after cached share and cost per model and topology for the trailing runtime-state layout; the action circuit breaker’s before/after; what to re-measure
Agent Prompt Caching Guide	Repo-level guide: the trailing-state layout, the PrefixPromptCache flag from the model catalog, the Anthropic breakpoint seam, specialization placement, the circuit breaker
Client Tools Guide	Browser-side tool invocation, runtime decoration, timeout config, prompt design, security
Sub-Agents Guide	Child agents, related agents, payload flow, context propagation, loops
Human-in-the-Loop	Feedback requests, assignment strategies, request lifecycle
Agent Memory Scoping	Multi-tenant memory (notes/examples) with UserScope support
Iterative Operations	ForEach and While loop patterns, parallel execution
State Management	Payload management, agent type state, stray-key & wrapper-alias recovery
Expression Context (PRD)	Expression evaluation in agent contexts
Agent Profiles (Proposal)	Proposed agent profile system
Code Refactoring Notes	Internal refactoring notes
Real-Time Co-Agents Guide	Repo-level flagship guide: Realtime agent type, Realtime Co-Agent, dual topologies, sessions, channels, whiteboard, narration, security

Re-exports

For backward compatibility, this package re-exports the following from @memberjunction/ai-reranker:

• RerankerService
• RerankerConfiguration
• parseRerankerConfiguration
• RerankServiceResult
• RerankObservabilityOptions
• LLMReranker

New code should import these directly from @memberjunction/ai-reranker.

Dependencies

• @memberjunction/ai-prompts — AIPromptRunner for prompt execution
• @memberjunction/aiengine — AIEngine for metadata and vector search
• @memberjunction/ai-core-plus — Shared types (ExecuteAgentParams, ExecuteAgentResult)
• @memberjunction/ai-engine-base — Base metadata cache and permissions

Storage Account Resolution

When agents create file-based artifacts (PDF, Excel, Word), the system resolves which FileStorageAccount to use via a hierarchical chain. The first non-null value wins:

Priority	Source	Field
1 (highest)	Runtime	ExecuteAgentParams.override.storageAccountId
2	Agent	AIAgent.DefaultStorageAccountID
3	Category tree	AIAgentCategory.DefaultStorageAccountID (walks up ParentID)
4 (lowest)	Agent Type	AIAgentType.DefaultStorageAccountID
Fallback	System	Single active account (if only one exists)

How it works

• BaseAgent.getStorageAccountID(params) implements the resolution logic. It is protected so subclasses can override it for custom routing.
• The resolved ID is stored in ExecuteAgentResult.resolvedStorageAccountId and passed to AgentRunner.ProcessFileArtifacts() for upload routing.
• AgentRunner.uploadBase64ToStorage() uses FileStorageEngine.Instance.GetAccountWithProvider() to get the account + provider, then initializeDriverWithAccountCredentials() for proper OAuth credential handling.

Startup validation

AIEngine.validateStorageAccountDefaults() runs at server startup. If 2+ active storage accounts exist but agent types lack a DefaultStorageAccountID, it auto-assigns the highest-priority account and logs a prominent warning.

Configuration

Set DefaultStorageAccountID at any level via the admin UI or metadata sync:

• Agent Type — broadest default (e.g., all Loop agents → Dropbox)
• Agent Category — business-domain default (e.g., Marketing → Box, Finance → SharePoint)
• Agent — per-agent override
• Runtime — ExecuteAgentParams.override.storageAccountId for programmatic callers

Dependencies

• @memberjunction/ai — Core AI abstractions
• @memberjunction/ai-reranker — Two-stage retrieval reranking
• @memberjunction/actions — Server-side action execution
• @memberjunction/actions-base — Action framework base types
• @memberjunction/core — MJ framework core
• @memberjunction/core-entities — Generated entity classes
• @memberjunction/global — Class factory and utilities
• lodash — Utility functions