# Agent Conversation Compaction & Recursive Context Access

**Status:** Design / implementation plan (pre-build)
**Date:** 2026-06-01
**Owner:** Amith Nagarajan
**Scope:** `packages/AI/Agents`, `packages/AI/CorePlus`, `packages/MJCoreEntities` (ConversationEngine + entities), `packages/MJServer` (resolvers), one migration in `migrations/v5/`.

---

## 0. TL;DR

MemberJunction already has **in-turn** context compaction inside `BaseAgent` (the
`pruneAndCompactExpiredMessages` machinery, `AgentChatMessageMetadata` with
`expirationMode`/`compactMode`/`originalContent`/`canExpand`, message-expansion via the
`messageIndex` Retry path, media stripping, and a 5-tier emergency context-recovery
escalation). It also already has per-agent config for this:
`AIAgent.ContextCompressionMessageThreshold`, `.ContextCompressionPromptID`,
`.ContextCompressionMessageRetentionCount`.

This plan adds the **missing durable, cross-turn layer**: a persistent conversation
**summary baseline** that rides on the existing, currently-unused
`ConversationDetail.SummaryOfEarlierConversation` field, plus **RLM-style addressable
retrieval tooling** so agents can page back into exact pre-summary history on demand.

We are **not** building a greenfield system. We are promoting the in-memory model to a
durable, sequence-addressable, multi-agent-aware layer and wiring it into the existing
config surface.

Design influences, triangulated:
- **MJ's existing in-turn compaction** (the foundation we extend).
- **Industry convergence** (Anthropic compaction API + context-editing + memory tool,
  OpenAI Responses compaction, MemGPT/Letta recursive summary + tiered paging, LangGraph
  `RunningSummary`): trigger on tokens before the window fills, keep user messages
  high-fidelity, recursive summaries (prior summary + delta), structured > prose, pair
  compaction with just-in-time retrieval.
- **Recursive Language Models** (Zhang, Kraska, Khattab — MIT CSAIL,
  [arXiv:2512.24601](https://arxiv.org/abs/2512.24601)):
  keeping full content in an external, addressable environment and letting the model
  *programmatically examine/slice/recursively sub-summarize* it beats lossy compaction on
  tasks that need dense recall. Our summary is the "constant-size metadata/handle"; the
  sequence-addressable history is the "external environment"; the retrieval +
  `summarizeRange` tools are the REPL/sub-call.

---

## 1. Core mechanism

### 1.1 The baseline rides on an existing field

`ConversationDetail.SummaryOfEarlierConversation` already exists (and is documented as
"summary of the conversation leading up to this record, for long-running performance").
We populate it. **A summary stored on the row at sequence N covers everything with
`Sequence < N`** (i.e., everything *below* it in the thread). No new detail type, no
`Compaction` table.

### 1.2 Context-window assembly (the load algorithm)

```
GetAgentContextWindow(conversationId):
    details = ConversationEngine cache for this conversation (in-memory; one query on cold miss)
    boundary = the detail with the HIGHEST Sequence whose SummaryOfEarlierConversation IS NOT NULL
    if boundary exists:
        return [ asSummaryMessage(boundary.SummaryOfEarlierConversation) ]
             + details where Sequence >= boundary.Sequence        # boundary row + tail, raw
    else:
        return all details                                        # no compaction yet
    # system prompt is added by the existing prompt-assembly path, on top of this.
```

The summary on the boundary row covers `< boundary.Sequence`; the boundary row itself and
everything after are included raw. No gap, no overlap.

### 1.3 Recursive (compounding) summarization

When we compact again later at sequence M, the summary prompt's **input is the previous
summary + only the raw delta (`boundary..M-1`)** — never a re-read of the full raw
history. Output is written to the row at M. The older summary at the previous boundary
simply stops being selected (it is not the most recent non-null summary). This bounds
compaction cost as conversations grow indefinitely (MemGPT/LangGraph `RunningSummary`
pattern).

### 1.4 Relationship to the in-turn compaction

The two tiers stack cleanly:
- **Tier A — cross-turn (new):** `GetAgentContextWindow` hands BaseAgent a pre-windowed
  message array (summary + tail) instead of the full raw history.
- **Tier B — in-turn (existing, unchanged):** `pruneAndCompactExpiredMessages` and the
  context-recovery escalation continue to manage pressure *within* a single run, now
  starting from a smaller set.

Tier A feeds Tier B. No changes to Tier B in this plan.

---

## 2. Where the work lives (architecture)

### 2.1 `ExecuteAgentParams` gains `conversationId` (preferred input)

Today BaseAgent receives `conversationMessages` assembled by the caller
(`RunAIAgentResolver` parses them from client JSON). We add an optional, **preferred**
`conversationId` param.

- **We do NOT always have a conversation id.** Programmatic runs, internal sub-agent
  invocations, and tests may have none. All compaction/windowing features are **gated on
  the presence of `conversationId`**.
- **Precedence when `conversationId` is present:** the assembly layer
  (`ConversationEngine.GetAgentContextWindow`) is authoritative and builds the windowed
  message array from cache; an explicitly passed `conversationMessages` is treated as an
  override/escape hatch (used by callers that deliberately supply their own context).
- **When `conversationId` is absent:** behave exactly as today — caller supplies
  `conversationMessages`, no persistent summary, Tier B only.

BaseAgent stays transport/DB-agnostic: it never queries the DB. The resolver (or any
server-side caller) resolves `conversationId` → windowed messages via `ConversationEngine`
before calling `Execute`.

### 2.2 Reuse `ConversationEngine` — do not add an engine

`ConversationEngine` already has the cache we want:
`_detailCache: Map<convId, ConversationDetailCache>`, lazily populated by a single
`GetConversationComplete` query and served from memory thereafter. It is **per-conversation
and on-demand** — it never bulk-loads all conversations, so the "engines are bad for big
tables" concern does not apply.

New method:
```typescript
// ConversationEngine
public async GetAgentContextWindow(
    conversationId: string,
    contextUser: UserInfo
): Promise<AgentChatMessage[]>   // summary-as-message + tail, or all details
```
First touch of a conversation pays one query (~tens of ms, once); every subsequent run is
a warm in-memory slice. No per-run DB query.

### 2.3 Cross-instance cache note (future)

`_detailCache` is per-process. In multi-instance MJAPI, run N+1 may hit a cold instance
(one query to warm — acceptable for v1). **Future option:** elevate this cache to the
Redis-backed `CacheManager` for cross-instance sharing. Flagged, not built.

---

## 3. Data model changes

### 3.1 `ConversationDetail`

| Column | Type | Notes |
|---|---|---|
| `Sequence` | `INT NOT NULL` | Stable, monotonic-per-conversation ordinal. The **symbolic handle** for retrieval tools and summary markers. Must be persisted (an in-memory index would renumber on cache reload and break markers). |
| `SummaryPromptRunID` | `UNIQUEIDENTIFIER NULL` | FK → `MJ: AI Prompt Runs`. Links a populated `SummaryOfEarlierConversation` to the prompt run that produced it (model/tokens/cost/version audit for free). |

- **Reuse `SummaryOfEarlierConversation`** (exists) for the summary text. No new field.
- **No `SummaryCoversThroughSequence`** — coverage is always "`< this row's Sequence`",
  so it is redundant (`= Sequence - 1`).
- **Edits:** reuse the existing `OriginalMessageChanged` boolean to flag a row edited after
  it was folded into a summary; Record Changes already captures the diff. No downstream
  regeneration required (see §6).

#### Sequence assignment (insert-time)

Concurrency risk is low (two simultaneous inserts in the *same* conversation is rare).
**Decision: a DB trigger** on `ConversationDetail` (`trgConversationDetail_AssignSequence`,
`AFTER INSERT`) assigns `Sequence = ISNULL(MAX(Sequence),0) + row-number-within-batch`
scoped to `ConversationID`, reading the current max under `UPDLOCK, HOLDLOCK` to stay safe
under the rare concurrent case. This covers **all** insert paths uniformly (agent flow,
metadata sync, manual) and avoids an app-layer round trip. The column carries
`DEFAULT (0)` so the row satisfies `NOT NULL` at insert; the `AFTER INSERT` trigger then
overwrites it with the real value (reflected back through CodeGen's `spCreate` SELECT).

#### Backfill (REQUIRED in the migration)

Backfill existing rows:
```sql
WITH numbered AS (
    SELECT ID,
           ROW_NUMBER() OVER (PARTITION BY ConversationID
                              ORDER BY __mj_CreatedAt ASC, ID ASC) AS rn
    FROM ${flyway:defaultSchema}.ConversationDetail
)
UPDATE cd SET cd.Sequence = numbered.rn
FROM ${flyway:defaultSchema}.ConversationDetail cd
JOIN numbered ON numbered.ID = cd.ID;
```
(Backfill runs before the column is switched to `NOT NULL` / before the trigger is relied
upon. Order the DDL accordingly.)

### 3.2 Agent context-control metadata (designer knobs)

The headline knob: **the effective context-window budget**. Making it bigger/smaller is
how an agent designer dials how much raw long-context to include vs. how much to lean on
RLM-style tooling. Defined at **`AIAgentType` (defaults)**, overridable at **`AIAgent`
(per-agent / per-sub-agent)**.

**`AIAgent` already has** `ContextCompressionMessageThreshold`,
`ContextCompressionPromptID`, `ContextCompressionMessageRetentionCount`. **`AIAgentType`
has none.** So part of this work is:

1. **Add the existing trio to `AIAgentType`** as type-level defaults (same names).
2. **Add new token-based fields to BOTH `AIAgentType` and `AIAgent`:**

| Column (both entities) | Type | Meaning |
|---|---|---|
| `ContextWindowMaxTokens` | `INT NULL` (both) | Effective working-context budget. `NULL` ⇒ inherit / use the selected model's `MaxInputTokens`. |
| `CompactionTriggerPercent` | `AIAgentType: INT NOT NULL DEFAULT 75` · `AIAgent: INT NULL` | % of effective budget at which cross-turn compaction fires. Type always provides a floor; agent `NULL` inherits. |
| `CompactionTargetPercent` | `AIAgentType: INT NOT NULL DEFAULT 30` · `AIAgent: INT NULL` | Target % of budget after compaction. Type always provides a floor; agent `NULL` inherits. |
| `ConversationSummaryPromptID` | `UNIQUEIDENTIFIER NULL` (both) | FK → `MJ: AI Prompts`. The **cross-turn** summary prompt (distinct from the in-turn `ContextCompressionPromptID`). |

The promoted trio on `AIAgentType` (`ContextCompressionMessageThreshold`,
`ContextCompressionPromptID`, `ContextCompressionMessageRetentionCount`) is `NULL`able
(compression is optional). Only the two percents are `NOT NULL DEFAULT` so the type level
always yields a usable value.

**Resolution order:** `AIAgent` value ?? `AIAgentType` value ?? (for
`ContextWindowMaxTokens` only) model `MaxInputTokens`. The two percents never bottom out at
a hardcoded constant — the type's `DEFAULT` guarantees a value.

**Model validation (per run, against the model about to run):**
- Resolve effective budget. If it exceeds the selected model's `MaxInputTokens`, **clamp to
  the smaller value** and **log a warning** to (a) the console and (b) the agent run log
  (e.g., a warning on the `AIAgentRun` / step). Never silently exceed the model.

### 3.3 `AIAgentRunStep` — record the compaction as a step

Cross-turn compaction is part of the agent run lifecycle and must be observable. Add a new
`StepType` value **`'Compaction'`** (current values: `Actions`, `Chat`, `Decision`,
`ForEach`, `Prompt`, `Sub-Agent`, `Tool`, `Validation`, `While`). The compaction step links
to the `AIPromptRun` of the summary prompt; that same `AIPromptRun.ID` is stored on
`ConversationDetail.SummaryPromptRunID`.

Lineage chain:
```
AIAgentRun → AIAgentRunStep (StepType='Compaction') → AIPromptRun (summary prompt)
                                                          ▲
                              ConversationDetail.SummaryPromptRunID ┘
```
The `summarizeRange` retrieval tool (§5.3) likewise records an `AIAgentRunStep` +
`AIPromptRun` for its sub-call.

### 3.4 Migration checklist (single file, `migrations/v5/`)

> **Implemented:** `migrations/v5/V202607201104__v5.49.x__Agent_Conversation_Compaction.sql`
> *(renumbered three times as `next` advanced under the open PR: from `V202606012156__v5.40.x`
> after merging `next` — the old timestamp was out-of-order vs. `next`'s v5.47.x migrations —
> then to v5.49.x post-5.48 release (commit `2cf0213185`), then to `V202607201104` after
> `next`'s `V202607191254` FK-index backfill leapfrogged the previous timestamp; the re-added
> StepType CHECK also now includes `'Plan'` and `'Skill'`, which `next` added in v5.44.)*

- `ALTER TABLE ConversationDetail ADD Sequence INT NULL, SummaryPromptRunID UNIQUEIDENTIFIER NULL` (single consolidated ALTER).
- Backfill `Sequence` (§3.1).
- Set `Sequence NOT NULL` after backfill.
- Create the Sequence-assignment trigger.
- `ALTER TABLE AIAgentType ADD` the trio + new token fields + `ConversationSummaryPromptID`.
- `ALTER TABLE AIAgent ADD` the new token fields + `ConversationSummaryPromptID`.
- Extend the `AIAgentRunStep.StepType` check constraint to include `'Compaction'`.
- `sp_addextendedproperty` for every new column (CodeGen descriptions).
- **Do NOT** add `__mj_CreatedAt/__mj_UpdatedAt` or FK indexes (CodeGen handles those).
- Hand off to CodeGen for entity classes / views / SPs. Write dependent TypeScript only
  **after** CodeGen generates the typed properties.

---

## 4. Cross-turn compaction (a prompt, not an agent)

- A single versioned MJ prompt (`ConversationSummaryPromptID`). One LLM call: **input =
  prior summary + raw delta**, **output = new summary**. No tools, no loop. Invoke via
  `AIPromptRunner.ExecutePrompt` so an `AIPromptRun` is recorded automatically.
- **Trigger:** model-relative — fire when estimated tokens cross `CompactionTriggerPercent`
  of the effective budget (resolved + validated per §3.2), reserving output headroom so the
  triggering turn never truncates. Use the existing `getModelContextLimit()` /
  `estimateTokens()` helpers.
- **When:** prefer **post-turn** (after the agent response is delivered) to hide latency;
  run **pre-turn** only if already over budget before assembling the prompt.
- **Recorded** as an `AIAgentRunStep` (StepType `'Compaction'`) on the current run.

### 4.1 Summary content: a lean **map**, plus a short prose gist

Per the RLM lesson, the summary is primarily an **addressable map**, not an exhaustive
recap:
> Inspired by the Recursive Language Models paradigm
> ([arXiv:2512.24601](https://arxiv.org/abs/2512.24601)): treat the full history as an
> addressable external environment and expose a compact handle, rather than relying on a
> lossy in-window summary.

- A brief **200–400 token prose gist** of the conversation (orientation only).
- **Sequence-numbered timeline markers** for decisions / key events / artifacts / actions
  (`decision X — seq 42`, `artifact Y introduced — seq 64`).
- **User messages kept high-fidelity** (verbatim or near-verbatim) to prevent task drift.
- **Per-agent ownership / handoffs** preserved (multi-agent, §7).
- Open tasks / unresolved questions.
- An explicit instruction block telling the consuming agent: **the markers + retrieval
  tools are the source of truth; the prose gist is lossy — do NOT rely on it for exact
  wording, IDs, or decisions; page in via tools instead.**
- Never invent; never treat in-progress/error turns as completed outcomes.

---

## 5. RLM-style retrieval tooling (first-class, not a fast-follow)

These are the "external environment + REPL + recursion" half — elevated, per the RLM
paper, because biasing the agent toward paging in exact slices beats trusting a lossy
summary. All tools default to the current conversation, are served from the
`ConversationEngine` cache (no per-call DB hit when warm), and are logged.

### 5.1 `getMessageBySequence(seq)` / `getMessagesByRange(start, end)`
Exact paging by symbolic handle. Range enforces a sane max to avoid context blow-up.

### 5.2 `searchConversation(query | regex, filters?)`
Grep/keyword/regex over pre-summary history, filterable by role / agent / sequence range.
Returns `{sequence, role, agent, snippet, matchType}` so the agent can then page the hit.

### 5.3 `summarizeRange(start, end, lens)` — the recursive sub-call
Spins a **sub-LM over a slice** of history and returns a focused summary through a
task-specific `lens`. This is the RLM recursion: the agent is not limited to the one
baseline summary's framing — it can re-summarize any window however the current task needs.
- **Model:** a small/cheap model (e.g. Gemini 3.1 Flash Lite tier), mirroring the paper's
  "cheap sub-call model, strong root model" split.
- **Recorded** as its own `AIAgentRunStep` + `AIPromptRun`.

(Existing artifact tools and action-result retrieval remain the way to inspect artifact
contents and historical action results; the summary references them by id, never inlines
them.)

### 5.4 Prior-turn tool-result carry-forward (+ process cache)

Inline tool results live only in the run that produced them — the next turn rebuilds its
messages from the conversation window, so a result paged in on turn N is gone on turn
N+1 and the agent would have to re-call the tool. `injectPriorTurnToolResults` re-injects
the same agent's immediately previous settled root run's successful read-tool results
(eligibility decided structurally by the `toolFamily` stamped in each Tool step's
`OutputData`) as one transient message (`expirationTurns: 2`, compactable). One-turn
memory by construction — context never compounds.

The lookup is kept off the hot path by `PriorTurnToolResultCache` (a `BaseSingleton`
wrapping `MJLruCache`, 500 entries / 30-min TTL, keyed by **conversation + agent** so in
a multi-agent conversation agent B never inherits agent A's results): the settling root
run publishes its own completed Tool-step projections from memory at `finalizeAgentRun`
— **including an empty array** for tool-free runs, the common case, which then costs
zero DB queries per turn — and the loader falls back to the original RunView pair only
on a cache miss (first turn, restart, other node). "Settled" means `Completed` OR
`AwaitingFeedback` (`BaseAgent.settledRunStatuses`): a Chat final step is the normal
per-turn ending for conversational agents, so gating on `Completed` alone would disable
carry-forward (and post-turn compaction, which shares the constant) for the most common
agent shape. Freshness on a node is mutation-driven (each settled run replaces the
entry); the TTL only bounds idle memory and the multi-node staleness window, and both
loaders share one typed predicate (`carryForwardPredicate` + the AgentID scoping at the
call sites) so the SQL filter and the in-memory projection cannot drift.

---

## 6. Edits, deletion, staleness

- **Deletion:** current UX only allows deleting a message *and everything after it* — which
  can never invalidate an earlier summary's coverage. Non-issue.
- **Edits (if/when allowed):** flag the row via the existing `OriginalMessageChanged`
  boolean; the diff is already captured by **Record Changes** (so the UI can surface
  "edited after summarization" cheaply). We **store the edit and the flag** rather than
  forcing downstream regeneration — the summary is explicitly lossy and the agent is
  instructed to page in exact rows anyway. Optional future: lazily regenerate the affected
  summary on next run.
- **Locking (considered, not v1):** hard-locking summarized rows was considered; given the
  deletion-only UX it is unnecessary for v1. Noted for future.

---

## 7. Multi-agent

- v1: **one shared baseline** on the shared thread with **explicit per-agent sections** in
  the summary map (it is a shared conversation; all participants legitimately see the shared
  summary). Preserve `AgentID` and handoff/mention events in the markers.
- Sub-agent runs keep their own isolated contexts as they do today (context isolation =
  "compaction by architecture").
- Agent-specific overlay summaries: future, not v1.

---

## 8. Telemetry

- Compaction cost/tokens/model: free via the summary `AIPromptRun` + the `'Compaction'`
  run step.
- Counters: tokens-before/after compaction, effective-budget clamp warnings,
  retrieval-tool calls by type, `summarizeRange` sub-call count.
- Surface the per-run effective budget + which model bounded it.

---

## 9. Phasing

> **STATUS: ALL PHASES SHIPPED (2026-07-13).** Implementation notes below record where
> the build deliberately refined the sketch; the sections above remain the design rationale.

1. **Migration** ✅ — `V202607201104__v5.49.x__Agent_Conversation_Compaction.sql`:
   `Sequence` (+ backfill wrapped in `DISABLE TRIGGER trgUpdateConversationDetail` so the
   ordinal assignment doesn't stamp UpdatedAt on every historical row + AFTER INSERT
   trigger), `SummaryPromptRunID`, composite index
   `IX_ConversationDetail_ConversationID_Sequence` (trigger MAX-read seek + retrieval
   lookups; review-requested), agent context-control fields on `AIAgentType`/`AIAgent`
   with range CHECKs on the compaction knobs (percents 1–100; Target < Trigger on
   AIAgentType only — AIAgent's NULL-inherit semantics push that invariant to the
   code-side clamp), `StepType='Compaction'` (CHECK re-added with next's full value set
   incl. `'Plan'`/`'Skill'`), and a hand-authored corrective tail (Sequence
   `AllowUpdateAPI=0` — trigger-owned ordinal; 12 `ExtendedType='Code'` restores the
   generated category sweep had nulled).
   PostgreSQL variant `migrations-pg/v5/V202607201104__v5.49.x__Agent_Conversation_Compaction.pg.sql`,
   produced in the PR-review round via the pg-migrate-v2 split-and-regenerate pipeline:
   SQLConverter for the mechanical sections + hand-authored PL/pgSQL trigger — BEFORE
   ROW with a per-conversation `pg_advisory_xact_lock` as the UPDLOCK/HOLDLOCK analog
   (correct for batch inserts via PG's SPI command-visibility rule; documented caveat:
   duplicate risk under SERIALIZABLE/RR isolation — MJ runs READ COMMITTED).
2. **CodeGen** ✅ — output appended to the migration per the separator convention (single
   artifact); fresh-DB migrate verified on SQL Server. The PostgreSQL fresh-DB migrate +
   batch-insert Sequence SELECT-back check run through the pg-migrate-v2 Phase 3/4 gates
   (workbench + `pg-migrations.yml` CI) as part of the PR-review round.
3. **Assembly layer** ✅ — the pure fold lives once in the static
   `ConversationEngine.AssembleContextWindow(rows, { excludeDetailIds?, maxTailMessages? })`
   (the cap applies ONLY when no boundary exists — legacy last-N parity; cutting a
   post-boundary tail would create a coverage gap). The cache-backed instance method
   (`GetAgentContextWindow`) delegates to it and remains the public API for
   programmatic/cache-warm consumers (today exercised by the integration suite — no
   production caller remains after the server paths moved off it); ALL server callers
   (the agent resolver's history loader and the compaction manager) load rows FRESH per
   request via the single-sourced `ConversationEngine.LoadWindowRowsFresh` (per-request
   provider + contextUser: entity RLS applies; load failure fails the mutation loudly;
   the process-global engine cache is never populated server-side — no unbounded MJAPI
   growth, no cross-node staleness) and fold through the same function. `ExecuteAgentParams.conversationId`; the in-flight
   agent-response placeholder row is excluded from BOTH the resolver window and the
   compaction pass (`CompactIfNeededInput.ExcludeDetailIds`), and the boundary selector
   additionally never picks the newest raw row.
4. **Cross-turn compaction** ✅ — `ConversationCompactionManager` (static
   `ResolveEffectiveBudget` + `CompactIfNeeded`). Post-turn hook is **fire-and-forget after
   the run row saves** (an awaited pass would delay the caller's completion event); the
   pre-turn fallback runs synchronously but ONLY under an explicitly configured budget
   (pre-prompt the model is unknown, and the conservative default would over-trigger on
   large-context models) and splices the summary into live messages in place, preserving
   attachment-enriched tails. Quiet no-ops record no step; fired/failed passes record the
   `'Compaction'` step with `TargetLogID` = the summary `AIPromptRun`.
5. **Retrieval tools** ✅ — `ConversationToolManager` via the `conversationToolCalls`
   inline response field (mirrors `artifactToolCalls`: zero turn cost, per-call
   `StepType='Tool'` steps, one-shot next-turn result messages, `_CONVERSATION_TOOLS`
   docs block, `includeConversationToolsDocs` prompt param).
6. **Recursive tool** ✅ — `summarizeRange` through a `ConversationToolSummaryHost` seam
   (BaseAgent runs the seeded `Summarize Conversation Range` prompt on cheap models). The
   sub-call's `AIPromptRun` links via the existing Tool step's `TargetLogID` — one step +
   one prompt run per call rather than a duplicate `'Prompt'` step.
7. **Edit handling** ✅ — `MJConversationDetailEntityServer.ShouldFlagOriginalMessageChanged`
   (the pre-existing implementation had an inverted `IsSaved` check and never fired); the
   predicate exempts the framework's own Message rewrites (In-Progress progress updates,
   the finalization Status-transition save).

Deterministic coverage lives in
`packages/MJServer/integration-test-scripts/conversation-compaction-tests.ts` plus vitest
suites in `@memberjunction/ai-agents`, `@memberjunction/core-entities`, and
`@memberjunction/core-entities-server`; live end-to-end (real agent + real summary model,
lineage assertions) was verified against a workbench database during the build.

### 9.1 Handoff — CodeGen must run locally before any code is written

The migration is committed but **not yet applied**. The next action is to **run the
migration + CodeGen on a local environment** (CodeGen scans the live DB schema + extended
properties to regenerate generated code). This step is intentionally done locally, not in
this remote session, because it needs a database.

**Steps (local):**
1. Pull branch `claude/sharp-faraday-nB59F`.
2. Apply migrations (Flyway/Skyway) so the schema changes land in the dev DB.
3. Run CodeGen. Expected regenerated/changed (do **not** hand-edit these):
   - `packages/MJCoreEntities/src/generated/entity_subclasses.ts` — new typed properties:
     `ConversationDetailEntity.Sequence` / `.SummaryPromptRunID`; the new
     `AIAgentTypeEntity` and `AIAgentEntity` context-control fields; `AIAgentRunStepEntity`
     `StepType` union now including `'Compaction'`.
   - `packages/MJServer/src/generated/*` — resolvers/types.
   - Generated Angular core-entity-forms for the touched entities.
   - A `migrations/v5/CodeGen_Run_*.sql` capturing the metadata sync.
4. Build affected packages; commit the generated output (separate commit, e.g.
   "CodeGen output for conversation compaction schema").

**Only after CodeGen** is it safe to write the strongly-typed code in Phases 3–7 (per the
repo rule: never reference fields via `.Get()/.Set()` or before generated types exist).

### 9.2 Resuming after CodeGen

Work can resume **either** locally **or** back in this remote session. To resume here:
push the CodeGen output to the branch first, then pick up at **Phase 3** (the assembly
layer) — start with `ConversationEngine.GetAgentContextWindow` and the
`ExecuteAgentParams.conversationId` wiring, since everything downstream depends on it. This
plan file is the source of truth for the remaining phases.

---

## 10. Open items to confirm during build

- Exact default values for `CompactionTriggerPercent` (~75) / `CompactionTargetPercent`
  (~30) and whether they belong as system constants or config.
- Whether the in-turn `ContextCompressionPromptID` and the new cross-turn
  `ConversationSummaryPromptID` should ship with distinct seeded prompts (recommended) or
  share one.
- Trigger-assignment final call (DB trigger vs. app-layer) once the dominant
  ConversationDetail insert paths are confirmed.
- Cross-instance cache: keep per-process for v1; revisit `CacheManager`/Redis if warm-miss
  latency matters in production.

---

## 11. What we explicitly are NOT doing

- No new `DetailType`/`ConversationDetailType` field.
- No separate `ConversationCompaction` metadata table (the `AIPromptRun` +
  `SummaryPromptRunID` cover audit).
- No deletion of original history (ever).
- No change to the existing in-turn compaction (Tier B) behavior.
- No bulk-load engine over `ConversationDetail` (reuse `ConversationEngine`'s lazy
  per-conversation cache).