# `@memberjunction/codegen-lib` — CodeGen

CodeGen keeps the database schema, TypeScript types, SQL objects, and Angular UI components in
sync. This file covers what it generates, when it runs, how it connects to the database, and the
class-registration manifest system.

## What CodeGen generates and maintains

1. **Entity classes** (`packages/MJCoreEntities/src/generated/entity_subclasses.ts`)
   - TypeScript classes for all database entities, Zod schemas with validation rules
   - Strongly-typed getters/setters, FK relationships, computed fields
   - Value list unions derived from database CHECK constraints

2. **Database objects** (`migrations/v5/CodeGen_Run_*.sql`)
   - Stored procedures (`spCreate`, `spUpdate`, `spDelete`)
   - Views with proper joins and computed fields
   - Foreign key indexes (`IDX_AUTO_MJ_FKEY_<table>_<column>`)
   - Database permissions and security grants
   - Entity field metadata synchronization

3. **Angular UI components** (`packages/Angular/Explorer/core-entity-forms/src/lib/generated/`)
   - Complete CRUD forms per entity, form field components with proper types
   - Dropdown lists populated from FK relationships
   - Validation derived from database constraints

4. **Server APIs** (`packages/MJServer/src/generated/generated.ts`)

### Field-level security touches three of the above

CodeGen owns part of the field-level-security (FLS) lifecycle, so a change in any of these areas
needs [`guides/FIELD_LEVEL_SECURITY_GUIDE.md`](../../guides/FIELD_LEVEL_SECURITY_GUIDE.md) read
first:

- **Permission reconciliation** (`reconcileFieldLevelSecurity.ts`, phase `reconcileFieldPermissions`)
  — a column added to an FLS-enabled entity has no permission rows, and on an enabled entity a field
  with no rows is **denied**. Without this pass a new column is invisible to everyone, including the
  administrator who added it. Must run **after** the metadata refresh that follows `manageMetadata`,
  or it computes the delta from a field list that predates the columns it exists to cover. Failures
  are logged and swallowed on purpose — failing the run after schema, views and procs are already
  written would trade a recoverable permissions gap for an unrecoverable half-finished build.
- **Database permissions** — SQL Server emits column-level `DENY SELECT` on base views for explicit
  `ReadAccess = 'Deny'` rows, restricted to custom roles and skipping any role a service login
  belongs to (a DENY there beats every sibling GRANT and would break the API for everyone).
  **PostgreSQL emits nothing** — it has no DENY primitive, so Deny-wins cannot be expressed. Related
  and not FLS-specific: permission emission is now **wipe-and-reassert** within the managed scope,
  so deleting a permission row actually revokes the grant instead of leaving it until the view
  happens to be rebuilt.
- **GraphQL output types** — non-nullability derives from `EntityFieldInfo.IsUnrestrictableField`
  (primary keys and `__mj_` columns), **not** from `AllowsNull`. A NOT NULL column says no *row*
  stores an empty value; a GraphQL `!` says every *response, to every caller* carries one, and the
  second does not follow from the first once a field can be withheld per user. Input types still
  derive from `AllowsNull` — they carry the write contract. Every object type also carries the
  `ReadableFields___` transport field.

## Base views: generated, custom, or LAYERED

An entity's `BaseView` is its public surface — field discovery, permissions and the generated CRUD
routines all target it. Two `Entity` columns decide who writes it:

| `BaseViewGenerated` | `GeneratedBaseViewName` | Result |
|---|---|---|
| `1` | `NULL` | CodeGen writes `BaseView`. The default. |
| `0` | `NULL` | The app owns `BaseView` entirely; CodeGen writes nothing. |
| `0` | `vwFooGenerated` | **Layered** — CodeGen writes the inner view, the app wraps it. |

**Prefer LAYERED over fully custom.** Fully custom means the application inherits ~80 lines of
generated SQL — every display join, the geo join, the recursive root-ID apply — to add one column,
and must hand-maintain it forever. A foreign key added later then **silently** never appears: the
column is absent rather than wrong, so nothing errors and no test notices. Layering keeps all of that
regenerating underneath:

```sql
CREATE VIEW [orders].[vwOrderHeaders] AS
SELECT g.*, CASE WHEN ... END AS IsOverdue
FROM   [orders].[vwOrderHeadersGenerated] g;
```

PostgreSQL: CodeGen writes the inner view the same way. The outer view is custom SQL shipped via
pg-migrate. After inner regeneration, CodeGen restars the outer (`restarLayeredOuterView` /
`spRebindLayeredOuterView`) so `g.*` re-expands. `CREATE OR REPLACE` of the inner view alone does
**not** update the outer.

Rules if you touch this:

- **Use `EntityInfo.GeneratedViewName`**, never re-derive from `BaseView`. It is the one answer to
  "which view does CodeGen write"; several call sites decide where to write, what to name the file,
  and what to refresh, and any two disagreeing produce a view under a name nothing reads.
- **`EntityInfo.HasLayeredBaseView`** is the layering test. It compares names case-insensitively —
  a view cannot select from itself, and a CHECK constraint on `Entity` refuses equal names too.
  `GeneratedViewName` is derived FROM it, so the two cannot drift; keep it that way rather than
  re-testing the raw column.
- **Refresh inner before outer.** The custom layer does `SELECT g.*` and a view caches its column
  list; refreshing the outer against a stale inner re-caches the old columns and the new one stays
  missing. CodeGen already emits `sp_refreshview` in that order — keep it that way.
- **Guard anything aimed at the outer view.** CodeGen refreshes and grants on `BaseView` but never
  creates it, and on the first pass after layering is enabled it does not exist yet — it selects
  from the inner view that pass is creating. Emit those through
  `generateIfViewExistsSQL`, or the run that is supposed to bootstrap the arrangement fails.
- **CRUD routines stay on `BaseView`.** They return the affected row, so custom columns come back on
  create/update/delete. Do not point them at the inner view.

## When CodeGen runs

CodeGen runs when:
- Database schema changes are detected (new tables, columns, constraints)
- Entity metadata is updated in the MJ metadata tables
- Field descriptions or validation rules change
- Foreign key relationships are added or modified

Common triggers: `ALTER TABLE` adding columns, adding CHECK constraints or foreign keys, updating
`sp_addextendedproperty` descriptions, modifying value lists in `EntityFieldValue`, adding new
entities to the `EntityField` metadata.

### Worked example: adding `PromptRole` / `PromptPosition`

1. The **migration** creates the columns with constraints
2. **CodeGen detects** the schema change automatically
3. **Generated code** appears in three places:

```typescript
// entity_subclasses.ts
PromptRole: z.union([z.literal('System'), z.literal('User'), z.literal('Assistant'), z.literal('SystemOrUser')])

get PromptRole(): 'System' | 'User' | 'Assistant' | 'SystemOrUser'
set PromptRole(value: 'System' | 'User' | 'Assistant' | 'SystemOrUser')
```
```sql
-- CodeGen migration file
INSERT INTO EntityField (Name, Type, Description, ...)
INSERT INTO EntityFieldValue (Value, Code, ...)  -- dropdown options
```
```html
<!-- Angular form component -->
<mj-form-field FieldName="PromptRole" Type="dropdownlist" />
```

## Working with CodeGen

**✅ Do:**
- Run CodeGen after every schema change
- Review generated migration files before applying
- Use entity field descriptions — they become the generated documentation

**❌ Don't:**
- Modify files in any `/generated/` directory (they're overwritten)
- Skip CodeGen after database changes
- Assume TypeScript types are current without running CodeGen
- Hand-write CRUD operations — CodeGen owns them

---

## CodeGen Database Connections (SQL Server + PostgreSQL)

CodeGen (`mj codegen`) is a separate process from MJAPI with its own short-lived pool, configured
via `codegenPool` at the top level of `mj.config.cjs`. (The runtime MJAPI pool is
`databaseSettings.connectionPool` — see [`packages/MJAPI/CLAUDE.md`](../MJAPI/CLAUDE.md).)

```javascript
module.exports = {
  // ... other top-level codegen-lib config (dbHost, codeGenLogin, etc.)
  codegenPool: {
    // PG-only today (mssql doesn't honor these from this block yet)
    max: 20,                        // Max pool connections
    min: 2,                         // Min idle connections kept open
    idleTimeoutMillis: 30000,       // Close idle connections after this many ms
    connectionTimeoutMillis: 30000, // New-connection acquisition timeout
    ssl: false,                     // PG SSL (default false — matches the pre-refactor inline pool)

    // Cross-platform (both providers honor it)
    statementTimeoutMs: 120000,     // Per-statement timeout (ms)
  },
};
```

**Per-provider applicability** — not all fields apply to both providers today:

| Field | SQL Server | PostgreSQL |
|---|---|---|
| `statementTimeoutMs` | ✅ mssql `requestTimeout` | ✅ libpq `-c statement_timeout` |
| `max` / `min` / `idleTimeoutMillis` / `connectionTimeoutMillis` | ❌ ignored | ✅ `pg.Pool` config |
| `ssl` | ❌ ignored (SQL Server uses `dbTrustServerCertificate` + mssql's own SSL) | ✅ `pg.Pool` ssl |

The PG-only pool-sizing knobs reflect the asymmetry between mssql and `pg.Pool` configurability
today; they'll converge in a follow-up.

All fields are **optional** — when omitted, each driver's own defaults apply (mssql: 10 max +
`requestTimeout` 120000; `pg.Pool`: 20 max, 2 min, SSL off in codegen). This matches historical
CodeGen behavior, so adding the block is opt-in tuning, not a required change.

### Behavior

- **Lazy + module-cached pool**: both `MSSQLConnection()` (SQL Server) and `PGConnection()` (PostgreSQL) build their config and open the pool on first call, then cache the pool at the module level so repeated CodeGen operations within a single process reuse the same pool. The config is built **after** `initializeConfig()` runs, so config values from `mj.config.cjs` / `.env` are picked up correctly (the previous module-load-time destructure produced empty values when callers did `await import('@memberjunction/codegen-lib')` before `initializeConfig()`).
- **Platform dispatch via factory**: `RunCodeGenBase.setupDataSource()` resolves the concrete `CodeGenDatabaseProvider` via `MJGlobal.Instance.ClassFactory.CreateInstance(CodeGenDatabaseProvider, configInfo.dbPlatform)` and calls its `SetupDataSource()` method. Adding a new platform is `@RegisterClass(CodeGenDatabaseProvider, 'newplatform')` on the new provider class — no orchestrator changes.
- **`statementTimeoutMs`** is the cross-platform per-statement timeout. On SQL Server it maps to the mssql pool's `requestTimeout` (and takes precedence over the legacy top-level `dbRequestTimeout` / `MJ_CODEGEN_REQUEST_TIMEOUT` when both are set). On PostgreSQL it is carried via the libpq `-c statement_timeout=<ms>` startup option, so the server applies it from connection #1 — including the verify-`SELECT 1` connection that `PGConnectionManager.Initialize()` opens. When unset, each driver applies its own default (mssql: 120000ms; PG: no statement timeout).
- **`ssl` (PostgreSQL only)**: defaults to `false` to preserve the pre-multi-provider-refactor inline `pg.Pool` behavior (no SSL key passed → pg default OFF), so local/non-SSL codegen runs against PostgreSQL aren't broken by `PGConnectionManager`'s production-environment SSL auto-default. Set explicitly when the target Postgres requires SSL.

### CodeGen Environment Variables

CodeGen-time connection params come from `configInfo.dbHost` / `dbPort` / `dbDatabase` /
`codeGenLogin` / `codeGenPassword`, resolved (in order) from `mj.config.cjs`, then env vars, then
defaults. When `dbPlatform === 'postgresql'`, the PG-prefixed env vars take precedence over their
SQL-Server-named siblings — so an existing PG-targeted `.env` keeps working without renaming:

| Field            | PostgreSQL env (preferred)  | SQL Server / generic env | Fallback |
|------------------|-----------------------------|--------------------------|----------|
| `dbHost`         | `PG_HOST`                   | `DB_HOST`                | `localhost` |
| `dbPort`         | `PG_PORT`                   | `DB_PORT`                | `5432` (PG) / `1433` (SQL Server) |
| `dbDatabase`     | `PG_DATABASE`               | `DB_DATABASE`            | `''` |
| `codeGenLogin`   | `PG_USERNAME`               | `CODEGEN_DB_USERNAME`    | `''` |
| `codeGenPassword`| `PG_PASSWORD`               | `CODEGEN_DB_PASSWORD`    | `''` |

Env-var precedence is resolved **once**, in `DEFAULT_CODEGEN_CONFIG` inside `Config/config.ts`.
CodeGen provider code reads `configInfo.*` directly — `process.env.PG_*` is not consulted at the
connection layer. This keeps env resolution in one place and avoids the two-layer trap of
"resolved at config time, then re-resolved at connection time."

When both env vars in a row are set AND they differ (e.g. `PG_HOST=postgres.dev` AND
`DB_HOST=localhost`), `Config/config.ts` emits a one-line `console.warn` at module load
identifying which value wins and which is being ignored. The PG-prefixed value continues to take
precedence (existing behavior), but the silent override is now visible. Set only one — or set both
to the same value — to silence the warning.

---

## Class Registration Manifests (Tree-Shaking Prevention)

MemberJunction uses `@RegisterClass` decorators with a dynamic class factory
(`MJGlobal.ClassFactory`). Modern bundlers (ESBuild, Vite) cannot detect dynamic instantiation and
tree-shake these classes out. The **manifest system** prevents this.

**How it works**: `mj codegen manifest` walks the dependency tree, finds all
`@RegisterClass`-decorated classes via TypeScript AST, and emits a manifest with named imports +
an exported `CLASS_REGISTRATIONS` array that creates a static code path the bundler cannot
eliminate.

**Dual-manifest architecture for distribution:**
- **Pre-built manifests** ship inside bootstrap packages (`@memberjunction/server-bootstrap`, `@memberjunction/ng-bootstrap`). Generated at MJ build time; cover all `@memberjunction/*` classes.
- **Supplemental manifests** are generated by MJAPI/MJExplorer's `prestart`/`prebuild` scripts with `--exclude-packages @memberjunction` to capture only user-defined classes.
- This solves the npm distribution gap: published packages only have `dist/` (no `src/`), so the manifest generator can't scan them externally.

**Key scripts:**
- `pnpm run mj:manifest` — regenerates all **9** manifests, serially (server-bootstrap,
  server-bootstrap-lite, ng-bootstrap, ng-bootstrap-lite, MJAPI, MJExplorer, A2AServer,
  MCPServer, MJCodeGenAPI). Runs automatically from the root `postbuild`.
- `pnpm run mj:manifest:server-bootstrap` / `:server-bootstrap-lite` / `:ng-bootstrap` /
  `:ng-bootstrap-lite` — regenerate the pre-built bootstrap manifests
- `pnpm run mj:manifest:api` / `:explorer` / `:a2a-server` / `:mcp-server` / `:codegen-api` —
  regenerate the app supplemental manifests

> The two `ng-bootstrap*` manifests ship to the **browser**. Regenerating them can pull a
> server-only package into the bundle — run `pnpm run check:browser-manifest` afterwards
> (CI runs it too). See [`packages/Angular/Bootstrap/CLAUDE.md`](../Angular/Bootstrap/CLAUDE.md).

> **Ordering caveat.** These 9 steps are not order-independent: each runs with
> `syncDependencies` on, so it may rewrite `<appDir>/package.json` while another step is
> walking it. The current serial order does not fully respect that — see
> [`plans/manifest-generation-parallelization.md`](../../plans/manifest-generation-parallelization.md)
> before reordering or parallelizing them.

**See**: [CLASS_MANIFEST_GUIDE.md](../../plans/complete/codegen/CLASS_MANIFEST_GUIDE.md) for comprehensive
documentation on the manifest system, including how external consumers and MJ distribution users
should configure their projects.

> ⚠️ The browser-facing manifest packages carry a hard "no server-only dependencies" rule —
> see [`packages/Angular/Bootstrap/CLAUDE.md`](../Angular/Bootstrap/CLAUDE.md) and
> [`packages/Angular/BootstrapLite/CLAUDE.md`](../Angular/BootstrapLite/CLAUDE.md).

## Idempotency and Churn-Free CodeGen Contract

CodeGen guarantees **100% idempotency relative to database state** and **minimal blast radius** for schema changes:

1. **Idempotency (No-Change Runs)**:
   - Running CodeGen twice against an unchanged database state produces **0 diffs** across all generated code, schemas, and forms.
   - Run 2 reports counters: `fieldsNew = 0`, `fieldsChanged = 0`.
   - Empty SQL capture files (`CodeGen_Run_*.sql`) are automatically removed upon run completion; no empty migration artifacts survive.

2. **Minimal Blast Radius (Single-Column Changes)**:
   - Adding one column to one table strictly modifies **only** that entity's artifacts (`__mj.ts`, specific entity Zod/schema JSON, `generated.ts` type block, and `mjentity.form.component.*`).
   - Sibling fields on the entity are untouched: existing `DisplayName`, `Category`, `ExtendedType`, `CodeType`, `GeneratedFormSection`, `DefaultInView`, `IncludeInUserSearchAPI`, and `IsNameField` do not churn.
   - `generated-forms.module.ts` is not modified by adding a column (only by adding or deleting entire entities).

3. **Field Metadata Lock & Migration Single Source of Truth**:
   - Field categorization and metadata decisions are locked in the database via the field-metadata lock (`field-metadata-lock.ts`) and committed via the migration's CodeGen capture SQL, making migration SQL the authoritative single source of truth.
   - Re-runs against an existing schema lock established categories and metadata unless the underlying schema definition materially changes.

4. **Stable Partitioning & Deterministic Ordering**:
   - Submodule partitioning uses stable hash buckets based on entity names rather than array index-chunking, preventing ripple effects across form submodules.
   - All sorting (entities, fields, relationships) uses deterministic ordinal comparisons (`OrdinalCompare` / `String_CS_AS`) across SQL Server and PostgreSQL.

## Related

- **Migration authoring rules** — [`migrations/CLAUDE.md`](../../migrations/CLAUDE.md)
- **Migration → CodeGen end-to-end workflow** — [`guides/MIGRATION_CODEGEN_WORKFLOW_GUIDE.md`](../../guides/MIGRATION_CODEGEN_WORKFLOW_GUIDE.md)
- **Generated entity classes** — [`packages/MJCoreEntities/CLAUDE.md`](../MJCoreEntities/CLAUDE.md)
- **Field-level security** — [`guides/FIELD_LEVEL_SECURITY_GUIDE.md`](../../guides/FIELD_LEVEL_SECURITY_GUIDE.md)
- **PostgreSQL schema casing** — [`guides/POSTGRES_SCHEMA_CASING_GUIDE.md`](../../guides/POSTGRES_SCHEMA_CASING_GUIDE.md)