primaryKeyFlagsChanged(
    current: { isPrimaryKey: boolean; isUnique: boolean },
    want: { wantPrimaryKey: boolean; wantUnique: boolean },
    makePrimaryKey: boolean,
    reconcile: boolean,
): boolean
Protected_ProtectedapiProtecteddbReturns the CodeGenDatabaseProvider for the current database platform. Lazily initialized from dbPlatform() configuration.
Lookup goes through MJGlobal.ClassFactory keyed by the platform string,
which matches the @RegisterClass(CodeGenDatabaseProvider, '<platform>')
decorators on the concrete providers ('sqlserver', 'postgresql').
Mismatched keys silently fall back to the abstract base class — fail loud
with an explicit error so a misconfigured platform doesn't ship as a
runtime breakage in dialect-specific methods.
ProtecteddialectReturns the SQLDialect for the current database platform, derived from the provider.
ProtectedtimestampReturns the timestamp column type name for this platform. Delegates to the database provider.
StaticchangedUse ChangedFieldCount.
StaticChangedStaticchangedUse ChangedFieldReport.
StaticChangedStaticdeletedStaticDeletedSchemas that lost an entity during this run.
Deletion cannot be reported the way new/modified entities are. Those lists carry entity NAMES, which downstream code resolves to a schema by looking the entity up in live metadata — and a deleted entity is, by then, no longer there to look up. So the schema is captured here at deletion time instead.
Without this the schema is never marked dirty, its per-schema file is never rebuilt, and
the dead class keeps a live @RegisterClass registration on disk. The old monolith could
not drift this way: it was rewritten in full every run.
StaticEntitiesEntities that had late-phase changes requiring their base views to be regenerated after the main SQL generation pass. Each entry includes the reason for regen so downstream logic can apply reason-specific fixups.
StaticEntitiesEntities bound to a JSONType whose SQL @CHECK translation was newly generated this run. Their
generated files must be rebuilt even though no column changed, or a full run pays for the
translation and then skips emitting it (dirty-schema scoped emit only sees schema changes).
StaticGeneratedGlobally scoped translations of the SQL @CHECK rules of opted-in JSONTypes, loaded from
__mj.GeneratedCode (and extended by generation). Consumed when entity subclasses are emitted.
StaticgeneratedUse GeneratedValidators.
StaticGeneratedGlobally scoped list of validators that have been generated during the metadata management process.
StaticmodifiedUse ModifiedEntityList.
StaticModifiedGlobally scoped list of entities that have been modified during the metadata management process.
StaticnewUse NewEntityList.
StaticNewGlobally scoped list of entities that have been created during the metadata management process.
StaticnewUse NewFieldCount.
StaticNewProtectedaddAdds default permissions for a newly created entity based on config settings. Shared by both table-backed entity creation and virtual entity creation.
ProtectedaddAdds default ApplicationRole records for a newly created application based on config settings. This grants configured roles access to the new application automatically.
ProtectedaddAdds a newly created entity to the application(s) that match its schema name. If no application exists for the schema and config allows it, creates one. Shared by both table-backed entity creation and virtual entity creation.
ProtectedaddGrants read permissions to a newly-minted materialized QUERY entity scoped to the INTERSECTION of source read access (C2). A precomputed snapshot has no per-row scoping, so a role may read it only if it can read EVERY source entity — otherwise a user who cannot read a source could read its rows through the snapshot. Starts from the same config-default role list as addDefaultPermissionsForEntity (never grants a role the defaults wouldn't) and drops any role that lacks explicit read on every source. Always read-only (the entity is a virtual entity with no CRUD sprocs). If the intersection is empty the entity receives no role-based read grants — the fail-closed direction (§10); a human can grant read after review.
ProtectedapplicationProtectedapplyApply Advanced Generation features - Smart Field Identification and Form Layout Generation
ProtectedapplyUpserts FieldCategoryInfo (new format) and FieldCategoryIcons (legacy format) in EntitySetting.
Optional_entityName: stringProtectedapplyApplies the VirtualEntities keys, then refreshes metadata and rebuilds relationships when a key changed. Runs on every CodeGen run, right after the view-column sync.
ProtectedapplyGenerate SQL UPDATEs for DefaultInView on the identified default view fields
OptionalentityOrCtx: OptionalmaybeCtx: {ProtectedapplySets the entity icon if the entity doesn't already have one.
OptionalentityName: stringProtectedapplyApplies entity importance analysis to MJApplicationEntity records. Only called for NEW entities to set DefaultForNewUser.
OptionalentityName: stringProtectedapplyGenerate SQL UPDATEs for entity-level AllowUserSearchAPI.
Guardrail: even if the LLM proposes enabling AllowUserSearchAPI on an entity that looks like an audit/log/run-history table by name, refuse — those tables grow unboundedly, the LIKE fan-out across them dominates global-search latency, and they're virtually never the right target for a user-facing search box. The team can still enable search on such an entity by setting AutoUpdateAllowUserSearchAPI=0 and flipping the flag manually.
ProtectedapplyApplies category, display name, extended type, and code type to entity fields. Enforces stability rules: fields with existing categories cannot move to NEW categories. All SQL updates are batched into a single execution for performance.
ProtectedapplyApply form layout generation results to set category on entity fields. Delegates to shared methods for category assignment, icon, and category info persistence.
Database connection pool
Entity fields
Form layout result from LLM
If true, apply entityImportance; if false, skip it
ProtectedapplyGenerate SQL UPDATEs for entity-level and field-level FullTextSearch configuration
Optionalctx: {ProtectedapplyApplies LLM-generated field descriptions to entity fields that lack descriptions. All SQL updates are batched into a single execution for performance.
ProtectedapplyApplies LLM-identified foreign keys to entity fields. Sets RelatedEntityID, RelatedEntityFieldName, and IsSoftForeignKey=1. Only applies high and medium confidence FKs. All SQL updates are batched into a single execution for performance.
ProtectedapplyApplies LLM-identified primary keys to entity fields. Sets IsPrimaryKey=1 and IsSoftPrimaryKey=1 for identified fields. First clears any default PK that was set by field-sync (field #1 fallback). All SQL updates are batched into a single execution for performance.
ProtectedapplyGenerate SQL UPDATEs for IsNameField — SINGLE-WINNER semantics.
The LLM's result.nameFields is a RANKED candidate list (it may propose several
fields that "together" name the record, e.g. FirstName + LastName), but every
downstream consumer — EntityInfo.NameField, the base-view generator's FK-name
virtual columns, RelatedEntityNameFieldMap resolution — assumes exactly ONE name
field per entity. Historically this method flagged EVERY candidate and never
cleared anything, so flags accumulated across runs (57 core entities ended up with
2–4 IsNameField=1 rows) and the FK-name pick silently DRIFTED between CodeGen
runs as the first-by-sequence winner changed (observed: Conversation Details
flipping Message → Role, reshaping every view that joins to it).
Winner selection (deterministic, mirrors EntityInfo.NameField's runtime
preference and the v5.41 metadata backfill):
Name, else the first in field order (fields arrive ordered by
Sequence from the metadata query).Every NON-winner with IsNameField=1 and AutoUpdateIsNameField=1 is CLEARED, so
historical accumulation self-heals on the next analysis pass. Fields pinned with
AutoUpdateIsNameField=0 are never touched in either direction — the user owns
them, and EntityInfo.NameField's literal-Name preference arbitrates at runtime.
Optionalentity: { Name?: string }ProtectedapplyApplies schema-level entity name prefix/suffix rules from additionalSchemaInfo.json. The file may contain a top-level "Schemas" array with entries like:
{ "name": "CRM", "entityNamePrefix": "CRM: ", "entityNameSuffix": "" }
These are merged into the cached SchemaInfo records so that getNewEntityNameRule()
can resolve them during entity creation. Config-file rules (mj.config.cjs) still
take priority over these — this just provides a fallback from DBAutoDoc output.
ProtectedapplyGenerate SQL UPDATEs for IncludeInUserSearchAPI on the identified searchable fields.
Guardrails: even if the SmartFieldIdentificationResult flags a field as searchable, we refuse to enable IncludeInUserSearchAPI when the field is a primary key, a non-text type (LIKE forces an implicit per-row CONVERT and never seeks an index), or an unbounded text type whose parent entity is not FTX-enabled (the LIKE path cannot seek MAX/ntext/text columns; FTX is the right tool there). These are always-wrong cases — they cause the data provider to emit unindexed scans on every search.
Optionalentity: { FullTextSearchEnabled?: boolean; Name?: string }Optionalctx: {ProtectedapplyRun the deterministic search-flag hygiene pass (see buildSearchFlagHygieneSQL).
Runs OUTSIDE applyAdvancedGeneration and therefore regardless of whether advanced
generation is enabled — which is the whole point, since it is off by default and an entity
created under that default is exactly the one that needs this. Ordered after it so the
model's choices, when it did run, are already in place and the seed only fills a real gap.
COMPARE FIRST, then write — the T20 contract every-run config writers are held to (see
__tests__/idempotency/config-writers-compare-first.test.ts). The two UPDATEs converge on
their own, because the first pass makes noSearchableField false for every row it touches,
so re-running them changes nothing. That is not sufficient: LogSQLBatchAndExecute writes
whatever it is handed into the run's CodeGen_Run_*.sql capture whether or not a row moves,
and the drift gate's warm-twice stage fails on ANY capture surviving a second run. Emitting
unconditionally therefore reddened the gate on every PR while the data was perfectly
idempotent — the data converged, the log did not. So each statement is emitted only when its
own candidate probe finds work.
ProtectedapplyGenerate SQL UPDATEs for UserSearchPredicateAPI on each searchable field
OptionalentityOrCtx: OptionalmaybeCtx: {ProtectedapplyApply smart field identification results to entity fields and entity-level search configuration
Optionalctx: {ProtectedapplySets RelatedEntityID, RelatedEntityFieldName and IsSoftForeignKey = 1 on each configured foreign
key column. RelatedEntityID is the source of truth, IsSoftForeignKey protects it from schema sync.
A field in skipFields (declared with conflicting targets) and a target column the related
entity does not have are skipped; a target that is not unique on its own is applied with a warning.
the number of UPDATEs written
ProtectedapplyWrites the soft keys of each config entry, compare-first: a column that is already set, or that does not exist, gets no UPDATE.
clear IsUnique on the columns of a multi-column key. Only for virtual entities: no schema sync maintains their IsUnique.
the number of UPDATEs written
ProtectedapplyApplies soft PK/FK configuration from a JSON file specified in mj.config.cjs (additionalSchemaInfo property). For soft PKs: Sets BOTH IsPrimaryKey=1 AND IsSoftPrimaryKey=1 (IsPrimaryKey is source of truth, IsSoftPrimaryKey protects from schema sync). For soft FKs: Sets RelatedEntityID/RelatedEntityFieldName + IsSoftForeignKey=1 (RelatedEntityID is source of truth, IsSoftForeignKey protects from schema sync). All UPDATE statements are logged to migration files via LogSQLAndExecute() for CI/CD traceability. Covers table entries only. VirtualEntities entries are applied after the view-column sync by applyVirtualEntitySoftKeys.
ProtectedapplySets IsPrimaryKey = 1 AND IsSoftPrimaryKey = 1 on each configured key column. IsPrimaryKey is the source of truth, IsSoftPrimaryKey protects it from schema sync.
the number of UPDATEs written
ProtectedapplyThe Sequence expression for every EntityField INSERT CodeGen emits:
(SELECT COALESCE(MAX(Sequence), 0) + 1 FROM EntityField WHERE EntityID = '<id>').
Evaluated at APPLY time, never a literal. The value is disposable — the run's own spUpdateExistingEntityFieldsFromSchema pass and R__RefreshMetadata renumber every field from the schema — but it must be unique on ANY database in ANY order, because the INSERT is appended verbatim to a migration and Flyway runs every versioned migration before the repeatable renumber. Each INSERT re-evaluates MAX after the one before it, so a batch that executes sequentially in emission order rises in that order (#3670, #4202).
ProtectedapplyApplies soft value-list configuration from Fields[] entries in additionalSchemaInfo.json. Runs AFTER manageEntityFieldValuesAndValidatorFunctions() so CHECK-constraint-derived values are already in the database. If EntityFieldValue rows already exist for a field (from CHECK constraints), the soft-enum pass skips that field — CHECK is authoritative.
ProtectedapplyApplies category assignments from VE decoration results using the shared category methods. Loads field records from DB (needs ID, Name, Category, AutoUpdateCategory, AutoUpdateDisplayName) then delegates to the shared methods.
ProtectedapplyApplies the PrimaryKey and ForeignKeys of VirtualEntities entries. Runs after the view-column sync, so it sees the fields of entities and view columns created in the same run. On a composite key it also clears IsUnique, which spCreateVirtualEntity set on the first column.
success, and the number of UPDATEs written
ProtectedassessRLS-safety assessment for a QUERY materialization, shared by the mint-time gate and the per-run drift
re-check (plan §6.2 / §10). A materialized query entity does NOT inherit its source entities' row-level
security, so it is only safe to serve an unscoped snapshot when we can PROVE no source is RLS-protected.
Fails closed (returns { safe:false }) on three conditions, in order of decreasing severity of what we
cannot prove:
EntityInfo: if it secretly
carried a read RLS filter we'd never see it.ReadRLSFilterID, and the minted entity's NEW EntityID would not match the key's EntityID binding).
Over-restriction here is harmless (the query stays live-only, or an existing materialization is held); the
reverse — serving a protected source's rows unscoped — is the leak this guard exists to prevent.Optionalsql: stringProtectedboolReturns a boolean literal for the platform. SQL Server: 1/0, PostgreSQL: true/false
ProtectedbuildHandles relationship management for an entire entity pair at once. Matches N FK fields to N existing relationship records using a 1:1 mapping:
ProtectedbuildINSERT for one EntityPermission row, skipped when a row already exists for that
(EntityID, RoleID, Type).
The guard is not defensive tidiness — EntityPermission carries a UNIQUE constraint on those
three columns (UQ_EntityPermission_EntityID_RoleID_Type), so an unguarded INSERT is a failed
CodeGen run rather than a duplicate row. All three call sites are "grant the configured default
permissions", which is naturally re-entrant: an entity re-detected as new, or a second CodeGen
pass over the same entity, reaches them again. Before the constraint existed this silently
accumulated duplicates — a live database showed one (entity, role) pair with rows created three
years apart, and pairs whose verb flags disagreed.
Type is written explicitly rather than left to the column default so the row being inserted
and the row being tested for are keyed identically; a default that changed later would
otherwise put them out of step. Expressed as INSERT ... SELECT ... WHERE NOT EXISTS, which is
valid on both SQL Server and PostgreSQL, so no provider branch is needed.
ProtectedbuildBuilds a set of existing category names from entity fields. Used to enforce category stability (prevent renaming).
ProtectedbuildBuilds the DELETE for EntityFields no longer present in the freshly-introspected remote object, or '' when nothing should be removed. Field-name matching is case-insensitive. Pure/testable (no DB access) — the data-loss-sensitive computation guarded by externalObjectIsSyncable.
ProtectedbuildBuilds SQL to INSERT a new EntityRelationship record for a discovered FK field.
ProtectedbuildBuilds the entity lookup behind processISARelationshipConfig: match on Name first, else on BaseTable, optionally constrained to a schema.
The schema predicate is composed CONDITIONALLY rather than written as
(@SchemaName IS NULL OR SchemaName = @SchemaName). That form is fatal on PostgreSQL: the
parameter's only unambiguous use is $n IS NULL, which gives the planner no type to infer, so
the whole statement fails to prepare with "could not determine data type of parameter $n".
SQL Server infers the type from the other side of the OR and never saw the problem.
The failure was silent in the worst way — processISARelationshipConfig catches per-relationship errors and logs them, so CodeGen ran to completion with a zero exit code while every declared IS-A relationship on PostgreSQL was quietly skipped, leaving Entity.ParentID NULL. Downstream that means no mirrored parent fields, no parent JOIN in the child base view, and a child whose Save() never writes the parent row.
Emitting the predicate only when a schema was supplied keeps the parameter list free of type-ambiguous entries and is portable to both dialects without a cast.
OptionalschemaName: stringProtectedbuildBuilds the provider-neutral query behind validateISARelationships: one row per DECLARED IS-A child (Entity.ParentID IS NOT NULL) carrying everything the severity rules need. LEFT JOINs throughout so an unresolvable ParentID still returns a row (that is a hard error, not a missing row).
ProtectedbuildBuilds IS-A parent chain context for an entity, computing which parent each inherited field originates from. Used to provide the LLM with inheritance awareness during form layout generation.
Returns an empty object for entities without parents, so it can be safely spread into the entity object passed to generateFormLayout().
ProtectedbuildEmits the UPDATE that brings an existing relationship row's EntityKeyField into line with the
schema, or '' when it already agrees.
This is the half that actually repairs a live database. A relationship whose
RelatedEntityJoinField is already correct takes the "exact match" path and is skipped, so
rows created before EntityKeyField was written stay blank forever and keep returning zero
rows. Honours AutoUpdateFromSchema exactly as the join-field reassignment below does: a row
an operator has taken ownership of is never touched.
ProtectedbuildBuild the two deterministic search-flag hygiene statements.
These exist because the LLM-driven smart-field pipeline cannot be relied on to run at all:
AdvancedGeneration.enabled reads enableAdvancedGeneration ?? false, so on a default
configuration none of applySearchableFieldUpdates / applyEntitySearchConfig ever
executes — while every new entity is still INSERTed with AllowUserSearchAPI = 1
(see createNewEntityInsertSQL). The result is an entity flagged searchable with nothing
searchable on it, which is not a harmless default: UserSearchString against such an entity
is a documented no-op (MJ#4581/#4582), so the data provider ignores the term and returns the
UNFILTERED table. Global search then fans out to it on every keystroke and discards every row.
Both statements are deterministic, need no model, and are safe to run on every pass:
Seed — an entity with NO searchable field gets its name-like columns flagged
(NAME_LIKE_FIELD_NAMES: Name, Title, FirstName, LastName, …). Those are what a person
types into a search box, so they are the one defensible default. Only fires when the
entity has nothing flagged at all, so it fills a gap rather than overriding anyone —
including the LLM, which has already run by this point when it is enabled.
Note this keys on the field's NAME, not on IsNameField. That flag looks like the
obvious source and is the wrong one: measured against a real database, seeding from it
would flag 54 fields of which 41 are VIRTUAL — the denormalized FK display columns
CodeGen puts on views (Action, Agent, Artifact). Those are computed by JOIN, so a
LIKE against them cannot seek any index, and flagging them would push 49 junction
entities into the global-search fan-out with unindexable predicates — the exact cost
search-guardrails.ts exists to prevent. It also picked up identifiers (RecordID,
Token, ExternalSystemRecordID) and a Description, which isNarrativeFieldName
rejects on the LLM path. And it would still not have fixed MJ: Employees, whose only
IsNameField is the virtual FirstLast; keying on the name reaches its real
FirstName / LastName columns, which is what the reported bug needed.
Clear — an entity STILL left with no searchable field has AllowUserSearchAPI turned
off, so it drops out of the search fan-out instead of contributing noise.
Both honor the AutoUpdate* opt-outs, which is how an operator pins a hand-made decision —
and is exactly how the curated entries in metadata/entities/.entity-search-exclusions.json
protect themselves (they set AutoUpdateAllowUserSearchAPI to false alongside the flag).
Full-text-search entities are exempt from both. An FTS entity is searchable through its
INDEX: createViewUserSearchSQL takes the full-text branch before it ever reads
IncludeInUserSearchAPI, so those flags are dead metadata there and the entity is a
perfectly valid search target without them.
Written as UPDATE ... WHERE <key> IN (subquery) rather than UPDATE ... FROM ... JOIN,
which is T-SQL-only. The subquery form is ANSI and runs unchanged on both platforms.
ProtectedbuildParses a view definition SQL and resolves referenced tables to MJ entities. Returns enriched source entity context (all fields with descriptions and categories) for the LLM to use when decorating virtual entity fields.
ProtectedbuildINSERT for a config-declared virtual entity's Entity row. Logged rather than a stored-procedure call so the CodeGen_Run capture replays it with the same ID on every database, exactly like table-backed entities. Column set and flags match spCreateVirtualEntity, plus Description.
ProtectedbuildSeeds the first configured key column before the view-column sync runs, so the sync sees a key and does not promote the first view column. Type is a placeholder the sync replaces.
ProtectedcheckThis method will look for situations where entity metadata exist in the entities metadata table but the underlying table has been deleted. In this case, the metadata for the entity should be removed. This method is called as part of the manageMetadata method and is not intended to be called directly.
ProtectedcheckProtectedcheckINTEGRITY CHECK — in a well-formed entity every base (non-virtual) field sequences BEFORE the
virtual/related fields, so the EntityField order matches the base view's SELECT [base].*, <joins>
column output. The positional save-capture in the data providers (e.g. SQLServerDataProvider's
@ResultTable) relies on that alignment. CodeGen inserts newly-discovered columns at an apply-time
MAX(Sequence) + ordinal placeholder that updateExistingEntityFieldsFromSchema is supposed to renumber; if
a base column is left sequenced AFTER a virtual field, the save-capture would mis-route values by
position. The providers now compensate (saves stay correct), but the metadata is still wrong — so we
scan for it after every metadata pass and log a prominent warning to drive the root cause out over time.
ProtectedcheckChecks a soft FK's target column against the related entity's fields. The name matches
case-sensitively on PostgreSQL, where ID and id are different identifiers, and
case-insensitively on SQL Server. A column of a multi-column primary key is not unique on its
own, so a join on it alone can return several rows. That is decided before IsUnique is read:
on a live SQL Server run, both columns of a two-column primary key carried IsUnique = 1.
ProtectedclassifyPhase 2d — classifies a parameterized query's parameters (deterministic render-and-diff) and, when every parameter is a safe row filter, produces the BROAD source SQL the refresh engine materializes (the query with its row-filter WHERE predicates removed). Returns the persistence fields, or a precise refusal (per-parameter reasons) so non-qualifying queries are skipped loudly.
Public entry point for cleaning up stale EntityRelationship records. This must be called AFTER deleteUnneededEntityFields() has run, so that stale EntityField records (for dropped columns) are removed before we check which relationships are still valid. Called from sql_codegen.ts after the second manageEntityFields() pass.
database connection
schemas to exclude from FK field lookup
ProtectedcoalesceReturns ISNULL/COALESCE expression. Both platforms support COALESCE, but SQL Server also has ISNULL.
ProtectedcolumnProtectedconditionalGenerates SQL for conditional INSERT (IF NOT EXISTS pattern). Delegates to the database provider.
ProtectedconfiguredThe configured base-view exclusions, as bare field names for the prune's protected list.
ProtectedconfirmConfirms a target that only IsUnique vouches for against the database's unique indexes.
Schema sync sets IsUnique on every column of a composite unique constraint, so the flag
alone cannot say whether a join on the column returns one row. A failed or unsupported
lookup keeps the flag's answer.
ProtectedcreateCreates the default constraint for a special date field. This method is called as part of the ensureSpecialDateFieldExistsAndHasCorrectDefaultValue method and is not intended to be called directly.
ProtectedcreateProtectedcreateThe model-backed translator for SQL @CHECK rules; a seam so tests can supply a stub.
ProtectedcreateCreates a new application using direct SQL INSERT to ensure it's captured in SQL logging. The Path field is auto-generated from Name using the same slug logic as MJApplicationEntityServer.
SQL connection pool
Pre-generated UUID for the application
Name of the application
Schema name for SchemaAutoAddNewEntities
Current user for entity operations (unused but kept for signature compatibility)
The application ID if successful, null otherwise
ProtectedcreateProtectedcreateProtectedcreateProtectedcreateOptionalentityIDs: string[]OptionalexcludeSchemas: string[]ProtectedcreateProtectedcreateProtectedcreateProtectedcreateWrites the logged Entity INSERT and key seed for one config-declared virtual entity, registers it in NewEntityList, adds it to its schema's application and grants the default permissions.
true when the entity rows were written, even if the application or permission step failed
ProtecteddecorateApplies LLM-assisted field decoration to a single virtual entity. Parses the view SQL to identify source entities, enriches the LLM prompt with their field metadata (descriptions, categories), then applies PKs, FKs, descriptions, and categories.
Whether the entity was decorated, skipped, or encountered an error.
ProtecteddecorateIterates over all virtual entities and applies LLM-assisted field decoration to identify primary keys, foreign keys, and field descriptions. Only runs if the VirtualEntityFieldDecoration advanced generation feature is enabled. Idempotent: skips entities that already have soft PK/FK annotations.
ProtecteddeleteOptionalentityIDs: string[]ProtectedderiveDerives an entity name from a view name by removing common prefixes (vw, v_) and converting to a human-friendly format.
ProtecteddetectDetect if an entity has geo-capable fields based on persisted ExtendedType values in the database. If any field has a Geo* ExtendedType, set Entity.SupportsGeoCoding = 1. Respects the AutoUpdateSupportsGeoCoding flag — if 0, the value is locked.
ProtecteddetectPhase 4 (§13/§17.2): scan active materializations and flag those whose source shape has drifted
as DriftHold (stop refreshing, surface for review). Runs after the entity/field re-sync, so it
compares each materialization's provenance against the CURRENT metadata. Flag-and-hold only — never
auto-rebuilds. Already-held (DriftHold) and Disabled rows are skipped.
ProtecteddropDrops and recreates the default constraint for a special date field. This method is called as part of the ensureSpecialDateFieldExistsAndHasCorrectDefaultValue method and is not intended to be called directly.
ProtecteddropDrops an existing default constraint from a given column within a given entity, if it exists
ProtecteddropGenerates a conditional existence check + DROP statement. Delegates to the database provider.
ProtectedensureThis method ensures that the __mj_CreatedAt and __mj_UpdatedAt fields exist in each entity that has TrackRecordChanges set to true. If the fields do not exist, they are created. If the fields exist but have incorrect default values, the default values are updated. The default value that is to be used for these special fields is GETUTCDATE() which is the UTC date and time. This method is called as part of the manageEntityFields method and is not intended to be called directly.
ProtectedensureThis method ensures that the __mj_DeletedAt field exists in each entity that has DeleteType=Soft. If the field does not exist, it is created.
ProtectedensureThis method handles the validation of the existence of the specified special date field and if it does exist it makes sure the default value is set correctly, if it doesn't exist it makes sure that it is created. This method is called as part of the ensureCreatedAtUpdatedAtFieldsExist method and is not intended to be called directly.
ProtectedentityWhether the current database's Entity table has the ExternalDataSourceID column. External
Data Sources ship as a SQL Server migration only, so on any database/schema that predates it
(PostgreSQL today) the column is absent — and ANY raw SQL referencing it throws
"column does not exist" and aborts the entire CodeGen run, even for a pure MJ-DB schema with no
external entities. Callers gate EDS-specific queries on this so CodeGen stays green everywhere
and auto-activates once the column lands on other platforms (mirrors the PG stored-proc guard in
metadataSupportObjects.ts). Cross-platform via INFORMATION_SCHEMA; cached for the run (the schema
cannot change mid-run).
ProtectedentityTHE row-restriction gate for materialization. True when the entity is protected by ANY row-level restriction — role RLS or an API-key / API-application row filter — because a materialized snapshot reproduces neither. Fails closed when the API-key layer could not be enumerated (see apiKeyRowFilterTargets).
ProtectedentityROLE-RLS LAYER ONLY — true if any of the entity's role permissions carries a non-empty ReadRLSFilterID.
This is deliberately NOT a gate on its own, and no materialization gate may call it directly. MJ enforces
row restrictions in TWO layers (see EntityInfo.GetEffectiveRowFilterWhereClause, the sanctioned composer):
role RLS and API-key row filters, AND-composed. An entity bound ONLY by an API-key row filter carries no
ReadRLSFilterID at all, so a gate that stopped at this layer would judge it unprotected and fail OPEN —
minting a materialized entity with a NEW EntityID, which the API-key binding (which binds by EntityID) no
longer matches, handing a filtered principal a full unscoped snapshot of rows it cannot read live.
EntityInfo exposes no user-free accessor for either layer (GetEffectiveRowFilterWhereClause needs a
session UserInfo, and CodeGen has no session), so the composition is done explicitly here — by
entityHasRowLevelRestriction, which is what every gate calls.
ProtectedentityReturns a parenthesized subquery that resolves an entity's ID by its BaseTable + SchemaName. Used in logged SQL to keep INSERT statements portable across databases (avoids hardcoding runtime UUIDs that may differ on a fresh database).
ProtectedevaluateProcesses ONE materialization row: the C1 RLS re-check, the C2 read-grant re-narrow, the external base-view leak guard, and generic shape/provenance drift. Returns true if the row was held. Extracted from detectMaterializationDrift so a throw on one row can be isolated by the caller's try/catch instead of aborting reconciliation for every remaining materialization.
ProtectedexternalWhether an introspected remote object has columns we can safely sync. When false (object missing or zero columns), manageSingleExternalEntity skips the field sync rather than treating it as "the entity now has no fields" (which would DELETE every EntityField). Pure/testable.
ProtectedextractExtracts the top-level "Entities" array from the additionalSchemaInfo config file. Each entry identifies an entity by BaseTable + SchemaName and declares arbitrary Entity-table attributes to set (e.g., AllowMultipleSubtypes, TrackRecordChanges).
ProtectedextractExtracts value-list field configurations from the additionalSchemaInfo config file. Walks schema → tables → table.Fields[] and returns a map keyed by "schema.table".
ProtectedextractExtracts ISARelationships array from the additionalSchemaInfo config file. The config may contain a top-level "ISARelationships" key with an array of parent-child relationship definitions.
ProtectedextractExtracts the MaterializedBaseViews array from the additionalSchemaInfo config file. Each entry declares a 1:1 base-view materialization of an existing entity (plan §4.1).
ProtectedextractExtracts organic key configurations from the additionalSchemaInfo config file. Walks all schema-keyed table arrays and collects OrganicKeys arrays with their owning schema and table name.
ProtectedextractExtracts a flat array of table configs from the config file, handling both formats:
ProtectedextractExtracts VirtualEntities array from the additionalSchemaInfo config file. The config may contain a top-level "VirtualEntities" key with an array of virtual entity definitions.
ProtectedfindFinds fields declared as a soft FK to more than one target. Each declaration is applied as its
own UPDATE, so without this check the last one in the file would win silently.
Names compare case-insensitively: on SQL Server ID and id are the same column; on
PostgreSQL one of the two does not exist, which the target check reports.
ProtectedfindMaps a physical (schema, table/view) reference to the MJ entities backed by it — by BaseView or BaseTable,
case/whitespace-insensitive. Returns ALL candidates so the P1 guard can fail closed on an AMBIGUOUS ref.
Resolution: an explicit, non-default schema yields only that schema's exact matches; an empty schema OR the
parser-default 'dbo' — which SQLParser.ExtractTableRefs assigns to every UNQUALIFIED ref, while MJ
core/app entities live in __mj/app schemas (never dbo) — is treated as unqualified and yields every
schema-agnostic name match. That way an unqualified reference to a __mj/app-schema entity is still caught,
and a same-base-name collision across schemas surfaces every candidate rather than just the first (so the
guard refuses if any of them is under-linked). Used by the P1 under-linking guard.
ProtectedfindProtectedfindFor an inherited field, walks the parent chain to find which specific parent entity originally defines this field (by matching non-virtual fields on each parent).
Returns the names of entities whose BASE VIEW exposes a view-only column that MJ has no
EntityField for — the virtual-field drift probe.
WHY THIS IS NEEDED AND WHY NOTHING ELSE FINDS IT. Pass 2 of ManageEntityFields is scoped to
NewEntityList ∪ ModifiedEntityList, and the only writer of ModifiedEntityList is the
reconciler pass — it lists entities something CHANGED on during THIS run. A related-entity name
field whose EntityField row was missed once therefore drops out of scope permanently: nothing
changes on that entity again, so it is never in the list, so pass 2 never revisits it, so the
row is never created. The only recovery was forceRegeneration, which reprocesses every entity
in the database to heal one.
The cost of leaving it is not cosmetic. BaseEntity.InnerLoad reads the base view and then
looks up each returned column in metadata; a column with no EntityField throws, so EVERY
record form for that entity fails — measured at 145 entities on one imported schema.
WHAT THE PROBE MATCHES. vwSQLColumnsAndEntityFields is the catalog view both dialects already
use for exactly this comparison (EntityFieldID IS NULL is how getPendingEntityFieldsSQL
finds columns needing a field). Adding IsVirtual <> 0 narrows it to VIEW-ONLY columns: those
are the ones pass 2 exists to create, and they are the ones pass 1 structurally cannot see,
because their names are invented during Step 2's view generation and only exist in the view.
New PHYSICAL columns are excluded — they carry IsVirtual = 0, pass 1 already finds them, and
pulling them in here would widen the scoped pass back out to a full scan.
The observed spelling was a <fk>_Virtual alias, but the suffix is incidental (it is only
appended on a name collision, see sql_codegen.ts safeAlias); the load-bearing property is
"view-only column with no EntityField", so the probe tests that and not the name.
IsVirtual <> 0 rather than = 1 deliberately: the column is an INTEGER in the PostgreSQL view
and a bit-valued expression in the SQL Server one, and <> 0 is correct for both.
ProtectedgatherGathers the drift-relevant existence facts for one materialization against current metadata.
ProtectedgenerateThis method generates descriptions for entities in teh system where there is no existing description. This is an experimental feature and is done using AI. In order for it to be invoked, the EntityDescriptions feature must be enabled in the Advanced Generation configuration.
ProtectedgenerateGenerates a TypeScript field validator function from the text of a SQL CHECK constraint.
the data object containing the entity name, column name, and constraint definition
all of the entity fields in the system
the current user
a flag indicating whether or not to generate new code, this is set to false when we are just loading the generated code from the database.
a data structure with the function text, function name, function description, and a success flag
ProtectedgetProtectedgetReturns EntityNamingOptions derived from the entityNaming config section. These control ALL CAPS normalization and compound word splitting behavior.
ProtectedgetActual column names of a materialized table (via INFORMATION_SCHEMA), for drift comparison.
ProtectedgetResolves entity name prefix/suffix rules for a given schema. The resolution order is:
If both sources define rules for the same schema, the config file wins and a console warning is emitted to alert the user of the override.
ProtectedgetThis method builds a SQL Statement that will insert a row into the EntityField table with information about a new field.
the new field
ProtectedgetCreates a SQL statement to retrieve all of the pending entity fields that need to be created in the metadata. This method looks for fields that exist in the underlying database but are NOT in the metadata.
IMPORTANT: The sequence calculation uses a dynamic offset based on the maximum existing sequence for each entity, plus 100,000, plus the column sequence. This ensures no collision with existing sequences while maintaining deterministic ordering. The spUpdateExistingEntityFieldsFromSchema stored procedure runs AFTER this method and will correct the sequences to ensure they are in the correct sequential order starting from 1. In a migration, the spUpdateExistingEntityFieldsFromSchema runs afterwards as well so this behavior ensures CodeGen works consistently.
OptionalentityIDs: string[]OptionalexcludeSchemas: string[]ProtectedgroupGroups FK fields by their entity pair key (ParentEntityID|ChildEntityID). This ensures all FK fields for the same entity pair are processed together, preventing the ping-pong bug where multi-FK pairs swap JoinFields on each run.
ProtectedhasChecks if a table has a soft primary key defined in the additionalSchemaInfo JSON file (configured in mj.config.cjs)
ProtectediifReturns an IIF/CASE expression. SQL Server: IIF(cond, t, f), PostgreSQL: CASE WHEN cond THEN t ELSE f END
ProtectedisReturns true if the field is a sensible target for IsNameField (i.e. could serve as the entity's human-readable display name). A name field must be BOUNDED TEXT — so we reject primary keys, uniqueidentifiers, all non-text types, and unbounded (MAX) text. This stops Smart Field Identification from flagging a PK/uniqueidentifier as a name field, which would corrupt every related-entity name virtual field that joins to this entity (those resolve their SQL type from the related entity's NameField).
VIRTUAL fields are rejected too — with ONE carve-out. A view-only name column
forces the FK-name join to target the VIEW instead of the base table: the
self-referencing case is unbuildable on PostgreSQL and SQL Server (see the
self-FK skip in the view generator), and a name that is itself a BORROWED
FK-name (_RelatedEntityNameFieldMap) resolves circularly. But an IS-A
(Table-Per-Type) inherited field is neither of those: it is the child's
mirror of a real parent COLUMN, materialized by joining the parent's base
table (generateParentEntityFieldSelects), and it is the only sensible
display value the child has. Rejecting it left IS-A children with ZERO name
fields — selectNameFieldWinner returned null and the clear-loop then wiped
the IsNameField=1 the deterministic pass had correctly set on Name
(MJ issue #3551).
ProtectedisReturns true if the field is a sensible target for LIKE-based user search. Mirrors the runtime guard in GenericDatabaseProvider.isTextSearchableType / the Phase 1 hygiene migration so CodeGen stops re-introducing invalid flags.
ProtectedisIs this field an IS-A (Table-Per-Type) INHERITED field — the child entity's mirror of a column that physically lives on a parent table?
Uses the same discriminator as the rest of this file (see
syncISAParentFields and buildParentChainContext): IsVirtual=1 AND
AllowUpdateAPI=1. That pair is unambiguous — every virtual column
discovered from a base VIEW is inserted with AllowUpdateAPI=0 (the pending-
fields SQL forces IIF(IsVirtual = 1, 0, …)), and the IS-A sync is the only
thing that sets AllowUpdateAPI=1 back on a virtual field. So a borrowed
FK-name column can never satisfy it, which is exactly the separation the
name-field guardrail needs.
ProtectedisHeuristic: does the entity name match the shape of an audit / log / run-history / change-tracking table? These are the entities that drive the most LIKE-scan time and almost never belong in global search.
ProtectedisThe TYPE half of isFieldEligibleForNameField, with no virtuality opinion: a primary key, a non-text type, or unbounded (MAX) text can never be a name field, because those are the values that actively CORRUPT the related-entity name virtual fields that resolve their SQL type from this entity's NameField.
Split out so the two halves can be applied separately: virtuality only makes
a field a worse choice than a base-table column, while failing this check
makes it a wrong one. selectNameFieldWinner clears the wrong ones even
with no replacement, but preserves a merely-worse one (see its step 4).
ProtectedisProtectedloadLoads apiKeyRowFilterTargets once per CodeGen run. Called at the top of every materialization entry point, BEFORE any gate runs, so the gates never evaluate against an unloaded cache in production.
Resolution rules (all biased fail-closed):
RowFilterID column absent on a scope table ⇒ that binding layer cannot exist on this database
(pre-v6 schema / the PostgreSQL parallel world) ⇒ contributes nothing. This is CORRECT, not fail-open.ResourcePattern must name ONE exact entity (enforced at rule save). Mappability is
decided by ResolveSingleEntityResourceTarget, shared verbatim with the runtime refresher's
identical gate; a rule it cannot resolve collapses the whole set to 'unknown' — every entity is then
treated as restricted.'unknown' (never a silently-empty set).
Permission type is deliberately NOT narrowed to Read: mapping a scope rule to a permission type requires
the APIScope path taxonomy that lives outside CodeGen, and over-restriction here is harmless.Use LoadGeneratedCode.
This method will load all generated code from the database - this is intended to be used when you are bypassing managing the metadata.
ProtectedloadLoads existing SchemaInfo records from the database into the cache so that entity name prefix/suffix rules are available before createNewEntities() runs. This is a read-only SELECT — it does NOT create or update any records.
OptionalentityFilter: string[]Use ManageEntityFields.
Manages the creation, updating and deletion of entity field records in the metadata based on the database schema.
OptionalentityFilter: string[]Optional list of entity NAMES to scope the field-management work to. When provided,
the three SP/inline-SQL passes (delete unneeded, create new from schema, update existing from schema)
filter to those entities only. Other steps remain unscoped — they are cheap enough that scoping them
adds complexity without measurable benefit. An empty array short-circuits the entire method as a no-op,
which is the typical "no schema changes since last run" case in Pass 2.
undefined (default) preserves prior full-scan behavior.
ProtectedmanageProtectedmanageThis method creates and updates relationships in the metadata based on foreign key relationships in the database.
specify any schemas to exclude here and any relationships to/from the specified schemas will be ignored
ProtectedmanageProtectedmanageBaseline foreign-key consumption for an external entity: for each introspected single-column
relationship whose referenced remote object is ALSO an imported external entity in the same
data source, set the FK field's RelatedEntityID + RelatedEntityFieldName + IsSoftForeignKey=1.
The standard manageEntityRelationships pass then materializes these into EntityRelationship
records. Composite FKs and references to objects that aren't imported are skipped (logged) —
those are the follow-up hardening. Returns true if any FK field was updated.
ProtectedmanageLoads (always) and generates (only when generateNewCode and the ParseCheckConstraints
feature allow) the TypeScript translations of SQL @CHECK rules written on opted-in JSONTypes,
then publishes them on GeneratedJSONValidators for the entity emitter.
Failures are logged and never fail the run: a missing translation only means that one rule is not emitted, which the emitter reports.
ProtectedmanageManages M->M relationships between entities in the metadata based on foreign key relationships in the database. NOT IMPLEMENTED IN CURRENT VERSION IN BASE CLASS. M->M relationships ARE supported fully, but they are not AUTO generated by this method, instead an administrator must manually create these relationships in the metadata.
Use ManageMetadata.
Primary function to manage metadata within the CodeGen system. This function will call a series of sub-functions to manage the metadata.
the ConnectionPool object to use for querying and updating the database
ProtectedmanageManages 1->M relationships between entities in the metadata based on foreign key relationships in the database.
specify any schemas to exclude here and any relationships to/from the specified schemas will be ignored
ProtectedmanageManages virtual EntityField records for IS-A parent entity fields. For each entity with ParentID set (IS-A child), creates/updates virtual field records that mirror the parent entity's base table fields (excluding PKs, timestamps, and virtual fields). Runs collision detection to prevent child table columns from shadowing parent fields.
ProtectedmanageCreates/updates virtual EntityField records for a single child entity's parent fields. Detects field name collisions between child's own base table columns and parent fields.
ProtectedmanageIntrospects the remote schema for a single external entity and syncs its EntityField rows.
Mirrors manageSingleVirtualEntity but sources the field list from the driver's
IntrospectSchema (mapped to MJ types via mapExternalNativeTypeToMJ) instead of the
local view's columns, and reuses manageSingleVirtualEntityField for create/update.
ProtectedmanageProtectedmanageProtectedmanageProtectedmarkupApplies entity name prefix/suffix rules to a given entity name. Rules are resolved from mj.config.cjs first, then from SchemaInfo database metadata as a fallback.
the database schema name
the base entity name to apply prefix/suffix to
ProtectedmaterializedTHE single rule for "which of an entity's fields get materialized" in a base-view materialization. Every site that computes a base-view column set — the MINT (physical table + wrapper view), the DRIFT comparison, and the runtime REFRESH mirror — must use this one predicate, or they judge each other's output as drift.
Rule: for an EXTERNAL entity, virtual fields are MJ-computed and absent from the remote source, so the
refresh mirror only materializes non-virtual columns and the mint must match. A LOCAL base view computes its
virtual columns in the view itself, so SELECT * FROM <baseView> includes them and every field is kept.
(MaterializationRefresher.rebuildFromExternalEntity applies !f.IsVirtual on the external path only — the
same rule, expressed in a context where "external" is already established.)
ProtectedmaterializedProtectednewProtectednormalizeApply the code-level search guardrails to the LLM result, mutating
result in place so every downstream applier reads the normalized
version. The pure heuristics live in search-guardrails.ts; this
method wires them to the per-field metadata (Type, Length, IsPrimaryKey,
AutoUpdate flags) we have on fields.
Order of operations:
isFieldEligibleForUserSearch
and the narrative-field-name blocklist.Contains to a default when the
field isn't FTS-backed, fill in defaults for missing entries.ProtectedparentChecks if an existing virtual parent field record needs to be updated to match the parent field.
ProtectedparseParses a column-level CHECK constraint into the list of values it permits, or returns null when the
constraint is not a simple value list (a range, a LEN() predicate, a mixed condition, and so on).
Both of the ways IN (...) is rendered by SQL Server are handled. String, date and GUID literals come
back quoted — ([Status]='Active' OR [Status]='Inactive') — but numeric and bit literals come back
unquoted and parenthesized: ([Level]=(3) OR [Level]=(2) OR [Level]=(1)). Matching only the quoted
form left a numeric IN-list with no value list at all, and so no dropdown in Explorer either (#3978).
PostgreSQL's renderings are handled separately, by parsePostgreSQLConstraint.
A single-value list is captured too: CHECK (Status='Active') — which is also how SQL Server renders
IN ('Active') — permits exactly one value, which is a one-item list.
This is a parser, not a policy: it reports what the constraint permits. WHICH fields get a value list out of that is decided after the fact, by valueListForField.
ProtectedparseThis method takes the stored DEFAULT CONSTRAINT value from the database and parses it to retrieve the actual default value. This is necessary because the default value is sometimes wrapped in parentheses and sometimes wrapped in single quotes. This method removes the wrapping characters and returns the actual default value. Some common raw values that exist in SQL Server include 'getdate()', '(getdate())', 'N''SomeValue''', etc. and this method will remove those wrapping characters to get the actual underlying default value. NOTE: For future versions of MemberJunction where multiple back-end providers could be used, this method will be moved to the Provider architecture so that database-specific versions can be implemented, along with many other aspects of this current codebase.
ProtectedpreferDeterministic tie-break across several name-field candidates: the field
literally called Name wins, else the first in field order (fields arrive
ordered by Sequence from the metadata query). Mirrors EntityInfo.NameField's
runtime preference so CodeGen and runtime never disagree about which one it is.
ProtectedprimaryWhether a field's PK/Unique flags need to change. In RECONCILE mode (external entities, whose introspected PK set is authoritative across ALL fields) we sync in BOTH directions — setting the flags when a column becomes a PK AND clearing them when it stops being one (H5: otherwise a stale PK column keeps IsPrimaryKey=1 when the remote PK moves, yielding duplicate PKs). In NON-reconcile mode (virtual entities, where makePrimaryKey is a one-time first-column bootstrap, not authoritative) we only SET on acquisition and never clear — avoiding wiping a legitimately-configured PK, and avoiding a spurious UPDATE (+ __mj_UpdatedAt bump) every run when nothing changed. Pure function; unit-tested.
ProtectedprocessProcesses base-view materialization declarations from additionalSchemaInfo (plan §4.1). Each declares a 1:1 snapshot of an existing entity's base view. Because the shape is identical to the source entity, NO new entity is minted — the existing entity is reused (its RLS applies unchanged, §6.1). For each declaration this:
ProtectedprocessProcess entities in batches with parallel execution. Batch size is configurable via advancedGeneration.batchSize in mj.config.cjs (default: 5).
ProtectedprocessProcess advanced generation for a single entity
Database connection pool
Entity to process
Fields grouped by normalized EntityID (built once by the batch driver)
AdvancedGeneration instance
User context
ProtectedprocessProcesses Entity attribute configurations from the additionalSchemaInfo config. For each entry in the top-level "Entities" array, looks up the entity by BaseTable + SchemaName and applies any declared attribute updates to the Entity table. Reserved keys (BaseTable, SchemaName, AutoAddPermissions) are excluded from the UPDATE statement. Must run AFTER entities are created.
ProtectedprocessProcesses IS-A relationship configurations from the additionalSchemaInfo config. For each configured relationship, looks up both entities by name (or by table name within the given schema) and sets Entity.ParentID on the child entity. Must run AFTER entities are created but BEFORE manageParentEntityFields().
ProtectedprocessProcesses organic key configurations from additionalSchemaInfo. For each configured organic key:
All SQL is executed AND logged via LogSQLAndExecute for complete CI/CD traceability. Must run AFTER entities are created.
ProtectedprocessProcesses query materialization (CodeGen materialization phase, sub-step C — plan §4.2).
Scans queries flagged IsMaterialized = 1 and, for each that qualifies (unparameterized,
has declared output fields — see analyzeQueryForMaterialization, §9/§10):
ProtectedprocessCreates the virtual entities declared in additionalSchemaInfo that do not exist yet. Creation is two logged statements (Entity row with a CodeGen-generated ID, then the first key column) so the CodeGen_Run capture replays on any database. Each new entity joins its schema's application, gets the default permissions, and is registered in NewEntityList so pass 2 and file generation include it in this same run. Must run BEFORE manageVirtualEntities(). An entry whose EntityName is already in use is skipped before anything is written to the capture.
ProtectedpruneProtectedqiQuotes a database identifier (column, table, etc.). SQL Server: [name], PostgreSQL: "name"
ProtectedqsProduces a schema-qualified object reference. SQL Server: [schema].[object], PostgreSQL: "schema"."object"
ProtectedqsqlQuotes mixed-case identifiers in a SQL string for the current platform. Delegates to the database provider's quoteSQLForExecution method.
ProtectedqueryWhether the current database's Query table has the ExternalDataSourceID column. Same rationale as
entityHasExternalDataSourceColumn: External Data Sources ship as a SQL-Server-only migration, so
on a DB without it (PostgreSQL today) the column is absent and any raw SQL referencing it aborts the
CodeGen run. processQueryMaterializations gates its ExternalDataSourceID reference on this. Cached per run.
ProtectedqueryTrue if the Query table has the IsMaterialized column (added by the materialization Foundation migration). processQueryMaterializations gates on this so codegen doesn't throw on a DB where that migration hasn't run yet (e.g. the PostgreSQL parallel world's object-availability lag) — same defensive pattern as queryHasExternalDataSourceColumn. Cached per run.
ProtectedreconcileLeak-2 (C2 ongoing): re-narrows a minted materialized QUERY entity's read grants to the CURRENT source-read
intersection. The mint-time grant (addMaterializedQueryEntityPermissions) is computed once; if a role
later LOSES plain CanRead on a source, its grant on the snapshot must be revoked too — otherwise it keeps
reading snapshot rows it can no longer read live. Called from the drift pass on every codegen run. Only ever
REVOKES (the safe direction); re-granting a role that regained access is a human decision after review.
Returns the number of grants revoked.
ProtectedremoveRemoves stale One-To-Many EntityRelationship records whose RelatedEntityJoinField no longer corresponds to a valid FK field in the database. Only removes relationships where AutoUpdateFromSchema = true and Type = 'One To Many'.
ProtectedresolveResolves the AllowCaching default for a new entity in the given schema.
AllowCachingBySchema entries override the global AllowCaching default. The
${mj_core_schema} placeholder is expanded so core-schema rules apply
regardless of how the core schema is named in this deployment.
ProtectedresolveDecides whether base-view column exclusions may be applied in this run, and records the answer where the (synchronous, connectionless) base-view generators can read it.
Called during metadata management, which runs BEFORE SQL generation, so the flag is settled by
the time any base view is emitted. A run that never calls this leaves exclusions disabled and
emits alias.* exactly as before.
ProtectedresolveWhether CodeGen should grant the default role permissions (newEntityDefaults.PermissionDefaults)
when it creates the entity for this table. A per-table Entities entry may set
"AutoAddPermissions": false for a table that only server code may touch; the entity is then
created with no role permissions at all. Anything other than the boolean false keeps the default.
ProtectedresolveReturns the creation-time attribute overrides declared for one table in the additionalSchemaInfo
Entities array (matched case-insensitively on SchemaName + BaseTable).
Applying these at creation — not only in the later processEntityConfigs pass — matters
for flags CodeGen acts on in the same run. The clearest case is TrackRecordChanges:
ensureCreatedAtUpdatedAtFieldsExist runs before processEntityConfigs, so a table declared
TrackRecordChanges: false used to receive __mj_CreatedAt / __mj_UpdatedAt columns on its
first CodeGen run anyway.
Only the attributes in EntityInfo.ConfigurableAtCreationAttributes are returned, and only with a value of the right type
(boolean, or number for UserViewMaxRows); anything else is left to processEntityConfigs.
ProtectedresolveResolves a list of entity names to their UUIDs via the live Metadata cache. Names that don't resolve are dropped silently — they may be entities that have been queued for creation but aren't yet in the metadata.
Two safety properties:
new Metadata() is constructed OUTSIDE the per-name try/catch so a
metadata-load failure (e.g., refresh failed upstream) surfaces as an error
instead of being silently treated as "no entities resolved" — which would
misleadingly degrade Pass 2 to an unscoped full scan.EntityByName because that method throws
on null/empty names (_newEntityList and _modifiedEntityList can pick those
up from result-set rows whose EntityName column is null). The pre-filter on
string type + trim covers the common case; the catch is a safety net for any
other lookup failure (e.g., name shape we didn't anticipate).ProtectedresolveDesired PK/Unique flags for a virtual/external entity field. IsPrimaryKey mirrors makePrimaryKey; IsUnique is true ONLY for a SINGLE-column PK — a column of a COMPOSITE key is not unique on its own, so composite-key columns get IsUnique=false (M4). Pure function so the rule is unit-tested.
ProtectedresolveResolves the EntityRelationship.EntityKeyField value for a discovered FK field, or null
when the relationship joins the parent's primary key and the column therefore must stay NULL.
WHY THIS EXISTS. A foreign key does not have to point at a primary key. EntityField records
the referenced column in RelatedEntityFieldName — populated for declared FKs by the provider
introspection query and for soft FKs by applySoftPKFKConfig — but nothing carried it across
into the relationship row. EntityRelationshipInfo.EntityKeyField is the only place a consumer
can learn it, and all three consumers fall back to the parent's first primary key when it is
blank (entityInfo.ts BuildRelationshipViewParams, entity-helpers.ts,
DependencyGraphWalker.ts). So a relationship over a non-PK join column silently queried the
wrong column and returned zero rows for every parent record — measured on an imported schema
whose real keys are external ids: 60 relationships, 0 rows each.
NULL IS THE CORRECT VALUE FOR AN ORDINARY FK, not merely an acceptable one. The PK fallback is the documented contract, a stored 'ID' would duplicate it, and emitting the column only when it differs keeps the generated SQL byte-identical for every FK-to-PK relationship — which is all of them in core MJ.
Returns null when: no referenced column was recorded; the referenced column IS the parent's primary key; or the parent has a composite key (a single-column join cannot address it, and guessing which component to use would be worse than the existing fallback).
Pick an entity name that is actually free, disambiguating with the schema name and, if that is still taken, a counter.
Entity names are generated from the table name with trailing discriminators stripped, so distinct tables routinely generate the SAME name — NetSuite's customlist72, customlist74, customlist160, customlist436, customlist534 and customlist873 all generate "Custom Lists".
The previous logic appended the schema suffix ONCE and assumed the result was unique. It is not: the first table took "Custom Lists", and every table after it appended the same "__netsuite" and produced the identical "Custom Lists__netsuite". The second was a duplicate-key failure on UQ_Entity_Name and the rest were never created — a silent loss of 8 entities on one NetSuite tenant, reported only as repeated identical INSERT errors.
Comparison is case-INSENSITIVE on both sides, matching UQ_Entity_Name's collation. The old
in-run check used an exact === while the metadata check beside it lowercased, so two
names differing only in case read as free and then collided on INSERT.
the generated name to place
schema, used for the first disambiguation step
every name already spoken for — existing metadata AND names claimed earlier in this run (which are not in metadata yet)
the free name and the suffix used to reach it ('' when the name was already free)
ProtectedresolveName for a new config-declared virtual entity, or null when its EntityName is already in use. A name derived from the view gets the same collision suffix as a table-backed entity. Names are compared without case, as UQ_Entity_Name does, and include the names created earlier in this run.
ProtectedresolveSchema a VirtualEntities entry resolves to when it names none. Same default as table entries.
ProtectedrevokeRevokes read access on a minted materialized entity by setting CanRead=0 on all its EntityPermission
rows (the __mj_UpdatedAt trigger stamps the timestamp). Used by the drift re-check when a query source
gains RLS after minting: the snapshot can no longer be safely served, so it is made unreadable until a
human authors protection. Non-destructive (rows are kept, just flipped) so the grant can be restored.
ProtectedrevokeThe established fail-closed "stop serving this materialization" sequence, shared by every branch that must take a query materialization out of service.
Revoke read FIRST, then flag DriftHold. The revoke is the security-critical action (it closes the leak — a snapshot serving unscoped source rows); DriftHold only stops FUTURE refreshes. Revoke-first means that if the second statement fails, the readable window is already closed (fail-safe) — whereas DriftHold-then-revoke would leave read OPEN if the revoke failed. A row with no minted entity has no grant row to flip; it is still held.
ProtectedrunExecutes a SQL query with automatic identifier quoting for the current platform.
ProtectedrunExecutes a parameterized SQL query with automatic identifier quoting for the current platform.
ProtectedsanitizeSanitizes an LLM-supplied codeType against the CK_EntityField_CodeType CHECK constraint. Uses MJEntityFieldSchema.shape.CodeType from @memberjunction/core-entities as the single source of truth — no hardcoded enum duplication. Values that fail Zod validation (e.g. 'Python', 'Markdown', 'javascript' wrong case) coerce to 'Other', preserving null/undefined. Logs any coercion so the underlying prompt drift is visible instead of silently failing the batch UPDATE at the DB.
ProtectedselectPicks the single IsNameField winner for an entity per the rules documented on
applyNameFieldUpdates: stable current winner → deterministic repair
(literal Name, then field order) → first eligible LLM candidate → PRESERVE an
existing type-safe flag rather than clearing it with nothing to put in its place.
Returns null only when every flagged field is actively WRONG (a PK, a non-text
type, unbounded text) and the LLM named no usable replacement — there the entity
genuinely ends with no name field, which beats pointing FK-name virtual columns
at a uniqueidentifier or a blob.
ProtectedselectWraps a SELECT query with a row limit. SQL Server: SELECT TOP N ... , PostgreSQL: SELECT ... LIMIT N
OptionalorderBy: stringProtectedsetThis method updates the DefaultColumnWidth field in the EntityField metadata. The default logic uses a stored procedure called spSetDefaultColumnWidthWhereNeeded which is part of the MJ Core Schema. You can override this method to implement custom logic for setting default column widths. It is NOT recommended to modify the stored procedure in the MJ Core Schema because your changes will be overriden during a future upgrade.
ProtectedshouldProtectedsimpleProtectedsnapshotTakes a snapshot of EntityField rows from vwEntityFields for diffing. Only queries columns in TRACKED_FIELD_COLUMNS (Sequence is excluded).
OptionalentityIDs: string[]ProtectedsyncProtecteduniqueProtecteduniquenessDecides uniqueness from the unique indexes a column belongs to. A single-key-column index makes it unique. Otherwise a composite unique index, or being only an INCLUDE column, means it is not. No rows (a view, or IsUnique set by configuration) keeps the target as it is.
ProtectedupdateThis method is responsible for generating a Display Name for each field where a display name is not already set. The approach in the base class uses a simple algorithm that looks for case changes in the field name and inserts spaces at those points. It also strips the trailing 'ID' from the field name if it exists. Override this method in a sub-class if you would like to implement a different approach for generating display names.
This method handles updating entity field related name field maps which is basically the process of finding the related entity field that is the "name" field for the related entity.
ProtectedupdateProtectedupdateOptionalentityIDs: string[]ProtectedupdateSyncs SchemaInfo records from database schemas, capturing extended properties as descriptions. Creates new SchemaInfo records for schemas that don't exist yet and updates descriptions from schema extended properties for existing records.
SQL connection pool
Array of schema names to exclude from processing
Promise
ProtectedutcReturns the current UTC timestamp expression. SQL Server: GETUTCDATE(), PostgreSQL: NOW() AT TIME ZONE 'UTC'
ProtecteduuidReturns an explicitly-typed UUID literal for the platform. SQL Server: CAST('...' AS uniqueidentifier), PostgreSQL: CAST('...' AS uuid)
Needed wherever a UUID literal appears in the SELECT list of an INSERT ... SELECT. A bare
quoted literal is fine in INSERT ... VALUES, where both platforms coerce it to the target
column's type — but in the projection of a SELECT, PostgreSQL can resolve an untyped literal
to text and then refuse to assign it to a uuid column. CAST(x AS y) is ANSI and settles
it on both, with the type name coming from the dialect rather than a platform branch here.
ProtectedvalidateValidates an LLM-suggested ExtendedType against the allowed values in EntityField. Returns the valid value (case-corrected) or null if invalid.
ProtectedvalidateFORWARD VALIDATION — verifies that every DECLARED IS-A relationship (any Entity with a non-null ParentID) actually satisfies what the IS-A runtime requires. Never mutates metadata.
Channel-agnostic BY DESIGN: it validates the END STATE of Entity.ParentID, so it covers the
additionalSchemaInfo "ISARelationships" config, an @lookup on ParentID in a metadata-sync
file, and any future channel — one check instead of one per declaration mechanism.
Reports ONLY provable-cannot-work defects (hard errors) — never inference. If a declared IS-A merely "looks off" but would still function, it passes silently: flagging it would misfire on correct declarations (see the note in the loop). The hard errors: - Child has a composite PK. The runtime routes ONE shared PK value between child and parent; it has no model for a multi-column subtype key. - Parent has a composite PK. Same reason, from the other side. - Child PK type <> parent PK type. Parent and child SHARE one PK value (Save() writes the child's PK into the parent's PK via ParentEntityFieldNames; loads match the child by the parent's PK value), so the value must be legal as BOTH PKs. Note a physical FK already guarantees matching types — this only ever fires on a soft/declared IS-A, which is exactly the case with no DB constraint to catch it. - ParentID does not resolve to an existing entity. DEFENSE-IN-DEPTH ONLY: the FK_Entity_ParentID constraint (Entity.ParentID -> Entity.ID) makes this state unstorable and vwEntities has no WHERE clause, so it is not reachable in a healthy database. It is kept because the parent JOIN must be a LEFT JOIN regardless, and without this branch an unresolved parent would silently SKIP the remaining checks (ParentPKType would be NULL) rather than fail. Covered by unit test, not live.
Timing: runs AFTER every ParentID-writing pass (config + any previously-synced @lookup) and BEFORE the 2nd-pass manageParentEntityFields() materializes IS-A virtual fields + view JOINs, so a broken declaration fails before it produces generated code.
ProtectedvirtualVirtualEntities entries that declare keys, in the table-entry shape the soft key writer consumes. A virtual entity stores its view name in BaseTable, so the existing SchemaName + BaseTable lookup applies.
ProtectedvirtualTrue when a key field survives the removal of view columns that no longer exist. The cached field list still holds rows the sync just deleted, so they must be excluded.
StaticAddStaticaddStaticAddAdds a list of entity names to the modified entity list if they're not already in there
StaticclearUse ClearFieldTracking.
StaticClearStaticEntityPure form of the role-RLS layer (see entityHasRowLevelSecurity). IO-free so it can be reused
verbatim by the runtime refresher's equivalent gate. Deliberately WIDER than the runtime's own reader
(EntityInfo.GetUserRowLevelSecurityInfo collects a filter only from an Allow row whose CanRead is
set, since #4358): a leftover filter beside a cleared flag still counts as "protected" here, which errs
conservative for a leak gate.
StaticEntityPure, IO-free core of entityHasRowLevelRestriction — the predicate any other layer (e.g. the
runtime MaterializationRefresher) should mirror, supplying the API-key target set however it can obtain
it. 'unknown' for the target set ⇒ true (fail closed).
StaticfieldUse FieldChangeReasons.
StaticFieldStaticgetUse GetSoftPKFKConfig.
StaticGetLoads and caches the soft PK/FK configuration from the additionalSchemaInfo file. Cached per process to avoid repeated I/O within a CodeGen run; the in-process (RSU) path calls invalidateSoftPKFKConfigCache at the start of each run so a rewritten file is picked up.
StaticinvalidateStaticInvalidateDrops the cached soft PK/FK config so the next getSoftPKFKConfig call re-reads the
additionalSchemaInfo file from disk. REQUIRED for the long-lived in-process (RSU) CodeGen path:
the cache is process-static (load-once), which is correct for a one-shot mj codegen CLI run but
STALE in a long-running MJAPI where RSU rewrites additionalSchemaInfo on every ApplyAll. Without
invalidation, a connector's FIRST ApplyAll writes its soft PKs to the file but CodeGen returns the
pre-write cached config → "No primary key found" → the entity is never created → no entity map →
0 rows sync until an MJAPI restart. RunInProcess calls this at the start of every run (which always
follows RSU's WriteAdditionalSchemaInfo step), so each in-process run sees the freshly-written file.
Deterministic + event-driven (no mtime/TOCTOU race). The CLI Run() path does not call this, so its
load-once-per-process behavior is unchanged.
StaticisStaticIsStaticisStaticIsStaticisStaticIsStaticregisterUse RegisterFieldChange.
StaticRegisterStaticregisterStaticRegisterProtected StaticrelationshipsIndexes relationship rows by their (EntityID, RelatedEntityID) pair, so each FK pair finds its
relationships without scanning every relationship in the database. IDs match in any case, and
each pair's rows keep their original order — exactly what a filter over the full list returns.
Protected StaticsortSorts a parsed value list in place, so the sequence a value gets is stable across databases and runs. An all-numeric list is compared as numbers (1, 2, 10); anything else keeps the default lexical order.
Keyed off the VALUES, not the column type — so a string column whose list happens to be all-numeric
(CHECK (Code IN ('1','2','10')) on nvarchar) would re-sequence too. Measured over every column CHECK
constraint in a live 6.x database (MJ core + BizApps accounting/orders): of the 431 lists the previous
parser already captured, ZERO re-sequence under this comparison. If one ever turns up and the churn is
unwanted, gate allNumeric on the field being numeric as well.
Protected StaticvalueThe value list to actually store for a field, or null when a parsed list should NOT become EntityFieldValue rows (and so should not become a dropdown in Explorer either).
Runs AFTER parsing rather than gating it, so each exclusion is no broader than its own reason and neither one can drop a list that was captured before #3978:
bit: IN (0,1) is vacuous — bit already permits exactly 0 and 1 — and = 1 is a validator,
not a two-item dropdown over what renders as a checkbox. Every bit list is rendered unquoted, so
none of them was reachable before this change and nothing regresses.CHECK (ID=1) is the single-row-table guard MJ's own
sequence tables use — a structural invariant, not a set of values a user picks from. A
MULTI-value list on a primary key is left alone: CHECK (Code IN ('US','CA')) on a natural-key
PK is a real domain list, it was captured before this change, and CodeGen also runs over customer
schemas where natural-key primary keys are ordinary.A field whose metadata row was not found keeps whatever parsed, which is the pre-#3978 behaviour.
Per-run cache of the entity NAMES (trimmed + lowercased) that an API-KEY row filter (
APIKeyScope.RowFilterID) or an API-APPLICATION row ceiling (APIApplicationScope.RowFilterID) binds to.'unknown'— the INITIAL value — means "not loaded, or could not be enumerated", and makes entityHasRowLevelRestriction treat EVERY entity as row-restricted. That is the fail-CLOSED direction on purpose: refusing to materialize costs nothing (the entity/query stays live-only), while materializing an entity whose API-key row filter we failed to see is the precise leak these gates exist to prevent. Loaded by loadAPIKeyRowFilterTargets at the top of every materialization entry point.protected(notprivate) so a unit-test seam can declare "this fixture configures no API-key row filters".