AbstractProtected_Result from PreRunQueries hook containing cache status for batch operations
Protected_Result from PreRunQuery hook containing cache status and optional cached result
Protected_Result from PreRunView hook containing cache status and optional cached result
OptionalcachedResult?: RunViewResultOptionalcallerRequestedFields?: string[] | nullThe caller's original Fields list (lowercased), captured before PreRunView widened params.Fields to all entity fields for cache-superset storage. Non-null ONLY when that widening actually happened — PostRunView uses it to project cache-miss DB results back down to the requested shape.
Optionalfingerprint?: stringOptionaltelemetryEventId?: stringProtected_Result from PreRunViews hook containing cache status for batch operations
OptionalcachedResults?: RunViewResult[]OptionalcacheStatusMap?: Map<OptionalcallerFieldsMap?: Map<number, string[]>Per-param-index caller Fields lists (lowercased), captured before PreRunViews widened params.Fields to all entity fields for cache-superset storage. An index is present ONLY when that widening actually happened — PostRunViews uses it to project cache-miss DB results back down to the requested shape.
OptionalfingerprintMap?: Map<number, string>Per-param-index cache fingerprints computed during PreRunViews — carried forward so PostRunViews doesn't recompute the RLS where-clause and fingerprint string for every batch item.
OptionalpreResolvedCache?: Map<string, CachedRunViewResult | null>Pre-resolved local cache entries keyed by client cache fingerprint, populated during prepareSmartCacheCheckParams so executeSmartCacheCheck can reuse the already-parsed results for 'current' items without a second IndexedDB read.
OptionalsmartCacheCheckParams?: RunViewWithCacheCheckParams[]When CacheLocal is enabled, contains the cache check params to send to server
OptionaltelemetryEventId?: stringOptionaluncachedParams?: RunViewParams[]OptionaluseSmartCacheCheck?: booleanWhen CacheLocal is enabled, indicates we should use smart cache check
Protected Static_StaticCoalesceWhen enabled, concurrent RunViews calls arriving within the same microtask (or within CoalesceWindowMs) are merged into a single mega-batch before hitting the network. This dramatically reduces the number of HTTP round-trips during startup when multiple engines independently call RunViews in parallel.
Set to 0 to disable coalescing. Default 10ms — enough to capture all engines that fire in the same tick, without adding perceptible delay.
Static ReadonlyDatasetThe cache category every dataset read and write uses.
Every storage provider isolates by category — Redis keys are {prefix}:{category}:{key},
browser localStorage [mj]:[category]:[key], IndexedDB a dedicated object store, the
in-memory providers a nested map — so a read and a write that disagree about the category can
never meet. When the batched warm read in GetAndCacheDatasetByName named 'DatasetCache'
while every write and every other reader landed in default, the warm-serve path missed on
every transport and silently refetched from the server. Naming the category once is what keeps
all six call sites in agreement.
Static ReadonlyDatasetHow long a cached dataset lives.
Datasets have a category to themselves and it carries no category-wide expiry, so without a
per-write TTL every distinct filter set (GetDatasetCacheKey includes the filters) would
leave behind a blob nothing removes. An explicit TTL bounds that growth, and the store
prefers a per-write value over any category or global setting — which is also why giving
datasets their own category did not change how long one lives. Providers that do not
implement expiry ignore it.
Static ReadonlyDatasetHow long the _date key lives — deliberately SHORTER than the blob it vouches for.
It is written second, so with equal lifetimes it would usually be the one to survive, leaving
a freshness claim with nothing behind it: IsDatasetCached answers true and the blob is gone.
Expiring the claim first makes the pair fail in the safe direction — the cache reads as
absent and is refetched.
StaticDedupHow long (ms) a resolved RunViews result stays available for instant replay. Set to 0 to disable the linger window (in-flight dedup still applies). Default 5 000 ms.
StaticMaxSafety cap on the number of linger entries held simultaneously. The linger window is a latency optimization — under extreme churn (more distinct query keys than this resolving within one window) new resolutions skip lingering instead of accumulating result arrays in memory.
StaticMaxHow many consecutive windows a member-change refresh may wait on MetadataMemberRefreshMustWait before it is dropped. At the default 500ms window this is ten seconds — far longer than any transaction a Save or Delete holds.
StaticMetadataCoalescing window, in milliseconds, for metadata refreshes triggered by writes to the entities that compose this provider's metadata. A single administrative action produces a burst (enabling field security writes one permission row per field/role pair, each raising its own event) — one refresh per burst, not one per row. The window also gives an enclosing entity transaction time to COMMIT before the re-read: every event in the burst re-arms the timer, so the refresh runs no earlier than this long after the LAST write.
Static ReadonlyMetadataResult of SweepMetadataAgainstDatabase.
StaticMinMinimum interval (ms) between metadata refresh checks to prevent redundant network calls when Config()/RefreshIfNeeded() fire in quick succession (e.g., multiple engines during startup). The throttle is armed only by a check whose status request succeeded — a check that threw or got no timestamps proved nothing, so the next caller checks again. A caller arriving while a check is in flight is told no refresh is needed, as the in-flight owner acts on the answer (see CheckToSeeIfRefreshNeeded). Does NOT affect forced Refresh() calls. Default: 30 000 ms.
StaticPeerUpper bound, in milliseconds, of the random delay added to HandlePeerMetadataNotice's debounce. A tool that changes the schema removes the shared snapshot, which reaches every server at once; the spread lets the first server reload and republish, so the others adopt its snapshot instead of all querying the database together.
StaticPeerHow long a peer notice may be deferred by later notices before the check runs anyway. The debounce timer is reset by each arrival, so a fleet publishing steadily — exactly what a bulk import looks like — could postpone the check indefinitely.
Static ReadonlySaveHex characters kept from the sha1 digest in SaveCallVariableHash. 12 hex = 48 bits:
expected sha1-prefix collisions across 120k distinct save calls (a cheese-scale MetadataSync
capture) fall from ~1.7 at 8 hex to ~3e-5, so _n disambiguation is a same-record safety net
rather than something a large capture exercises. SQL Server identifiers allow 128 characters;
@CodeName_ plus 12 hex fits every generated name.
StaticServerMaximum row count for auto-caching on the server side. When
TrustLocalCacheCompletely is true and a RunView result has no
ExtraFilter, no OrderBy, and the result count is at or below this
threshold, the result is automatically stored in LocalCacheManager
even without an explicit CacheLocal flag.
This captures small reference/lookup tables that are repeatedly queried by multiple clients while avoiding caching large ad-hoc result sets. Invalidation is handled by the standard BaseEntity event-driven upsert (safe because unfiltered caches can be updated in-place).
Set to 0 to disable auto-caching. Default 250.
Gets all explorer navigation items including inactive ones.
Array of all ExplorerNavigationItem objects
Returns the currently loaded local metadata from within the instance
Protected AbstractAllowDetermines if a refresh is currently allowed or not. Subclasses should return FALSE if they are performing operations that should prevent refreshes. This helps avoid metadata refreshes during critical operations.
Gets all application metadata in the system.
Array of ApplicationInfo objects representing all applications
Gets all audit log types defined for tracking system activities.
Array of AuditLogTypeInfo objects
Gets the flat collection of authorization-role assignments.
Consumed lazily by AuthorizationInfo.Roles — consumers should
prefer accessing roles through AuthorizationInfo.Roles rather than
filtering this array directly.
Array of AuthorizationRoleInfo join-table objects
Gets all authorization definitions in the system.
Array of AuthorizationInfo objects defining permissions
Gets the configuration data that was provided to the provider.
The provider configuration including schema filters
ProtectedCurrentThe provider's current transaction nesting depth, for subclasses that track one.
Distinct from IsInTransaction, which some providers deliberately leave false so
that RunMaybeSerial keeps fanning out — SQL Server most notably. This accessor exists so
the entity-transaction machinery can still see real nesting on those providers: it feeds
EntityTransactionScope.IsNested and the out-of-order settle detection in
BeginEntityTransaction. Defaults to 0 for providers that do not track depth.
Gets the current user's information including roles and permissions.
UserInfo object for the authenticated user
AbstractDatabaseFor providers that have ProviderType==='Database', this property will return an object that represents the underlying database connection. For providers where ProviderType==='Network' this property will throw an exception. The type of object returned is provider-specific (e.g., SQL connection pool).
Protected AbstractDBRegex pattern matching known database default-value functions (non-UUID) for this provider's platform. SQL Server should match GETDATE, GETUTCDATE, SYSDATETIME, etc. PostgreSQL should match NOW, CURRENT_TIMESTAMP, clock_timestamp, etc. Case-insensitive, should match the full string with optional whitespace and parens.
The SQLDialect instance matching this provider's PlatformKey.
Use this whenever runtime code needs to emit dialect-specific SQL
(boolean literals, identifier quoting, casts, …) — it spares callers
from doing GetDialect(provider.PlatformKey) every time, and keeps
dialect resolution in one place. Resolves lazily and is cached so
repeated access is free.
Example:
const lit = provider.Dialect.BooleanLiteral(true); // '1' on SS, 'TRUE' on PG
Gets all entity metadata in the system.
Array of EntityInfo objects representing all entities
Returns the filesystem provider for the current environment. Default implementation returns null (no filesystem access). Server-side providers should override this to return a NodeFileSystemProvider.
ProtectedHasFast O(1) check for whether any SQL logging sessions are currently active.
Protected AbstractHasTrue when the driver has a begun physical transaction object. Must be truthful: a published-but-unbegun handle is a poison state.
AbstractInstanceThis property is implemented by each sub-class of ProviderBase and is intended to return a unique string that identifies the instance of the provider for the connection it is making. For example: for network connections, the URL including a TCP port would be a good connection string, whereas on database connections the database host url/instance/port would be a good connection string. This is used as part of cache keys to ensure different connections don't share cached data.
ProtectedIsTrue after the ambient physical TX was abandoned and frames are still settling.
Whether this provider currently has an active transaction. Subclasses
that track transaction state should override this. Used by callers
(e.g. runMaybeSerial) to decide whether to fan out concurrent saves
or run them sequentially. Defaults to false for providers that don't
expose this state.
Why the most recent metadata download (GetAllMetadata) failed; null when it succeeded
or none has run. GetAllMetadata logs and returns undefined on failure so a background refresh
keeps the last good graph; a boot that ends with no metadata reads this to report the cause
(e.g. a user with no roles cannot read MJ: User Roles) instead of a generic "nothing loaded".
Gets the latest metadata timestamps from local cache. Used for comparison with remote timestamps.
Array of locally cached metadata timestamps
Gets the latest metadata timestamps from the remote server. Used to determine if local cache is out of date.
Array of metadata timestamp information
Gets all library definitions in the system.
Array of LibraryInfo objects representing code libraries
ProtectedLocalThis property will return the prefix to use for local storage keys. This is useful if you have multiple instances of a provider running in the same environment and you want to keep their local storage keys separate. The default implementation returns an empty string, but subclasses can override this to return a unique string based on the connection or other distinct identifier.
Returns the active local storage provider, lazily creating an InMemoryLocalStorageProvider if none has been set.
This fulfills the abstract LocalStorageProvider requirement from
ProviderBase and is shared by all database providers
(SQL Server, PostgreSQL, and any future platforms).
Protected AbstractMetadataGets the metadata provider instance. Must be implemented by subclasses to provide access to metadata.
The metadata provider instance
ProtectedMetadataHow long a member-entity write waits before this provider's refresh runs. The base value is the short debounce window — right for the server, where the writer is the refresher and the delay only exists to coalesce a burst and let the enclosing transaction commit. Transport providers override this with a much longer, RANDOMIZED window: every browser receives every write broadcast, so the delay is what turns "N clients each re-pull the metadata graph within the same half-second of any member write" into "each client pays at most one staleness check per window, at a moment no other client shares".
ProtectedMetadataA debounced metadata refresh is timer-driven, so it can fire at any point of a caller's unit of work — including the microtask window while the ambient transaction is being committed. Joining that transaction puts the metadata batch on the transaction's single connection alongside the COMMIT, which tedious rejects (EINVALIDSTATE) or drops (ECLOSE) once the handle is torn down, and a transactional query is deliberately never retried. Metadata reads are not part of anyone's unit of work, so wait for the transaction to end before starting one (#4486).
Under scope affinity (#4786) the timer inherits the membership of the code that scheduled it (re-arms included), so the refresh can only ever join the transaction whose scope scheduled it — and only if that transaction is still open when the batch is issued, several round trips after this check ran. It never joins a transaction someone else begins later. That remaining window is harmless to the process — the batch fails, the dataset reports it, and the loaded metadata stays — but the refresh itself is lost until the next member write. The check is instance-wide, so an unrelated transaction also defers the refresh. Running the metadata batch on the pool regardless of the ambient transaction is #4514 (alongside #4454, the commit-side half of the same window).
ProtectedMetadataWhether a member-entity write arriving while the refresh timer is already armed RESTARTS the timer (debounce) or joins the pending window (coalesce/throttle).
The base is a true debounce (true): the server's refresh must run AFTER the last write
of the unit of work, so every event pushes the timer out — a burst costs one refresh, run
once the burst ends. Transport providers return false: with a long window, re-arming
would let steady org-wide write activity postpone the refresh indefinitely (starvation);
joining the armed window guarantees at most one refresh per window regardless of write
rate, which is the whole point of the window.
ProtectedMetadataTrue while RefreshWithinTransaction is running, so metadata-dataset reads join the ambient transaction instead of the pool.
Whether anything could read the metadata snapshot back, and therefore whether saving it is worth the cost.
The snapshot lets a cold process start from a cached copy of the metadata instead of querying
for it, which only works if the store outlives the writer. Where it does not, both halves of
the round trip are pure cost: the save serializes, gzips and base64-encodes the entire
metadata graph, and the load parses it and rebuilds every EntityInfo and EntityFieldInfo
from a copy of objects the heap already holds. On a large tenant that is expensive enough to
exhaust the heap.
A provider that does not declare ILocalStorageProvider.SupportsCrossProcessPersistence is treated as persistent, which fails in the safer direction.
Gets the MemberJunction core schema name (e.g. '__mj'). Subclasses should override if they have a different way to resolve this. Defaults to the value from ConfigData.
Number of tasks waiting for the outermost commit.
ProtectedPlatformReturns the batch separator token for the underlying database platform by delegating to
the SQLDialect instance returned by getDialect().
SQL Server → 'GO', PostgreSQL → '' (no separator needed).
Auto-injected as the default batchSeparator in CreateSqlLogger.
Returns the database platform key for this provider. Override in subclasses. Defaults to 'sqlserver' for backward compatibility. Inherited from ProviderBase; redeclared here for DatabaseProviderBase consumers.
ProtectedPreProtectedPreProtectedPreOptionalcachedResult?: RunViewResultOptionalcallerRequestedFields?: string[] | nullThe caller's original Fields list (lowercased), captured before PreRunView widened params.Fields to all entity fields for cache-superset storage. Non-null ONLY when that widening actually happened — PostRunView uses it to project cache-miss DB results back down to the requested shape.
Optionalfingerprint?: stringOptionaltelemetryEventId?: stringProtectedPreOptionalcachedResults?: RunViewResult[]OptionalcacheStatusMap?: Map<OptionalcallerFieldsMap?: Map<number, string[]>Per-param-index caller Fields lists (lowercased), captured before PreRunViews widened params.Fields to all entity fields for cache-superset storage. An index is present ONLY when that widening actually happened — PostRunViews uses it to project cache-miss DB results back down to the requested shape.
OptionalfingerprintMap?: Map<number, string>Per-param-index cache fingerprints computed during PreRunViews — carried forward so PostRunViews doesn't recompute the RLS where-clause and fingerprint string for every batch item.
OptionalpreResolvedCache?: Map<string, CachedRunViewResult | null>Pre-resolved local cache entries keyed by client cache fingerprint, populated during prepareSmartCacheCheckParams so executeSmartCacheCheck can reuse the already-parsed results for 'current' items without a second IndexedDB read.
OptionalsmartCacheCheckParams?: RunViewWithCacheCheckParams[]When CacheLocal is enabled, contains the cache check params to send to server
OptionaltelemetryEventId?: stringOptionaluncachedParams?: RunViewParams[]OptionaluseSmartCacheCheck?: booleanWhen CacheLocal is enabled, indicates we should use smart cache check
Maximum number of parameters a CRUD stored procedure (spCreate*/spUpdate*/spDelete*)
may declare on this database platform before CodeGen must emit a JSON-arg shape instead.
Infinity — SS supports up to 2,100 parameters per procedure, far beyond
any realistic CRUD sproc.90 — PG's hard FUNC_MAX_ARGS ceiling is 100 (compiled into the server,
not configurable on managed services like RDS/Aurora). 90 leaves headroom for column
adds without flipping sproc shape unexpectedly.Used by the shared useJsonArgShape predicate (CodeGen + provider call-site) to decide
whether a given sproc should emit as typed-args + _Clear companions (today's shape) or
as a single-JSONB-arg sproc with key-presence tri-state semantics. See
plans/json-arg-crud-sprocs.md and GitHub
issue #2552.
AbstractProviderReturns the provider type for the instance. Identifies whether this is a Database or Network provider.
Gets all security roles defined in the system.
Array of RoleInfo objects representing all roles
Gets all row-level security filters defined in the system.
Array of RowLevelSecurityFilterInfo objects for data access control
ProtectedRunMembers of the open transaction share RunViews only with each other (see ProviderBase.RunViewDedupScope).
Use SavepointStack.
Copy of the savepoint stack, outermost first.
Use TransactionDepth.
Public nesting depth of the transaction open on this instance; 0 = none. Read this, not
IsInTransaction (SQL Server leaves that false), when you need to know whether a transaction
is open at all. The depth is instance-wide: on a shared provider TransactionDepth > 0 does
NOT mean the caller's statements run in that transaction — only the async scope that began it
is part of it (#4786). Code that needs to be inside a transaction should simply call
BeginTransaction / BeginEntityTransaction: joining one the caller is already in
is automatic. Deprecated camelCase transactionDepth alias ships for one release.
ProtectedTrustServer-side providers trust the local cache completely because it is kept in perfect sync via BaseEntity save/delete events and cross-server Redis pub/sub. No lightweight DB validation needed on cache hits.
Protected AbstractUUIDRegex pattern matching known database UUID/ID generation functions for this provider's platform. SQL Server should match NEWID, NEWSEQUENTIALID. PostgreSQL should match gen_random_uuid, uuid_generate_v4. Case-insensitive, should match the full string with optional whitespace and parens.
Gets only active explorer navigation items sorted by sequence. Results are cached for performance.
Array of active ExplorerNavigationItem objects
StaticPartialMJ_Metadata item codes that may be reloaded on their own, without rebuilding the rest of the metadata graph: the saved-query family. Those rows are written continuously by agents (a query is persisted per analysis step), and every write moves the timestamps of these items. Before this list existed, each such write cost a FULL reload — every EntityInfo and EntityFieldInfo rebuilt (hundreds of MB on a large schema) to pick up a few QueryInfo rows; on a busy server that was a fresh half-gigabyte graph per refresh window until the process hit its memory guard.
Membership rule: an item qualifies only when no OTHER metadata class embeds or memoizes its
rows at construction. The query classes reference each other lazily through the provider
(QueryInfo.Fields filters QueryFields on first use), so they can be swapped as a set;
entity metadata cannot (EntityInfo owns its fields, permissions and relationships).
The family is always reloaded together — see PartialRefreshItemCodes.
Intersected with AllMetadataArrays so the list is valid for whatever this build knows.
Protected_Internal method to log SQL statement to all active logging sessions. This is called automatically by ExecuteSQL methods. Protected so platform-specific providers can reference it (e.g., to bind as a callback).
The SQL query being executed
Optionalparameters: unknownParameters for the query
Optionaldescription: stringOptional description for this operation
If true, this statement will not be logged
Whether this is a data mutation operation
OptionalsimpleSQLFallback: stringOptional simple SQL to use for loggers with logRecordChangeMetadata=false
OptionalcontextUser: UserInfoOptional user context for session filtering
ProtectedAbandonDrop a dead physical handle without going through public RollbackTransaction. Default is RollbackPhysicalTransaction if one is open. Subclasses override to unpublish even when the driver rollback itself rejects (EABORT).
ProtectedAdjustVirtual hook for platform-specific datetime field adjustments. Default implementation is a no-op (returns rows unchanged).
SQL Server overrides this to correct datetime2/datetimeoffset/datetime timezone interpretation issues in the mssql driver. PostgreSQL does NOT need to override — PG timestamp types are timezone-aware natively.
The data rows to process
Entity fields with TSType === Date
The entity metadata
The rows with datetime fields adjusted (or unchanged for default)
ProtectedAfterAfter a successful outermost commit, once depth is 0, the transaction lock is released, and the RunAfterCommit queue has been drained. Subclass hook; no-op by default.
ProtectedallocateProtectedallocateVariable suffix for DECLARE/SET (or any named-local dialect) in a save call.
Naming contract: _<12 lowercase hex> from
sha1(\${schema}.${table}|${normalized pk values joined by |}`), plus an optional _with n ≥ 2 when that hash repeats inside oneTransactionGroup (_abc123456789, _abc123456789_2, …). Outside a group there is no ordinal: each Save()is its own batch, or the SQL logger separates redeclarations.mj sync pushcaptures put a batch separator after every statement; threshold-mode sessions (Explorer logging,mj sync watch) concatenate saves into one batch, so SqlLoggingSessionImpl`
emits the separator before any statement that would redeclare a name already declared in
the current batch. Either way equal suffixes never share a scope.
Inside a BatchedSubmit group the ordinal is load-bearing: two items whose hashes
repeat (same record twice, or PK-less inserts) would otherwise declare the same locals in
one batch. SQLServerTransactionGroup.scopeItemVariables also appends _mjb<i> per item.
Ordinals are consumed at RENDER time, not at submit. An item whose SQL is regenerated
(the transaction-variables path re-renders Use items in HandleSubmit) carries _n
for a record rendered once before — deterministic run to run, but "same record → same
suffix" holds only for items rendered exactly once.
SQL Server's RenderSaveCallBinding consumes this; PostgreSQL positional/json-arg bindings do not name locals today but share the same GenerateSaveSQL orchestrator.
ProtectedApplyStrips fields the user cannot read from plain-object result rows.
This is the primary read-time control for list results, and it runs on BOTH cache hits and cache misses — the property the rest of the cache design is arranged around. Because it reads live metadata, a permission change takes effect on the next metadata refresh with no result-cache invalidation: the cached full-width superset stays valid and only the projection changes.
It composes with two per-request, never-cached narrowings that avoid pulling columns a
restricted service account has no business holding: the simple-path SELECT-list
intersection, and Load()'s allowed-column SELECT.
NEVER applied to entity_object results. Those become BaseEntity instances whose
fields round-trip through GenerateSaveSQL, which iterates ALL IsSPParameter fields
reading field.Value — not just dirty ones. A stripped field would therefore be written
back as a real NULL on the user's next save: silent data loss. Entity objects keep their
values in server memory exactly as encrypted fields do; the trust boundary is the API
output, which the GraphQL layer enforces separately.
OptionalcontextUser: UserInfoProtectedApplyApplies the PostRunView hook chain to a result that was served from cache, mutating
result in place so the caller's reference reflects the chain's output.
PostRunView is the OUTPUT half of the enforcement seam (data masking / audit). Hooks
receive contextUser, so masking is PER-USER, while the cache slot is shared across
users — there is no correct way to apply masking once at write time on behalf of a
reader who has not arrived yet. A hit that skips the chain therefore returns rows the
miss path would have masked.
This previously appeared to work by accident: PostRunView writes the cache BEFORE running the hooks, so a hook that masked rows in place was writing through into the cached objects — which both made later hits look masked and baked one user's masking decision into a shared slot. Freeze-on-write removes that write-through, which is what makes running the chain here necessary rather than merely tidier.
result in place is safeCache-hit results are FRESH wrapper objects built per hit by PreRunView/PreRunViews —
only .Results points at shared cache state. A hook that returns a replacement (the
required pattern now that rows are frozen) is copied onto that per-hit wrapper, so it
can never write back into the cache.
GetDataHooks is a memoized store read (~30ns), but await-ing the async chain costs
a microtask (~750ns) — comparable to the entire cache lookup this rides on. The
overwhelmingly common case is zero registered hooks, so check first and skip the await.
OptionalcontextUser: UserInfoProtectedapplyApplies in-memory pagination to query results based on StartRow and MaxRows parameters.
ProtectedApplyStrips or narrows the MJ: Record Changes payload columns a user may not read, using the
denied set of the entity each row is ABOUT rather than of Record Changes itself.
A sibling of ApplyFieldSecurityProjection rather than part of it, because that
method short-circuits on EnableFieldLevelSecurity for the entity named in the RunView
params — which here is MJ: Record Changes, whose flag is off in every default deployment.
Everything about the per-row denied set, the payload treatment, and the fail-closed decision
lives in RecordChangeFieldSecurityProjector; this is only the RunView wiring.
Runs at all four RunView projection points, matching the main projection: both cache-hit paths and both cache-miss paths. The cache-hit path is not optional — it is the exact path the original cross-user leak runs through, an unrestricted user warming a full-width slot that a restricted user then hits.
NEVER applied to entity_object results, for the reason the main projection is exempt plus
one specific to this one. The main reason transfers directly: an entity object's fields
round-trip through GenerateSaveSQL, which reads EVERY IsSPParameter field's value rather
than only dirty ones, so a withheld ChangesDescription would be written back as a real
NULL and a narrowed ChangesJSON as the narrowed payload — destroying audit history instead
of merely hiding it. Record Changes rows genuinely are saved through the entity layer
(replay writes Status/ErrorLog, users write Comments), so this is not hypothetical.
The additional reason is that the exemption cannot become a hole: the GraphQL RunView
resolver coerces entity_object to simple on the wire, so an entity_object Record
Changes result is by construction server-internal — and server-internal code holding full
values in memory is the documented trust boundary (FLS guide §3.4), exactly as for
encrypted fields.
OptionalcontextUser: UserInfoProtectedAssertThrow if a statement from a member of the ambient transaction would run on the pool while its
frames are still open after a server abort. Keyed on _doomed, not "depth > 0 with no handle"
— outermost begin has depth 1 before the handle is published, and concurrent reads on SQL
Server legitimately use the pool in that window. Callers outside the transaction's scope
(see IsCallerInAmbientTransaction) do not use its handle, so a subclass should apply
this check only to members' statements.
ProtectedAssertThrows unless a caller-supplied WHERE piece (ExtraFilter, OverrideExcludeFilter, a stored view
WhereClause) stays inside the parentheses this provider puts around it, read with this
provider's SQL dialect (see IsBalancedSQLFragment). The pieces are ANDed with row-level
security, which also carries API-key row filters and guest scope; a piece that closes its
parentheses early, such as 1=1) OR (1=1, would leave those filters on one side of an OR the
caller controls.
Optionaloptions: BalancedFragmentOptionsProtectedAssertSECURITY — refuses a statement separator or comment in a filter, sort or search fragment. Runs for every RunView caller, server-internal ones included, and cannot refuse a legitimate fragment. The full base-view screen runs at the client entry points; see ClientClauseScreen.
ProtectedassertGuards external-data-source reads against silently bypassing Row-Level Security. A remote system can't enforce MJ's RLS WHERE clauses, so if RLS would filter this user's rows we refuse the read with a clear error rather than returning unfiltered data. Users exempt from RLS (e.g. admins) get an empty clause and pass through — RLS wouldn't restrict them on an MJ-DB entity either. Called from the external RunView and Load dispatch points.
ProtectedassertRejects RunView params an external data source can't honor, rather than silently dropping them. Most important is AfterKey (keyset pagination): the external read path only supports offset paging, so a silently-dropped AfterKey would return the same page on every call — an infinite loop / duplicate processing in deep-pagination jobs. Aggregates and a non-empty UserSearchString likewise can't be evaluated remotely. Throws a clear error naming the param.
ProtectedAssertRejects a RunView whose ExtraFilter, OrderBy, or Aggregates expressions reference a
field the user cannot read.
Output projection alone is security theater. A user denied Salary can send
ExtraFilter: "Salary > 200000" or OrderBy: "Salary DESC" and reconstruct the values
from which rows come back and in what order — the column never appears in a result, so
every output-stripping point reports "secure." Aggregates are the same channel in a purer
form: Aggregates: [{expression: 'MIN(Salary)'}] under a narrow filter returns a denied
field's exact values directly. Predicate validation is a first-class enforcement point,
not a belt-and-braces afterthought. Together these cover every caller-authored expression
surface (UserSearchString is handled by excluding denied fields from the searched set,
not by rejection — see below).
Lives at the provider layer rather than in the GraphQL resolver (where the plan first
placed it) because every RunView funnels through here — the batch path, server-internal
agents and actions running under a restricted contextUser, and the resolver alike. One
gate, no path left uncovered.
The error deliberately does not say whether the field is missing or merely forbidden — see ProviderBase.FieldSecurityDenialMessage.
OptionalcontextUser: UserInfoProtectedassertThrows unless entityInfo has exactly one primary key column. For the few view features that
store or compare ONE bare key value per row (user view run logging / exclusion, the
{%UserView%} template's IN (subquery)), a composite key has no single column to use and
silently truncating it to the first column would return the wrong rows — so refuse loudly.
ProtectedauditCreates an audit log record for query execution (fire-and-forget).
Only logs if the query has AuditQueryRuns enabled or ForceAuditLog is set.
OptionalcontextUser: UserInfoBackground validation for the stale-while-revalidate fast-start pattern. Checks if local metadata is still current; if stale, fetches fresh metadata and atomically swaps it in. The app continues operating on cached data during this process — no blocking.
OptionalproviderToUse: IMetadataProviderBegins a transaction scope, or joins one already in flight on this provider.
This is the single transaction primitive for all multi-record entity work — IS-A parent
chains, composite save graphs and hand-written application cascades. It delegates to the
provider's existing depth-counted BeginTransaction / CommitTransaction /
RollbackTransaction, which already implement the join semantics: the outermost call
issues a physical BEGIN, nested calls create savepoints, and only the outermost commit
commits for real.
Routing IS-A through here is what closed the torn-write bug described in
EntityTransactionScope — the previous BeginISATransaction() opened a second
physical transaction on the same pool, blind to any transaction the caller had already
started.
The transaction belongs to the async scope that calls this: that code and everything it starts
run on the transaction; unrelated callers on the same provider instance run on the pool
(GenericDatabaseProvider, #4786). Call it from the function that performs the writes — a begin
reached only after an await inside a helper does not include the helper's caller.
The returned scope is settle-once: the first Commit() or Rollback() wins and later
calls are no-ops, so try { ...; Commit() } catch { Rollback() } is safe even when the work
already unwound its own scope.
A scope bound to this provider's ambient transaction.
Protected AbstractBeginOutermost BEGIN. Publish the driver object only after it has begun. Must not call public Begin/Commit/Rollback (the mutex is not reentrant).
Cache maintenance for the saves inside a transaction waits for the outermost settle: one rewrite per cached slot on commit instead of one per save, and nothing written for work that is rolled back.
The batch is opened HERE, on the raw primitive, not only in
DatabaseProviderBase.BeginEntityTransaction — 41 call sites across the repo use
BeginTransaction() directly, including new-user creation on first login, the magic-link
service, the roles/users sync resolver, MergeRecords and every generated cascade-delete
override. Those saves each published their own cache write, and a rollback left rows in the
shared cache that were never committed. BeginEntityTransaction still opens
its own level on top; the levels are depth-counted, so the batch applies once, at the
outermost settle, either way.
ProtectedBuildBuilds the aggregate query without running the caller's text. Each expression must be a single
aggregate function call over this entity's columns: the query runs the SQL that
CheckAggregateExpression rebuilds from the checked parse tree (with this provider's
dialect), once the keyword validator has also accepted that SQL. A refused expression becomes a
per-aggregate error in request order. The accepted ones are aliased Agg_0, Agg_1, … in
request order, which is how RunViewCore reads their values back.
ProtectedBuildThe freshness probe for one view: a row count, plus the newest __mj_UpdatedAt when the
entity has that column. Selecting it unconditionally made the probe fail outright for an
entity whose view does not carry it, instead of comparing by count alone the way
BaseEngine.SweepAgainstDatabase documents.
Protected AbstractBuildBuilds a UNION ALL query that checks each child entity's base table for a record with the given primary key. Used by FindISAChildEntity/FindISAChildEntities.
The child entities to search
The primary key value to find
ProtectedBuildBuilds dataset filters based on the provider configuration. Ensures MJ Core schema is always included and never excluded.
Array of filters to apply when loading metadata
ProtectedBuildBuilds the ExecuteSQLOptions for a Delete operation.
ProtectedBuildBuilds the query that reads a record's name fields by its primary key.
Key columns are written from the entity's metadata and key values are bound as parameters, so nothing the caller supplies reaches the SQL text. When a user is given, their effective read row filter (role RLS and API-key row filters) is ANDed in. A caller that gives no user is trusted server code, as for a single-record Load.
The entity name
The record's primary key
OptionalcontextUser: UserInfoThe acting user, whose read row filter applies
The SQL and its parameter values, or null if the entity has no name field or the key does not name the entity's primary key fields
ProtectedbuildReturns the SELECT list for a single-record load: * normally, or an explicit list of
the columns this user is allowed to read when field security denies them any.
Deliberately NOT routed through RunView, which is the other way to get this behavior.
That reroute is the tidier long-term shape and is planned separately (it must pass
BypassCache — a PK load has never been cache-served and must not silently start being
— and it changes relationship loading, which stays on its current path). Filtering the
column list here buys the same protection without touching either.
Protected AbstractBuildBuilds SQL for hard-link (foreign key) dependency queries. Returns a UNION ALL query across all dependent entities.
The entity-level dependency metadata
The primary key of the record being checked
ProtectedBuildValidates that the entity and RunViewParams are compatible with keyset (AfterKey) pagination,
then returns the SQL predicate (<pk> > 'value' or <pk> < 'value') and the resolved
order-by direction.
See AfterKeyNotSupportedError for the validation rules and RunViewParams.AfterKey for the API contract.
ProtectedBuildBuilds a platform-specific non-paginated row limit clause appended at end of query.
SQL Server: returns '' (already handled by TOP in SELECT clause).
PostgreSQL: returns LIMIT N.
Default: returns empty string. PG overrides.
Protected AbstractBuildBuilds a platform-specific pagination clause.
SQL Server: OFFSET X ROWS FETCH NEXT Y ROWS ONLY
PostgreSQL: LIMIT Y OFFSET X
Builds a parameter placeholder for parameterized queries. Default: PG-style ($1, $2, ...). SQL Server overrides to @p0, @p1, etc.
Zero-based parameter index
ProtectedbuildBuild the SQL fragment that compares one EntityField against a user search term.
Resolution order:
{0} replaced
by the single-quote-escaped raw term. Caller is responsible for escaping
LIKE metacharacters in their format string if needed.ProtectedbuildORDER BY column list covering EVERY primary key column of the entity, quoted for the dialect
— [ID] for a single-column key, [OrderID], [LineNo] for a composite one. Used as the
determinism fallback for row-limited queries with no caller ORDER BY: a composite key ordered
by its first column alone leaves rows sharing that value in undefined order.
ProtectedBuildBuilds the dialect-agnostic payload for a RecordChange row from the entity's old/new data and an optional restore or clone context. Concrete providers consume the returned payload to render their dialect-specific SQL (SQL Server EXEC, PostgreSQL INSERT, etc.).
Returns null when there's nothing to log — i.e., an Update where
DiffObjects found no field-level changes. Creates and Deletes
are always logged (one side of oldData/newData is null).
The payload's recordID is whatever the caller passes in. PG's
inline CTE save/delete paths can pass an empty string and resolve
the actual RecordID expression in SQL (because the post-INSERT PK
isn't known in JS); the standalone BuildRecordChangeSQL path
passes a fully-resolved composite-key string.
Post-change data (null for deletes).
Pre-change data (null for creates).
Composite-key serialized RecordID, or empty for CTE callers.
Entity metadata (provides EntityID + field shapes for diff).
Change type. Create and Delete skip the change-key short-circuit.
Acting user (provides UserID).
OptionalrestoreContext: RestoreContext | nullWhen non-null, populates source='Restore' plus the
lineage columns; otherwise source='Internal'.
OptionalquoteToEscape: stringQuote character for EscapeQuotesInProperties and
DiffObjects. Defaults to single quote.
OptionalcloneContext: CloneContext | nullWhen non-null, populates source='Clone' and the
structured changeContext JSON payload.
Protected AbstractBuildBuilds the SQL (and optional parameters) for inserting a record change entry. Each provider generates its own dialect: SQL Server uses EXEC spCreateRecordChange_Internal, PostgreSQL uses INSERT INTO "RecordChange" with parameterized values.
Returns null if there are no changes to log.
Implementations should delegate the dialect-agnostic assembly work to BuildRecordChangePayload and only handle SQL string rendering locally — that's how the duplication between SQL Server and PostgreSQL stays minimal.
OptionalrestoreContext: RestoreContext | nullWhen non-null, the resulting RecordChange row is
written with Source='Restore', RestoredFromID = SourceChangeID,
and RestoreReason = Reason. Read by callers from
BaseEntity.RestoreContext immediately before generating SQL.
OptionalcloneContext: CloneContext | nullWhen non-null, the resulting RecordChange row is
written with Source='Clone' and ChangeContext JSON provenance.
ProtectedBuildBuilds field projections for an RLS synthetic-row subquery (Create post-image and Update post-image checks).
Every non-virtual field is projected, null fields included: a filter referencing a
column the row leaves NULL must see SQL NULL, not a missing column (which fails to
bind with Invalid column name instead of evaluating). A bare SELECT NULL AS Col is
untyped and can evaluate differently from a real row, so nulls are emitted as
CAST(NULL AS <sqltype>) using the field's metadata-resolved type. An unresolvable
type fails the save (throw) rather than emitting an untyped NULL.
ProtectedBuildBuilds the ExecuteSQLOptions for a Save operation. SQL Server overrides to add connectionSource for IS-A shared transactions.
Protected AbstractBuildBuilds the SQL for a single sibling entity's record change entry in the propagation batch. SQL Server uses FOR JSON PATH + spCreateRecordChange_Internal. PostgreSQL uses json_build_object + INSERT INTO "RecordChange".
Optionalsource: RecordChangeSourceOptionalchangeContext: string | nullProtected AbstractBuildBuilds SQL for soft-link dependency queries (entities using EntityIDFieldName pattern). Returns a UNION ALL query across all soft-linked entities.
The entity name being checked for dependencies
The primary key of the record
ProtectedBuildBuilds a platform-specific TOP/LIMIT clause for non-paginated row limits.
SQL Server: TOP N; PostgreSQL returns empty (uses LIMIT via BuildPaginationSQL).
Default: returns empty string. SQL Server overrides.
ProtectedBuildBuilds the SELECT COUNT(*) AS TotalRowCount FROM ... SQL used to compute the total
row count for paginated views. Returns null when the view isn't row-limited (no count
query needed).
Two PG-parity details this method encapsulates — both caught real regressions:
When to emit the count query. Returns non-null whenever rows are being limited
— either explicit pagination or MaxRows/UserViewMaxRows. Earlier code keyed off
topSQL.length > 0, which was SQL-Server-specific: PG's BuildTopClause returns
empty (PG uses LIMIT appended at end via BuildNonPaginatedLimitSQL, not TOP
in the SELECT). So the old condition missed every PG case where MaxRows was set
without explicit StartRow — Explorer's Entity list was one such case and showed
"100 of 100" (no pagination) instead of "100 of 299".
Quote the alias via QuoteIdentifier. PostgreSQL folds unquoted column aliases
to lowercase, so AS TotalRowCount returns a row keyed totalrowcount on PG. The
caller reads countResult[0].TotalRowCount (PascalCase) and gets undefined — the
count falls back to retData.length (the page size), so pagination breaks silently.
SQL Server is case-insensitive so it worked unquoted there. Quoting via
QuoteIdentifier produces "TotalRowCount" on PG and [TotalRowCount] on SQL
Server — both preserve case.
OptionalbaseViewOverride: stringProtectedbuildBuilds the WHERE clause for cache status check, using same logic as InternalRunView. Handles ExtraFilter, UserSearch, and Row-Level Security. Subclasses can override to add platform-specific SQL transformations (e.g., identifier quoting).
Stores a dataset in the local cache. If itemFilters are provided, the combination of datasetName and the filters are used to build a key and determine a match in the cache
ProtectedcacheSECURITY — decide whether the shared cache must be BYPASSED for a RunView that targets
a saved VIEW rather than a named entity (no EntityName), under a context user.
The cache-hit path returns BEFORE the DB provider's read-permission gate
(CheckUserReadPermissions). The primary gate keys off the entity resolved from
params.EntityName, so a ViewID-/ViewName-only request (the Explorer-standard shape for a
saved view) yields no entity there and the gate is disarmed — a read-denied user could be
served rows a permitted user warmed for the same ViewID. The vw: fingerprint segment makes
the two users' requests collide on exactly one slot, so the leak is clean.
Returns true when the cache must be skipped for this call (fail-closed):
ViewEntity supplied and its entity resolves → apply the normal CanRead gate on it
(allow caching for a permitted user; deny for a read-denied one).ViewEntity absent/unresolvable but ViewID/ViewName present → fail closed: the view's
real entity (hence the user's permission) is only known after the async MJ: User Views
lookup that the cache-hit path deliberately skips, so we cannot safely consult the cache.
Returns false when there is no context user, when EntityName is set (the normal gate owns
that path), or when no view identifier is present at all (nothing to gate).OptionalcontextUser: UserInfoProtectedcacheCaches query results if caching is enabled for the query. Currently a no-op (query caching is not active).
ProtectedCancelCancels any pending debounced metadata refresh. Call during teardown (logout, provider disposal) so a timer armed just before teardown doesn't fire a refresh against a connection that no longer has a valid session.
ProtectedCanWhether a user may see this entity's record names: every field the name is built from must be readable to them. On an entity with field-level security on, a missing user may not, because there is nobody to check against.
OptionalcontextUser: UserInfoSnapshot of the caller's transaction frames. Synchronous, and never undefined on this
provider: outside a transaction it returns a token whose epoch is null, which says the work
is already durable rather than saying nothing. See
DatabaseProviderBase.CapturePostCommitToken. Work registered by a caller outside the
open transaction's scope (#4786) is not tied to that transaction, so such a caller gets the
same null-epoch token.
ProtectedCheckChecks whether a new record's field values pass the Create RLS filter. Builds a synthetic single-row subquery from entity field values, then tests the RLS filter against it. Null fields are projected as typed NULLs (see BuildRLSSyntheticRowProjections), so a filter that references a column the new row leaves NULL evaluates instead of failing to bind. The row is what the caller sends: a column the database fills on insert (a non-literal default or a trigger) is seen as NULL here.
ProtectedcheckChecks the paged cache for a specific page of query results. Returns a full RunQueryResult on hit, null on miss. Currently always returns null (query caching is not active).
ProtectedcheckChecks the query cache for existing results and returns them if valid. Currently always returns null (query caching is not active).
ProtectedCheckChecks whether an existing record passes the RLS filter for a given permission type. Executes: SELECT COUNT(*) AS cnt FROM view WHERE PK=value AND (RLS filter) Returns true if the record matches (cnt > 0), false otherwise.
Checks if local metadata is out of date and needs refreshing. Compares local timestamps with server timestamps.
OptionalproviderToUse: IMetadataProviderOptionalbypassMinCheckInterval: booleanWhen true, skips the MinRefreshCheckIntervalMs throttle. Event-driven callers pass true: they hold positive evidence that a metadata member entity was just written, and the throttle otherwise answers "fresh" for any check arriving within the window of the previous one — which would silently drop the second of two permission changes made less than the window apart. For the same reason a bypassing caller always runs its own check, even while another is in flight: that check may have started before the write.
The throttle is armed only when the status request succeeded (returned timestamps). A
check that throws or returns no timestamps leaves it unarmed, so a retried boot inside
the window really checks instead of reading "current" and loading nothing (#4887).
A non-bypassing caller arriving while a check is in flight is told no refresh is needed,
as the in-flight owner acts on the answer; that keeps N engines starting together down to
one request. It must not share the owner's answer: a shared true would send every caller
into its own full metadata reload (preValidateAndRefresh / backgroundValidateAndRefresh /
RefreshIfNeeded load outside Config's reload single-flight). For the same reason it gets
false, not the owner's rejection, when the owner's check throws.
True if refresh is needed, false otherwise
ProtectedCheckChecks whether an UPDATE's pending values still pass the Update RLS filter — the post-image side of the check. CheckRecordRLS validates the row as stored (pre-image); without this, a caller can update a row they legitimately own into a state they don't (e.g. reassign its owning organization) — privilege escalation the pre-image check cannot see.
Cost control: when the resolved filter FULLY decomposes into a conjunction of simple
column <op> value terms and none of those columns is dirty, the post-image equals
the pre-image on every column the predicate reads, so the (already-passed) pre-image
check suffices and the query is skipped. Any filter the decomposer does not fully
understand runs the check — a fail-open optimization in an authorization path is a
vulnerability with a benchmark attached, so partial understanding never skips.
ProtectedCheckChecks that the given user has read permissions on the specified entity. Throws if the user lacks CanRead permission.
The entity to check permissions for
The user whose permissions to check
If the specified datasetName is cached, this method will clear the cache. If itemFilters are provided, the combination of datasetName and the filters are used to determine a match in the cache
OptionalitemFilters: DatasetItemFilterType[]ProtectedCloneThe reuse-global fast path now builds a shared shell instead — see CreateSharedMetadataShell. The metadata graph is immutable after Config, so re-instantiating every Info object (~1s of synchronous constructor work for a ~600-entity install) bought no isolation the shell doesn't already provide. Subclass OVERRIDES of this method are still honored on the fast path (see CopyMetadataFromGlobalProvider) for backward compatibility; new customizations should override CreateSharedMetadataShell instead.
Protected AbstractCoercePer-field value transform applied before encryption + rendering. Each provider owns its type-coercion rules:
datetimeoffset → ISO string; non-PK uniqueidentifier
with () function-literal → skip (let DB default fire).gen_random_uuid()/uuid_generate_v4() → fresh UUID;
NOW()/CURRENT_TIMESTAMP → null (let DB default fire); bool /
binary / Date pass-throughs handled here.Returns { kind: 'use', value } to bind the (possibly transformed)
value, or { kind: 'skip' } to omit the field entirely.
Runs in GenerateSaveSQL before EncryptFieldValuesForSave and
before RenderSaveCallBinding.
Protected AbstractCommitOutermost COMMIT. Release the driver object in finally even if commit rejects.
Must not call public Begin/Commit/Rollback.
Commits the current transaction.
ProtectedCompleteFinalizes merge logging by updating the log record with completion status and creating deletion detail records. Uses BaseEntity with .Set() calls (no typed entity subclass imports).
OptionalcontextUser: UserInfoProtectedComputeThe CLIENT's field-security cache key: a canonical list of the fields this user MAY read.
Keyed on the ALLOWED set rather than the denied set for two reasons. Once metadata ships
to browsers filtered to what a user may see (#3485),
a denied field will not appear in the client's field list at all — so a denied-set key
would be empty and would silently stop segmenting. The allowed list also resolves the
f:* ambiguity in the client's projection segment, where "full width" means different
columns for different users.
Returns '' when the entity has field security off or the user is denied nothing, so unrestricted users keep byte-identical fingerprints and shared slots.
Only takes effect once the client's metadata refreshes. A client on stale metadata computes a stale key; the backstop is that the server strips denied columns from every fresh fetch regardless.
ProtectedcomputeComputes the latest update date for a dataset item from its result rows and dataset metadata. Used by both the cache-hit and cache-miss paths in GetDatasetByName.
The result rows (from cache or SQL). readonly because cache-hit callers
pass the cache's shared, frozen rows; this method only scans them.
The field name to scan for latest date
The dataset item metadata row (contains DatasetItemUpdatedAt, DatasetUpdatedAt)
The latest date across all rows and dataset metadata
ProtectedComputeThe fetch-widening field list for a cache-eligible request: always every entity field. One slot per (entity, filter, order) serves every caller regardless of the field subset they asked for, and field security narrows per request at read time via ApplyFieldSecurityProjection.
Optionalparams: RunViewParamsProtectedComputeThe field-security segment for a LocalCacheManager RunView fingerprint — the one place the client/server asymmetry is decided, so the two tiers cannot drift.
SERVER → no segment. Its slots are full-width and shared by every user; per-request narrowing happens at read time in ApplyFieldSecurityProjection, which runs on every hit and every miss. A segment here would fragment one shared slot into one per permission class and protect nothing the projection does not already handle.
CLIENT → the allowed-list key. Its slots are stored exactly as the server returned
them (already narrowed on the wire) and are not projected on read, so slot identity has to
carry the field set. Without it, a user whose access is tightened keeps being served their
persisted IndexedDB slot: the currency check compares maxUpdatedAt and rowCount only,
neither of which notices a column, so the server answers "current" and the browser keeps
showing a column that was just taken away.
Empty/undefined for unrestricted users on both tiers, so their fingerprints stay byte-identical and keep sharing slots.
ProtectedComputeComputes the per-user Row-Level-Security WHERE clause that InternalRunView will append to this query's SQL for the given user, so it can be folded into the cache fingerprint. RLS-scoped reads return a different result set than unscoped reads of the same entity+filter; without including the RLS clause in the cache key, a scoped user could be served a cached unscoped result set (a data leak).
Returns '' when the user is exempt from RLS on this entity (the common case), which makes the resulting fingerprint byte-identical to the pre-RLS format — preserving normal cache sharing.
Uses this (the active provider) to resolve the entity, never the global Metadata, so the
correct per-provider/per-tenant metadata is consulted.
OptionalcontextUser: UserInfoInstalls ProviderConfigDataBase.LocalStorageProvider, when the configuration carries one, before the base configuration loads metadata. A server fleet passes its shared (Redis) store this way so the metadata load and the engines loaded at startup use it from the first read: the first server to start fills the shared cache, and the others read it instead of each querying the database and broadcasting what they loaded.
OptionalproviderToUse: IMetadataProviderProtectedConvertConverts the byte-array values of an entity's binary fields to base64 strings, in place.
Database drivers return varbinary / binary / image (SQL Server) and bytea
(PostgreSQL) columns as Uint8Arrays (Node Buffers). MemberJunction's representation of
a binary value above the provider is a base64 string, so it is JSON-safe for dirty tracking,
Record Changes, the RunView caches and every transport. Values that are already strings,
null or absent are left untouched, which makes the conversion idempotent.
Rows as returned by the driver; mutated in place.
The entity's binary fields (EntityInfo.BinaryFields).
ProtectedConvertConverts every byte-array value in a set of rows to a base64 string, in place. For results with no entity metadata, such as RunQuery and ad-hoc SQL, where binary columns can only be recognised by value. See ConvertBinaryFieldsToBase64.
Rows as returned by the driver; mutated in place.
ProtectedConvertConverts dataset item filters into a unique string key for caching.
Array of filters to convert
JSON-formatted string representing the filters
ProtectedCopyAdopts the global provider's metadata for this instance without reloading it from the server: shares the (immutable post-Config) metadata arrays by reference via CreateSharedMetadataShell and builds this instance's entity lookup maps.
Creates an audit log record in the MJ: Audit Logs entity. Uses BaseEntity with .Set() calls (no typed entity subclass imports needed - can't use those from MJCore anyway). Callers typically fire-and-forget.
The user performing the action
Optional authorization name to look up
The audit log type name (must exist in metadata)
'Success' or 'Failed'
Optional details (JSON string, description, etc.)
The entity ID being audited
Optional record ID being audited
Optional description for the audit log
Save options to pass to the entity Save() call
The saved audit log BaseEntity, or null on error
Independent instance that shares the connection pool and metadata cache
but has its own transaction stack. Same pattern MJAPI uses for per-request
providers. Used by an entity directory that mj sync push writes with isolated
transactions, so parallel graphs do not interleave EntityTransactionScopes on one provider.
Not SQL Server-specific: each concrete provider implements this against its own pool. ReleaseIndependentInstance must NOT close the pool.
ProtectedCreateBuilds this instance's AllMetadata as a thin shell over another provider's already-loaded metadata: every metadata array is a PER-INSTANCE shallow copy whose elements are the SHARED Info object instances, and CurrentUser remains this instance's own.
Why sharing the instances is safe — and why this replaced the former deep clone (CloneAllMetadata) on the reuse-global fast path: the metadata graph is immutable after Config. Refreshes swap the WHOLE AllMetadata object (UpdateLocalMetadata), never mutate the Info objects in place, so the only per-instance datum inside the graph is CurrentUser — which this shell keeps independent. The deep clone cost ~1s of event-loop-blocking constructor work per provider on every server request (MemberJunction/MJ#3083); the shell is ~20 array-of-pointer copies (microseconds).
Why the array containers are copied rather than aliased: an in-place
.sort()/.push()/.splice() by request-scoped code then stays local to
that provider — matching the clone era's isolation for the common accidental
mutation class — instead of reordering the global graph for every other
in-flight request. Only the top-level AllMetadata collections get this
per-instance protection: everything below them is shared, including the
nested arrays owned by Info objects (entity.Fields,
entity.RelatedEntities, application.ApplicationEntities, ...) — an
in-place mutation of those is process-wide. Property writes on the shared
Info objects themselves are likewise visible process-wide (as they always
were on the client's global provider): treat Info objects and everything
they own as read-only; copy before sorting.
Owning provider: the shared Info objects deliberately keep the BaseInfo.OwningProvider of the provider that built them (the source global provider); the shell does NOT re-stamp them. Re-stamping a shared instance would re-point it for every other provider holding it. The sibling lookups it drives (IS-A children, RLS filters, query fields, ...) therefore read the source provider's graph — the same immutable graph this shell points into, so the answers are identical. A provider that needs its own graph (a different database) must load its own metadata, not adopt a shell.
Override precedence: if a subclass overrides BOTH this method and the deprecated CloneAllMetadata, the CloneAllMetadata override wins on the fast path (see CopyMetadataFromGlobalProvider) — the conservative back-compat choice, since pre-#3083 subclasses could only have customized adoption through CloneAllMetadata. Remove the CloneAllMetadata override to activate a CreateSharedMetadataShell override.
Creates a new SQL logging session that will capture all SQL operations to a file. Returns a disposable session object that must be disposed to stop logging.
Full path to the file where SQL statements will be logged
Optionaloptions: SqlLoggingOptionsOptional configuration for the logging session
Promise
// Basic usage
const session = await provider.CreateSqlLogger('./logs/metadata-sync.sql');
try {
// Perform operations that will be logged
await provider.ExecuteSQL('INSERT INTO ...');
} finally {
await session.dispose(); // Stop logging
}
// With migration formatting
const session = await provider.CreateSqlLogger('./migrations/changes.sql', {
formatAsMigration: true,
description: 'MetadataSync push operation'
});
AbstractCreateCreates a new transaction group for managing database transactions. Must be implemented by subclasses to provide transaction support.
A new transaction group instance
Converts a diff/changes object into a human-readable description of what changed.
The output of DiffObjects()
OptionalmaxValueLength: numberMaximum length for displayed values before truncation
OptionalcutOffText: stringText to append when values are truncated
ProtectedcreateBuilds user search SQL for the given entity and search string.
Supports full-text search (if enabled) and field-by-field LIKE searching. For the LIKE path:
OptionalcontextUser: UserInfoProtecteddatasetKey namespace for a dataset item's cached rows.
Dataset items are cached through the same fingerprint builder ordinary RunViews use, with
only { EntityName, ExtraFilter } — and every shipped item has a NULL WhereClause, so
without this segment a dataset item and a plain unfiltered read of the same entity produce
an IDENTICAL key and share one slot. That leaks the MJ_Metadata scaffolding exemption
(deliberately unfrozen rows) to ordinary callers of MJ: Entities / MJ: Entity Fields,
and lets an ordinary read repopulate an evicted slot FROZEN, which then breaks the next
metadata refresh.
Keyed by dataset + item code so two items over the same entity also stay distinct. Callers must use this on the read, the write-through, and the status paths alike — the three must agree or dataset reads stop finding dataset writes.
ProtecteddatasetWhether a dataset's reads run on the pool regardless of the ambient transaction. The metadata dataset is read by a timer-driven refresh that is not part of any caller's unit of work, so it must never land on a transaction's connection beside its COMMIT (#4514). Every other dataset keeps joining the ambient transaction: a caller that writes and then loads a dataset inside one transaction expects to see its own rows.
ProviderBase.RefreshWithinTransaction is the exception: the caller owns the transaction and is waiting for the reload, so the metadata reads join that transaction and see the caller's uncommitted rows (MJ#4836).
ProtectedDatasetCompares the per-entity row counts of a cached dataset against the server's status.
The timestamp comparison its callers run first catches new and updated rows; it cannot catch a
pure DELETE, which leaves LatestUpdateDate untouched while the count drops. This is the
check that catches those, and it is shared by GetAndCacheDatasetByName (which already holds
the blob) and IsDatasetCacheUpToDate (which reads it) so the two cannot drift apart again —
they had, and the differences were only ever masked by the warm path not running.
Two cases decide the shape of this, both load-bearing:
EntityUpdateDates never touches the blob
and is judged on its timestamp alone, so it answers true. Returning false here instead
looks strictly safer and is not: it makes such a dataset permanently stale, reloading on
every check (plan — the dataset-cache.DS2 regression)._date proxy expire independently and
a clear removes them in order, so the date can outlive the blob. Counts then cannot be
compared, which is false — refetch. A blob present but missing Results
answers false for the same reason rather than throwing, which is what the pre-unification
copy in the warm path would have done the moment that path started running.the cached dataset, which may be absent or incomplete
the server's status for the same dataset/filters combination
true when every entity the server reports matches the cache's row count for it
Deletes an entity record — the full orchestration flow shared by all DB providers.
Creates a changes object by comparing two JavaScript objects, identifying fields that have different values. Each property in the returned object represents a changed field, with the field name as the key.
The original data object to compare from
The new data object to compare to
Entity metadata used to validate fields and determine comparison logic
The quote character to escape in string values (typically "'")
A Record mapping field names to FieldChange objects, or null if either input is null/undefined. Only includes fields that have actually changed and are not read-only.
Disposes all active SQL logging sessions. Useful for cleanup on provider shutdown.
ProtectedEncryptEncrypts field values before saving to the database.
This method handles field-level encryption for any entity with encrypted fields. It is called by platform-specific providers (SQL Server, PostgreSQL) before building their SQL parameters, ensuring encryption is handled generically.
For each field marked with Encrypt=true:
EncryptionKeyID is null → throws an error (misconfiguration)The entity being saved
Map of field info to current values — values are mutated in-place
OptionalcontextUser: UserInfoUser context for encryption operations
ProtectedEnqueueEnqueues an after-save AI action for execution. By default, immediately adds to QueueManager. Subclasses with transaction support can override to defer until after transaction commit.
Optional_postCommitToken: PostCommitTokenThe transaction frames open when the save dispatched this action,
captured before any await — pass it to RunAfterCommit when deferring.
ProtectedenrichOptional, additive post-query enrichment step. Resolves the QueryResultEnricherBase
registered under params.Enrichment.EnricherKey via the MJGlobal ClassFactory and awaits
it on the result rows, returning whatever (column-appended) rows it produces.
Fully decoupled + resilient by design:
null and we no-op,
returning the original rows.The loaded QueryInfo (when resolvable from the executed query's id) is passed through so an enricher can read the query's associated entity/fields.
the assembled, paginated result rows to enrich
the run params carrying the RunQueryEnrichment directive
the executed query entity (used to resolve its QueryInfo metadata)
OptionalcontextUser: UserInfothe request user, threaded through for isolation/audit
the enriched rows on success, or the original rows on any failure / no-op
O(1) entity lookup by ID (UUID-normalized). Falls back to linear search if the internal Map hasn't been built yet.
O(1) entity lookup by name (case-insensitive, trimmed). Falls back to linear search if the internal Map hasn't been built yet.
ProtectedEntityUsed to check to see if the entity in question is active or not If it is not active, it will throw an exception or log a warning depending on the status of the entity being either Deprecated or Disabled.
OptionalcontextUser: UserInfoProtectedescapeEscape characters that have special meaning in SQL Server LIKE patterns.
The backslash itself must be escaped first so its replacement isn't reprocessed.
Pair with ESCAPE '\\' on the LIKE clause.
ProtectedEscapeRecursively escapes the specified quote character in all string properties of an object or array. Essential for preparing data to be embedded in SQL strings.
The object, array, or primitive value to process
The quote character to escape (typically single quote "'")
A new object/array with all string values having quotes properly escaped
ProtectedeventWhether the write described by entityEvent happened against the backend THIS provider's
metadata comes from. In a multi-provider process (a client connected to several MJ servers,
a server connected to several databases) a write on one backend must not refresh another's
metadata. Deliberately fails OPEN — when the event does not identify its provider, or a
connection string is unavailable, the answer is "yes": a spurious refresh is a bounded
cost, a suppressed one is a stale-permissions window.
ProtectedExecuteExecutes an ad-hoc SQL query directly, with security validation. SQL must be a SELECT or WITH (CTE) statement — mutations are rejected.
OptionalcontextUser: UserInfoProtectedExecuteExecutes an aggregate query and maps results back to the original expressions.
The SQL query to execute (from BuildAggregateSQL)
Original aggregate expression definitions
Any validation errors from BuildAggregateSQL
OptionalcontextUser: UserInfoUser context for query execution
Array of AggregateResult objects with execution time
Executes a query from a QueryExecutionSpec — the lower-layer interface-based entry point.
Runs the full pipeline: composition resolution → Nunjucks template processing → SQL execution.
Subclasses (GenericDatabaseProvider) provide the concrete implementation via InternalExecuteQueryFromSpec.
The execution spec describing the query, parameters, and inline dependencies
OptionalcontextUser: UserInfoOptional user context for permissions (required server-side)
Query results including data rows and execution metadata
ProtectedexecuteExecutes the query SQL and tracks execution time.
OptionalcontextUser: UserInfoOptionalparameters: unknown[]Optionaloptions: ExecuteSQLOptionsAbstractExecuteExecutes a SQL query with optional parameters and options.
The type of the result set
Optionalparameters: unknown[]Optionaloptions: ExecuteSQLOptionsOptionalcontextUser: UserInfoA promise that resolves to an array of results of type T
Executes multiple SQL queries and returns an array of result arrays, one per query.
The default implementation runs queries in parallel using Promise.all(ExecuteSQL(...)).
Platform-specific providers can override for true multi-result-set batching:
Array of SQL query strings to execute
Optionalparameters: unknown[][]Optional array of parameter arrays, one per query
Optionaloptions: ExecuteSQLBatchOptionsOptional batch execution options
OptionalcontextUser: UserInfoOptional user context for logging/filtering
Array of result arrays, one for each query
ProtectedexecuteOptionally wraps a view query with user view run logging. SQL Server overrides to use spCreateUserViewRunWithDetail. Default: returns null (no view run logging).
ProtectedextractProtectedextractExtracts the MAX value of a specified date field from result rows as an ISO string. Used for write-through caching of dataset item results.
The result rows
The field name to scan
ISO string of the max date, or current time if no dates found
ProtectedfindOptionalcontextUser: UserInfoDiscovers ALL IS-A child entities that have records with the given primary key. Used for overlapping subtype parents (AllowMultipleSubtypes = true) where multiple children can coexist.
The parent entity whose children to search
The primary key value to find in child tables
OptionalcontextUser: UserInfoOptional context user for audit/permission purposes
Array of child entity names found (empty if none)
Discovers which IS-A child entity, if any, has a record with the given primary key. Executes a single UNION ALL query across all child entity tables for maximum efficiency.
The parent entity whose children to search
The primary key value to find in child tables
OptionalcontextUser: UserInfoOptional context user for audit/permission purposes
The child entity name if found, or null if no child record exists
ProtectedformatFormats a CompositeKey value as a SQL literal for use in a keyset seek predicate.
This bypasses parameter binding because the entire WHERE clause is built as a string elsewhere in this provider (consistent with ExtraFilter / OrderBy handling). The seek value comes from server-side application code, not raw user input — but we still type- check and escape to defend against any caller passing a tainted value.
Strategy:
Performs a full-text search across all entities that have FullTextSearchEnabled=true. Uses the existing RunView + UserSearchString infrastructure which routes through the database-native FTS capabilities (SQL Server FREETEXT functions, PostgreSQL tsvector).
This is the default implementation that works across all database providers. Each provider's createViewUserSearchSQL() method handles the platform-specific SQL generation.
OptionalcontextUser: UserInfoProtected AbstractGenerateoptions carries per-delete behavior the SQL builder must honor (e.g. SkipRecordChanges).
Optionaloptions: EntityDeleteOptionsGenerates a new UUID suitable for use as a primary key or unique identifier. Uses uuidv4() from @memberjunction/global. Subclasses may override to provide platform-specific ID generation if needed.
A new UUID string
ProtectedGenerateConcrete implementation of the abstract save-SQL builder defined on
DatabaseProviderBase. Iterates fields via the single IsSPParameter
predicate, applies provider-specific value coercion, encrypts, then
delegates parameter binding, result-capture wrapping, and record-change
wrapping to abstract hooks the provider subclass implements.
See plans/sp-save-builder-generic-layer-refactor.md (rev 4) for
the design and the rev-3 lesson that motivated this shape.
Optionaloptions: EntitySaveOptionsGets information about all active SQL logging sessions. Useful for monitoring and debugging.
Array of session information objects
ProtectedGetRetrieves all metadata from the server and constructs typed instances. Uses the MJ_Metadata dataset for efficient bulk loading.
OptionalproviderToUse: IMetadataProviderOptionalforceRefresh: booleanComplete metadata collection with all relationships
Gets a database by name, if required, and caches it in a format available to the client (e.g. IndexedDB, LocalStorage, File, etc). The cache method is Provider specific If itemFilters are provided, the combination of datasetName and the filters are used to determine a match in the cache
OptionalitemFilters: DatasetItemFilterType[]OptionalcontextUser: UserInfoOptionalproviderToUse: IMetadataProviderProtectedgetExecutes cache status checks for multiple queries using their CacheValidationSQL. Default: parallel individual queries. SQL Server overrides for batch execution.
OptionalcontextUser: UserInfoProtectedgetExecutes cache status checks for multiple views. Default: parallel individual queries (works on all platforms). SQL Server overrides to use ExecuteSQLBatch for multi-result-set efficiency.
OptionalcontextUser: UserInfoThis routine gets the local cached version of a given datasetName/itemFilters combination, it does NOT check the server status first and does not fall back on the server if there isn't a local cache version of this dataset/itemFilters combination
OptionalitemFilters: DatasetItemFilterType[]Returns a record's name for synchronous display code, or undefined when it is not at hand.
This provider keeps no record names, so without loadIfNeeded the answer is always
undefined. A provider that serves a single user (GraphQLDataProvider) overrides the
record-name methods with a cache; one shared by several users must not, since a name can be
withheld from some of them.
The name of the entity
The primary key value(s) for the record
OptionalloadIfNeeded: booleanLook the name up when it is not cached
The display name, or undefined if not cached and not looked up
A record's cached name, without ever starting a lookup. Always undefined here; see GetCachedRecordName.
ProtectedgetValidates columns for a dataset item and returns the column list string. Returns null if columns are invalid.
An item that names no columns selects every column EXCEPT binary ones, matching RunView's
default (binary columns are large and rarely wanted; see guides/BINARY_FIELDS_GUIDE.md). A
binary column can still be named explicitly. When the entity's metadata is not loaded yet —
the first MJ_Metadata read at boot — the item falls back to *, and the rows' byte arrays
are converted to base64 by the caller.
Returns the stored procedure / function name for a Create or Update operation. Pure metadata lookup — no SQL execution needed. SQL Server uses spCreate/spUpdate naming, PostgreSQL uses the same pattern.
The entity being saved
True for Create, false for Update
The SP/function name
Protected AbstractGetGets the current user information from the provider. Must be implemented by subclasses to return user-specific data.
Current user information including roles and permissions
Retrieves a dataset by name, executing all item queries via ExecuteSQLBatch and aggregating results. Uses dialect-neutral quoting for all SQL construction.
ExecuteSQLBatch gives SQL Server true multi-result-set batching automatically, while PG (and the default) use parallel individual queries.
An ItemFilter that fails the denylist refuses its item. With a context user, each item needs CanRead on its entity and gets the user's read row filter (MJ_Metadata excepted). A refused item runs no SQL and fails the dataset.
OptionalitemFilters: DatasetItemFilterType[]OptionalcontextUser: UserInfoOptionalproviderToUse: IMetadataProviderOptionalforceRefresh: booleanCreates a unique key for the given datasetName and itemFilters combination coupled with the instance connection string to ensure uniqueness when 2+ connections exist
OptionalitemFilters: DatasetItemFilterType[]Retrieves status information for a dataset by name: per-entity row count and latest update date. Uses ExecuteSQLBatch for per-item status queries.
Items read with the same scope as GetDatasetByName. When any item is refused, the status fails with the refusal and no status SQL runs.
OptionalitemFilters: DatasetItemFilterType[]OptionalcontextUser: UserInfoOptionalproviderToUse: IMetadataProviderProtectedGetBuilds the dialect-quoted, comma-separated column list for EntityInfo.DatestampReturnFields
(primary key column(s) followed by __mj_CreatedAt and __mj_UpdatedAt) using QuoteIdentifier.
Examples:
[ID], [__mj_CreatedAt], [__mj_UpdatedAt]"ID", "__mj_CreatedAt", "__mj_UpdatedAt"ProtectedgetGets IDs of records deleted since a given timestamp. Uses dialect-neutral quoting. Subclasses can override for parameterized queries.
OptionalcontextUser: UserInfoProtectedgetReturns the SQLDialect instance for this provider's platform.
Subclasses override to return the appropriate dialect (e.g. SQLServerDialect, PostgreSQLDialect).
Used by PlatformBatchSeparator to retrieve the correct batch separator token via
@memberjunction/sql-dialect rather than hardcoding platform strings.
ProtectedGetResolves the view a RunView reads from: the entity's live base view by default, or its materialized
wrapper view when the caller opts into the snapshot via DataSource: 'Materialized' (plan §7). The
choice is explicit (never silent), so the same RLS/paging/field-selection apply against the identical shape.
Two materialization shapes:
BaseView stays the LIVE view; the
snapshot lives beside it as materialized_vw<CodeName> (the name CodeGen's base-view path emits).
'Materialized' swaps the live view for that snapshot.BaseView ALREADY IS the materialized wrapper
view (materialized_vw<...>), so there is no separate live source to swap — 'Materialized' is a
no-op and we return the entity's own base view. (Deriving materialized_vw<CodeName> here would be
wrong: the minted entity's CodeName need not match the query-derived view name.)Convention-based for the base-view case: if the entity has no such materialization the wrapper view
won't exist and the read will error — opting into 'Materialized' asserts the snapshot exists.
ProtectedGetReturns AI actions configured for the given entity and timing. Uses AIEngine metadata to find matching EntityAIAction records.
Returns a list of entity dependencies, basically metadata that tells you the links to this entity from all other entities.
Creates a new instance of a BaseEntity subclass for the specified entity and automatically calls NewRecord() to initialize it. This method serves as the core implementation for entity instantiation in the MemberJunction framework.
The name of the entity to create (must exist in metadata)
OptionalcontextUser: UserInfoOptional user context for permissions and audit tracking
Promise resolving to the newly created entity instance with NewRecord() called
Creates a new instance of a BaseEntity subclass and loads an existing record using the provided key. This overload provides a convenient way to instantiate and load in a single operation.
The name of the entity to create (must exist in metadata)
CompositeKey containing the primary key value(s) for the record to load
OptionalcontextUser: UserInfoOptional user context for permissions and audit tracking
Promise resolving to the entity instance with the specified record loaded
Gets the display name for a single entity record. Uses the entity's IsNameField or falls back to 'Name' field if available.
The name of the entity
The primary key value(s) for the record
OptionalcontextUser: UserInfoThe acting user; field- and row-level security are applied for them
OptionalforceRefresh: booleanBypass any cache. This provider has none, so it always looks up.
The display name of the record, or an empty string if not found or not readable
Gets display names for multiple entity records in a single operation.
Array of entity/key pairs to lookup
OptionalcontextUser: UserInfoThe acting user; field- and row-level security are applied for them
OptionalforceRefresh: booleanBypass any cache. This provider has none, so it always looks up.
Array of results with names and status for each requested record
ProtectedGetRecursively enumerates an entity's entire sub-tree from metadata. No DB queries — uses EntityInfo.ChildEntities which is populated from metadata.
ProtectedGetRetrieves the latest metadata update timestamps from the server.
OptionalproviderToUse: IMetadataProviderArray of metadata update information
Returns the timestamp of the local cached version of a given datasetName or null if there is no local cache for the specified dataset
the name of the dataset to check
OptionalitemFilters: DatasetItemFilterType[]optional filters to apply to the dataset
Retrieves the change history for a specific record. Uses the vwRecordChanges view which exists in both SQL Server and PostgreSQL.
The entity name
The record's composite primary key
OptionalcontextUser: UserInfoOptional context user
Returns a list of record-level dependencies — records in other entities linked to the specified entity/record via foreign keys (hard links) or EntityIDFieldName soft links. Uses abstract SQL builders for dialect-specific query generation.
The entity name to check
The primary key(s) of the record
OptionalcontextUser: UserInfoOptional context user
Initiates duplicate detection for a list of records. Uses BaseEntity to create a Duplicate Run record. Subclasses may override to provide additional functionality.
The duplicate detection request parameters
OptionalcontextUser: UserInfoThe acting user
A response indicating the duplicate detection status
Gets the favorite record ID if the record is a favorite for the given user, null otherwise.
OptionalcontextUser: UserInfoChecks if a record is marked as a favorite for a given user.
OptionalcontextUser: UserInfoProtectedgetResolves the list of EntityFieldInfo objects for a view query. Priority: params.Fields > view columns > all entity fields (wildcard).
Field-level security intersects every resolution path with the user's ALLOWED set, so a denied column never appears in the SELECT list and its values never leave the database:
params.Fields and saved-view columns are silently narrowed (a denied entry
is dropped without the "Field not found" error — Fields describes output shape, not
a predicate, and the caller learns nothing projection didn't already show them);SELECT *, becomes the explicit
allowed-column list;entity_object requests are EXEMPT — entities must hydrate from every column (see
ApplyFieldSecurityProjection) and their enforcement stays at the output boundary;OptionalcontextUser: UserInfoProtectedgetBuilds the SQL field list string for a view query, using dialect-neutral quoting. Returns '*' if no specific fields are resolved.
OptionalcontextUser: UserInfoRow count and newest __mj_UpdatedAt for each view, read from the database and never from
a cache. Uses the same WHERE clause (filter, user search, row-level security, pre-run hooks)
and the same batched status query as RunViewsWithCacheCheck. One entry per param,
in order; a failure is reported per entry.
OptionalcontextUser: UserInfoGets a specific SQL logging session by its ID. Returns the session if found, or undefined if not found.
The unique identifier of the session to retrieve
The SqlLoggingSession if found, undefined otherwise
ProtectedGetReturns provider-specific extra data to attach to a TransactionItem. SQL Server overrides to include { dataSource: this._pool }.
ProtectedgetGets rows updated/created since a given timestamp. Uses dialect-neutral quoting and TransformExternalSQLClause for OrderBy.
OptionalcontextUser: UserInfoProtectedHandleHandles entity actions (non-AI) for save, delete, or validate operations. Uses EntityActionEngineServer to discover and run active actions.
After-hooks run under EntityActionDispatchGuard; before-hooks and Validate do not.
The guard suppresses an action re-entering itself on the same record (the enrich-and-write-back
loop) and coalesces a burst of saves into one pending rerun. Neither belongs on the
synchronous half of the pipeline: Validate and Before* participate in the save and can
abort it, so skipping one would let a record through that should have been refused, and
deferring one would decide the save's outcome after it had already happened.
An After-hook binding may also ask to run durably (EntityAction.RunMode = 'Durable').
After-hooks are dispatched fire-and-forget, so a process that dies mid-flight loses the work
with nothing to retry it; durable dispatch hands it to the task-graph substrate instead (D14).
Validate and Before* ignore RunMode entirely — deferring work that decides whether the
save succeeds is not a durability improvement, it is a different feature.
OptionaloriginatingEntityActionIDs: string[]ProtectedHandleHandles Entity AI Actions for save or delete operations.
For "before save" actions: blocks (awaits) until complete. For "after save" actions: fires and forgets via QueueManager.
Subclasses that manage transactions can override to defer after-save tasks until after transaction commit (see SQLServerDataProvider).
ProtectedHandleNested savepoint rollback failed. Abandon the physical handle and keep frames until the outer settle.
ProtectedhandleStatic fan-out callback: a BaseEntity save/delete (or a remote-invalidate from another
server) touched lowerEntityName. If that entity is one of the entities this provider's
metadata is BUILT FROM, the metadata this provider is serving — and, on the server, the
metadata every per-request provider adopts from it — is now stale, so schedule a debounced
refresh. Permission metadata is the load-bearing case: field-level security is enforced
FROM metadata at every enforcement point, so a rule an administrator just tightened is
simply not enforced until this re-read happens.
Reacts to another server saving its metadata snapshot: after a short debounce, runs the normal staleness check (RefreshIfNeeded, bypassing the check throttle). The check compares this process's timestamps with the database and adopts the shared snapshot, so a metadata change made on one server reaches the others in seconds instead of at the next poll. A server that is already current does no refresh and saves nothing, so notices do not echo around the fleet.
true when the event was a metadata notice (and a check was scheduled)
Whether a record's name is cached for synchronous retrieval. Always false here; see GetCachedRecordName.
ProtectedInternalLower-layer execution: resolves composition, processes templates, executes SQL. This is the single execution pathway used by both saved queries (via RunQuery upper layer) and transient test queries (via TestQuerySQL resolver).
Processing order:
OptionalcontextUser: UserInfoProtectedInternalRetrieves the display name for a single entity record. Uses BuildEntityRecordNameSQL for dialect-neutral SQL generation, which limits the lookup to rows the acting user's read row filter allows.
Answers with an empty string, without querying, when field-level security withholds any of the name fields from the acting user — the same answer as a record that does not exist, so the lookup cannot be used to tell the two apart.
OptionalcontextUser: UserInfoProtectedInternalRetrieves display names for multiple entity records.
OptionalcontextUser: UserInfoProtectedInternalServer in-process transport for Remote Operations: resolves the registered operation by key
and runs it via ExecuteServer. Inherited by both SQL Server and PostgreSQL providers. The
client (GraphQL) provider overrides this to marshal over the wire instead.
ProtectedInternalBatch query execution — runs all queries in parallel.
OptionalcontextUser: UserInfoProtectedInternalInternal implementation of RunQuery that subclasses must provide. This method should ONLY contain the query execution logic - no pre/post processing. The base class handles all orchestration (telemetry, caching).
The query parameters
OptionalcontextUser: UserInfoOptional user context for permissions
ProtectedInternalShared InternalRunView implementation. Handles: view resolution, permissions, field selection, WHERE clause building (view + extra filter + user search + exclude + RLS), ORDER BY, pagination, aggregates, parallel query execution, post-processing, and audit logging.
OptionalcontextUser: UserInfoProtectedInternalInternal implementation of RunViews that subclasses must provide. This method should ONLY contain the batch data fetching logic - no pre/post processing. The base class handles all orchestration (telemetry, caching, transformation).
Array of view parameters
OptionalcontextUser: UserInfoOptional user context for permissions
ProtectedinvalidateDrops every in-flight/lingered RunView entry whose params touch the given entity (lowercased name). Called on BaseEntity save/delete/remote-invalidate.
Drops this provider's in-flight/lingered RunView dedup entries so the next identical RunView goes to the server instead of replaying a result from the last DedupLingerMs.
Saves and deletes made through a BaseEntity in this process already invalidate the entries
for their entity automatically. Call this after anything that can write on the SERVER
without such an event reaching this process — a Remote Operation, an Action, an agent run,
any other mutation. RouteOperation and the GraphQL client's mutation transport call
it for you.
Dropping an in-flight entry never affects the callers already waiting on it; it only stops later callers from joining (and stops the result from lingering), since that query may have read pre-write data.
OptionalentityNames: readonly string[]entities known to have been written. Only entries touching them are dropped. Omit (or pass an empty list) when the writes are unknown: every entry is dropped.
ProtectedisCompares client cache status with server status to determine if cache is current. Checks both row count and maxUpdatedAt timestamp.
ProtectedIsWhether the calling async scope is part of this provider's ambient transaction (#4786). A shared provider's open transaction belongs to the scope that began it and its descendants; every other caller must run on the pool, as if no transaction were open. Subclasses consult this — not merely "is a handle open" — before routing a statement to the transaction handle.
ProtectedisDetects infrastructure-level connection errors (timeout, refused, pool closed) as opposed to query-level errors (bad SQL, constraint violations). Delegates to the dialect's driver-specific error classification, with a fallback for POOL_CLOSED errors thrown by our own code.
Determines if a given datasetName/itemFilters combination is cached locally or not
OptionalitemFilters: DatasetItemFilterType[]This routine checks to see if the local cache version of a given datasetName/itemFilters combination is up to date with the server or not
OptionalitemFilters: DatasetItemFilterType[]ProtectedIsChecks whether a given entity matches the target name, or is an ancestor of the target (i.e., the target is somewhere in its descendant sub-tree). Used to identify and skip the active branch during sibling propagation.
ProtectedIsA saved query is "external" when it resolves to a Query bound to an external data source. Used by the base RunQuery CacheLocal layer to defer external-query caching to InternalRunQuery's runExternalQueryWithCache. Non-throwing — resolves from cached metadata.
ProtectedisTrue when the entity is a CodeGen materialized-query wrapper (materialized_vw*) whose snapshot is refreshed out-of-band — the same one IsServerCacheAllowedForEntity excludes from the server cache.
True when a shared-cache event says the metadata snapshot changed: another process saved one
(set), or a tool that changed the database removed it (removed, e.g. mj migrate). Only
the timestamps key counts — it is written last (see SaveLocalMetadataToStorage) and
removed last — so each save or removal is one notice.
Whether entityName is one of the entities this provider's metadata is built from.
False when the membership set has not been loaded yet — a caller must not treat "unknown"
as "reload". Names are compared case-insensitively, matching the set recorded by
registerMetadataDatasetMembership.
Checks whether a string value looks like a known database default-value function that is NOT a UUID generator for this provider's platform.
The string value to check
true if the value matches a known non-UUID database function pattern
ProtectedIsChecks whether server-side caching is allowed for the entity in the given RunViewParams. Returns false for entities that have TrustServerCacheCompletely = false, or for Record Changes which is always exempt (rows are created via raw SQL side-effects, not BaseEntity.Save(), so cache invalidation events never fire).
ProtectedisTrue if the field's column type is appropriate for text-pattern search. Non-text types are rejected because LIKE forces an implicit per-row CONVERT to nvarchar. Unbounded (MAX/ntext/text) columns are rejected because the LIKE path cannot seek them; FTX is the right tool for those.
Checks whether a string value looks like a database UUID generation function for this provider's platform.
The string value to check
true if the value matches a known UUID generation function pattern
Loads a single entity record by composite key, with optional relationship loading. Uses dialect-neutral quoting for all SQL construction.
ProtectedLoadLoads metadata from local storage if available. Deserializes and reconstructs typed metadata objects.
Checks if local metadata is obsolete compared to remote metadata. Compares timestamps and row counts to detect changes.
Optionaltype: stringOptional specific metadata type to check
True if local metadata is out of date
ProtectedLogLogs a record change entry by diffing old/new data and executing provider-specific SQL to insert the record change. Concrete orchestration; SQL generation is delegated to BuildRecordChangeSQL.
The new record data (null for deletes)
The old record data (null for creates)
The entity name
The record ID (CompositeKey string)
The entity metadata
The change type
The acting user
OptionalrestoreContext: RestoreContext | nullOptionalcloneContext: CloneContext | nullProtectedMapProtectedmarkProtectedMaterializationReports whether a materialization-metadata RunView FAILED, logging the failure when it did.
Every materialization read path falls back to LIVE data when it cannot confirm an Active snapshot, and
that fallback is correct — live data is always right, materialization is a transparent optimization,
never a correctness dependency. The defect this guards is the silence: RunView signals an
authorization or query failure through Success === false rather than by throwing, so collapsing a
failure into the same branch as the legitimate "no materialization row exists" case makes the two
indistinguishable to operator and caller alike.
That matters because read access to the materialization entities is role-gated (CanRead is granted
only to the UI / Developer / Integration roles). A user on a restricted role — including MJ's
magic-link / external-access pattern — therefore has every one of these lookups fail, and so
permanently reads LIVE data for every DataSource:'Materialized' request, while an admin issuing the
identical request is served the snapshot. Two users, different data, no error surfaced to either.
Logging leaves the safe fallback intact but makes the divergence diagnosable.
the RunView result to inspect (structurally a RunViewResult).
human-readable description of what was being resolved, for the log message.
true if the lookup failed and the caller should take its live-data fallback; false otherwise.
ProtectedmergeMerges cached and fresh results for RunViews, maintaining original order.
The pre-processing result with cache info
OptionalcachedResults?: RunViewResult[]OptionalcacheStatusMap?: Map<OptionalcallerFieldsMap?: Map<number, string[]>Per-param-index caller Fields lists (lowercased), captured before PreRunViews widened params.Fields to all entity fields for cache-superset storage. An index is present ONLY when that widening actually happened — PostRunViews uses it to project cache-miss DB results back down to the requested shape.
OptionalfingerprintMap?: Map<number, string>Per-param-index cache fingerprints computed during PreRunViews — carried forward so PostRunViews doesn't recompute the RLS where-clause and fingerprint string for every batch item.
OptionalpreResolvedCache?: Map<string, CachedRunViewResult | null>Pre-resolved local cache entries keyed by client cache fingerprint, populated during prepareSmartCacheCheckParams so executeSmartCacheCheck can reuse the already-parsed results for 'current' items without a second IndexedDB read.
OptionalsmartCacheCheckParams?: RunViewWithCacheCheckParams[]When CacheLocal is enabled, contains the cache check params to send to server
OptionaltelemetryEventId?: stringOptionaluncachedParams?: RunViewParams[]OptionaluseSmartCacheCheck?: booleanWhen CacheLocal is enabled, indicates we should use smart cache check
The fresh results from InternalRunViews
Combined results in original order
ProtectedmergeFolds a saved view's stored WhereClause and OrderByClause into the RunView params before external dispatch. The external branch returns before the normal SQL path that applies them, so without this a UserView over an external entity would silently return unfiltered, unordered rows. The view's WhereClause is ANDed with any caller ExtraFilter; the view's OrderByClause is used only when the caller supplied no OrderBy. Returns params unchanged when there is no saved view.
ProtectedmergeMerges cached and fresh results for RunQueries, maintaining original order.
The pre-processing result with cache info
The fresh results from InternalRunQueries
Combined results in original order
Merges multiple records into a single surviving record. Full orchestration: transaction, field map update, dependency re-pointing, deletion, and merge logging.
The merge request with surviving record and records to merge
OptionalcontextUser: UserInfoThe acting user
Optional_options: EntityMergeOptionsOptional merge options
The merge result
The entities this provider's metadata is built from that declare they can change without
firing an event (Entity.TrustServerCacheCompletely === false).
Empty — the usual case — means a metadata sweep has nothing to look for, so a caller can skip the work (and any lease) without asking the database anything.
ProtectedNormalizeNormalizes non-entity ('simple') result rows so Date and numeric columns hold real
Dates and numbers on EVERY tier, matching what the generated entity types declare.
Before this existed, the value a simple read returned for a DATETIME column depended on
where the code happened to run: a fresh server-side query yields real Date objects (the
driver parses them and AdjustDatetimeFields timezone-adjusts them), a server-side Redis
cache hit yields ISO strings (JSON.parse with no reviver), and a browser client over
GraphQL yields ISO strings (rows are JSON.stringify'd on the wire). Same call, three
shapes. MJ's contract is a unified programming interface on both sides of the wire, so the
one representation the platform's own generated types declare — Date — is enforced here,
at the one choke point every provider's RunView pipeline flows through.
It makes date and number VALUES match the generated types; it does not make a caller's T
honest in general. A Status column typed as a closed union still holds whatever string the
database held, and plain rows never have entity methods. If you need the type to be fully
true, use ResultType: 'entity_object'.
The field-key lists are computed once per view from EntityInfo, not per cell. Rows already
in the right shape — the common server-side case, where the driver returned Dates — are
detected and the ORIGINAL array is kept untouched: same array identity, same row objects,
zero copying. A row is shallow-copied only when a cell actually converts, and that copy is
load-bearing: on a cache hit the rows handed back can be the cache's OWN objects (the
in-memory server store holds them by reference), so converting in place would write Dates
into the cache entry itself and corrupt it for serialization and for later readers.
Per-cell rules:
Date instances pass through untouched, so the pass is idempotent on every path.NULL/undefined cells are left alone rather than becoming epoch-1970 dates.Invalid Date, which renders
as that literal string and destroys the evidence of what the database actually held.Number.MAX_SAFE_INTEGER stays a string: the PostgreSQL
provider deliberately returns unsafe-range BIGINTs as strings to avoid precision loss,
and Number('9007199254740993') "succeeds" while silently corrupting the value.View-based runs (ViewID/ViewName with neither EntityName nor a loaded ViewEntity)
skip normalization: resolving the entity would take an async User Views read this late in
the pipeline. Pass EntityName alongside the view identifier to get normalized rows.
ProtectedOnCalled after a successful delete. Intentionally synchronous — see OnAfterSaveExecute.
ProtectedOnCalled after a successful save (both direct and transaction-callback paths). Intentionally synchronous (fire-and-forget) — SQL Server overrides to dispatch after-save entity actions and AI actions without awaiting.
ProtectedOnCalled before the delete SQL is executed. SQL Server overrides to fire before-delete entity actions and AI actions.
ProtectedOnCalled before the save SQL is executed. SQL Server overrides this to fire before-save entity actions and AI actions.
ProtectedOnCalled when a begin fails and depth is back to 0 — unpublish any leftover driver object.
ProtectedOnCalled after a save/delete SQL operation completes (success or failure) to resume refresh.
ProtectedOnCalled after a direct (non-transaction) save succeeds, before the result is returned to the caller and loaded into the entity via finalizeSave().
This hook can optionally return a Record<string, unknown> containing field values
that should be patched onto the SP result row before it is loaded into the entity.
This solves a timing problem: the SP result is captured before OnSaveCompleted runs,
so any data created by post-save hooks (e.g., geocoding writing to a JOINed table)
would be stale in the returned entity without this patch.
result[0] with the current view dataOnSaveCompleted runs post-save logic (geocoding, ISA propagation, etc.)result[0] via Object.assign(), overwriting stale valuesAfter geocoding updates RecordGeoCode, the new lat/lng are returned as patches
for the __mj_Latitude and __mj_Longitude virtual fields that come from the
RecordGeoCode JOIN in the entity's base view. Without this patch, those fields
would contain the pre-geocoding values until the next query.
null if no patches are needed (default behavior)await super.OnSaveCompleted(...) and merge its patches with yoursPatch fields to apply to the SP result, or null if no patches needed
ProtectedOnCalled before starting a save/delete SQL operation to pause background metadata refresh. SQL Server overrides to set _bAllowRefresh = false.
ProtectedOnProtectedparseParses a timestamp string sent by the client.
Accepts both ISO 8601 strings (the canonical form) and all-digit strings
representing milliseconds since epoch. The numeric form has been observed
in the wild from clients whose cache layer round-tripped a Date through a
lossy serializer — new Date('1778004618383') returns Invalid Date,
but new Date(Number('1778004618383')) is a valid timestamp. Returning
null on unparseable input lets callers degrade to a stale-cache fallback
instead of throwing RangeError: Invalid time value on a downstream
.toISOString().
ProtectedPartialDecides whether the current staleness can be served by a scoped reload of the saved-query family. Compares local and remote timestamps item by item (the same test as LocalMetadataObsolete) and maps each stale entity back to its MJ_Metadata item code.
the family's item codes (the whole family — the query classes are swapped as a set) when every stale item belongs to it; null when anything else is stale, nothing is, or the inputs needed to attribute the change are missing — the full reload is then the answer.
ProtectedPostOptionalorganicKeys: OrganicKeyMetadataRow[]OptionalorganicKeyRelatedEntities: OrganicKeyRelatedEntityMetadataRow[]OptionalfieldPermissions: EntityFieldPermissionMetadataRow[]ProtectedPostPost-processes rows: first applies platform-specific datetime adjustments
via the virtual AdjustDatetimeFields hook, then handles field-level
decryption for encrypted fields.
Subclasses should NOT override this method. Instead, override
AdjustDatetimeFields for platform-specific datetime corrections.
ProtectedPostBase class post-processor that all sub-classes should call after they finish their RunView process
OptionalcontextUser: UserInfoProtectedPostBase class utilty method that should be called after each sub-class handles its internal RunViews() process before returning results This handles the optional conversion of simple objects to entity objects for each requested view depending on if the params requests a result_type === 'entity_object'
OptionalcontextUser: UserInfoProtectedPostPost-processing hook for RunQueries (batch). Handles telemetry end.
Array of query results
Array of query parameters
The pre-processing result
OptionalcontextUser: UserInfoOptional user context
ProtectedPostPost-processing hook for RunQuery. Handles cache storage and telemetry end.
The query result
The query parameters
The pre-processing result
OptionalcontextUser: UserInfoOptional user context
ProtectedPostPost-processing hook for RunView. Handles result transformation, cache storage, and telemetry end.
The view result
The view parameters
The pre-processing result
OptionalcachedResult?: RunViewResultOptionalcallerRequestedFields?: string[] | nullThe caller's original Fields list (lowercased), captured before PreRunView widened params.Fields to all entity fields for cache-superset storage. Non-null ONLY when that widening actually happened — PostRunView uses it to project cache-miss DB results back down to the requested shape.
Optionalfingerprint?: stringOptionaltelemetryEventId?: stringOptionalcontextUser: UserInfoOptional user context
ProtectedPostPost-processing hook for RunViews (batch). Handles result transformation, cache storage, and telemetry end.
Array of view results
Array of view parameters
The pre-processing result
OptionalcachedResults?: RunViewResult[]OptionalcacheStatusMap?: Map<OptionalcallerFieldsMap?: Map<number, string[]>Per-param-index caller Fields lists (lowercased), captured before PreRunViews widened params.Fields to all entity fields for cache-superset storage. An index is present ONLY when that widening actually happened — PostRunViews uses it to project cache-miss DB results back down to the requested shape.
OptionalfingerprintMap?: Map<number, string>Per-param-index cache fingerprints computed during PreRunViews — carried forward so PostRunViews doesn't recompute the RLS where-clause and fingerprint string for every batch item.
OptionalpreResolvedCache?: Map<string, CachedRunViewResult | null>Pre-resolved local cache entries keyed by client cache fingerprint, populated during prepareSmartCacheCheckParams so executeSmartCacheCheck can reuse the already-parsed results for 'current' items without a second IndexedDB read.
OptionalsmartCacheCheckParams?: RunViewWithCacheCheckParams[]When CacheLocal is enabled, contains the cache check params to send to server
OptionaltelemetryEventId?: stringOptionaluncachedParams?: RunViewParams[]OptionaluseSmartCacheCheck?: booleanWhen CacheLocal is enabled, indicates we should use smart cache check
OptionalcontextUser: UserInfoOptional user context
ProtectedPrepareApplies TransformExternalSQLClause to a caller's ExtraFilter and checks the result with AssertClauseBalanced. When the transform changes the text and the filter is a chain of top-level AND terms, each term is transformed, checked and parenthesized on its own. A PreRunView hook that ANDs a server predicate (such as tenant scope) onto the caller's filter runs before this transform, so a rewrite that changes the caller's term cannot reach the hook's term.
ProtectedPreOptionalcontextUser: UserInfoProtectedPreBase class implementation for handling pre-processing of RunViews() each sub-class should call this within their RunViews() method implementation
OptionalcontextUser: UserInfoProtectedPrePre-processing hook for RunQueries (batch). Handles telemetry for batch query operations.
Array of query parameters
OptionalcontextUser: UserInfoOptional user context
Pre-processing result
ProtectedPrePre-processing hook for RunQuery. Handles telemetry and cache lookup.
The query parameters
OptionalcontextUser: UserInfoOptional user context
Pre-processing result with cache status and optional cached result
ProtectedPreOptionalcontextUser: UserInfoProtectedPrePre-processing hook for RunViews (batch). Handles telemetry, validation, and cache lookup for multiple views.
Array of view parameters
OptionalcontextUser: UserInfoOptional user context
Pre-processing result with cache status for each view
Synchronous pre-validation of cached metadata before engine startup.
On a warm load we serve the metadata graph from IndexedDB so the app can boot without pulling MBs of metadata from the server. Before engines run, this method makes one batched timestamp round-trip to confirm the snapshot is still current:
Cost on the warm-current path is one batched timestamp fetch (~50–200 ms depending on RTT). On the warm-stale path we additionally pay the full metadata fetch but avoid serving stale data to the UI in the first place.
On a cold boot, where Config() just loaded the graph (and the current user) from the server, the first call after that load returns immediately — no check, no current-user re-fetch. Later calls, and calls after a background refresh, check as usual.
Caller contract: invoke this before StartupManager.Startup().
OptionalproviderToUse: IMetadataProviderPublic entry point to PostProcessRows for code that runs SQL outside the provider's own read paths but returns entity rows — the transaction groups. Every row a provider hands back must pass through here, so binary columns become base64, datetimes are adjusted and encrypted fields are decrypted no matter which path saved the record.
Raw rows for entityInfo.
The entity the rows belong to.
OptionalcontextUser: UserInfoThe user the rows are processed for (decryption runs as this user).
The processed rows.
ProtectedprocessProcesses query parameters: resolves {{query:"..."}} composition tokens,
then applies Nunjucks template substitution for {{param}} tokens.
Delegates to RenderPipeline.Run for the composition → template pipeline. Paging is handled separately by the caller (InternalRunQuery).
Optionalparameters: Record<string, string>OptionalcontextUser: UserInfoProtected AbstractProjectProjects a save call's result set down to SaveReturnContext.datestampSelectColumnsSQL
when a caller requests SaveReturnMode: 'datestamps_only' at runtime on an entity whose
compiled stored procedure / function returns the full row (returnContext.needsDatestampProjection === true).
EXEC into DECLARE @ResultTable TABLE (...) and emits
SELECT [ID], [__mj_CreatedAt], [__mj_UpdatedAt] FROM @ResultTable;.SELECT "ID", "__mj_CreatedAt", "__mj_UpdatedAt" FROM schema."fn"(...).ProtectedPropagatePropagates record change entries to sibling branches of an IS-A hierarchy. Called after saving an entity with AllowMultipleSubtypes (overlapping subtypes). Collects SQL from BuildSiblingRecordChangeSQL for each sibling and executes as a batch.
The parent entity info
The changes JSON and description
The primary key value
The acting user ID
The child entity that initiated the save (to skip)
OptionalextraExecOptions: Record<string, unknown>Optional provider-specific execution options (e.g. connectionSource for SQL Server transactions)
Optionalsource: RecordChangeSourceThe source discriminator for the record change row ('Internal' or 'Clone')
OptionalchangeContext: string | nullOptional serialized JSON provenance for clone operations
ProtectedQueryReads the first row a query's CacheValidationSQL returned into a cache status. No row means an empty result: zero rows, no timestamp.
Both spellings of the count are accepted. The shipped CacheValidationSQL field description
tells authors to return TotalRows, while older queries return RowCount. Reading only one
of them yields Number(undefined) = NaN for the other, NaN !== NaN makes the slot never
validate, and NaN is then written back into the client's cache entry where ?? cannot clear
it: a silent, permanent degradation to always-refetch, with no error. Platform providers that
batch the validation SQL read their rows through this too.
ProtectedQueryThe definition signature of query on this provider's platform: what a RunQuery cache slot
of the query must have been written under to be served. Computed by QueryEngine from the
same metadata this provider executes. Undefined when the engine cannot compute it, which
disables the definition check for that read rather than failing it.
AbstractQuoteQuotes a database identifier (table, column, view name) using the provider's dialect convention. SQL Server uses [brackets], PostgreSQL uses "double quotes".
The identifier to quote
AbstractQuoteQuotes a schema-qualified object name (e.g. schema.viewName) using the provider's dialect convention. SQL Server uses [schema].[view], PostgreSQL uses "schema"."view".
The schema name
The object name (table, view, etc.)
ProtectedRebuildRebuilds the O(1) entity lookup Maps from the current AllEntities array. Called automatically from UpdateLocalMetadata().
ProtectedRecordThe fields a record's display name is built from, in order: every IsNameField field by
Sequence (so FirstName + LastName gives "Elizabeth Rodriguez"), else the entity's single
NameField. Empty when the entity has neither.
ProtectedRecordThe primary key field each pair of a record-name key names, in key order. Names match case-insensitively. Null, after logging, when the key is empty or names a field that is not one of the entity's primary keys.
Refreshes all metadata from the server. Respects the AllowRefresh flag from subclasses.
OptionalproviderToUse: IMetadataProviderTrue if refresh was initiated or allowed
ProtectedRefreshHow this provider refreshes after a metadata member entity changed. The base behavior is a hard Refresh — correct for database providers, where the process that PERFORMED the write is the one refreshing, so re-checking staleness first is wasted work and the re-read must bypass every cache layer. Transport providers (GraphQL) override this with a staleness check so a browser doesn't re-pull the full metadata graph for a change the server-side timestamp comparison can disconfirm.
Refreshes the CurrentUser from the server and updates local metadata in place. Useful on warm boot or when user roles/permissions change dynamically without entity schema changes.
Refreshes metadata only if needed based on timestamp comparison. Combines check and refresh into a single operation.
OptionalproviderToUse: IMetadataProviderOptionalbypassMinCheckInterval: booleanPassed through to CheckToSeeIfRefreshNeeded; event-driven callers set true so the throttle cannot eat a check they have positive evidence for.
True if refresh was successful or not needed
Refreshes the remote metadata timestamps from the server. Updates the internal cache of remote timestamps.
OptionalproviderToUse: IMetadataProviderTrue if timestamps were successfully refreshed
The same hard reload as Refresh, but the metadata dataset's reads join the ambient transaction for the duration of the call (MJ#4836).
A timer-driven refresh must stay on the pool so it cannot land beside COMMIT (#4514).
The caller that owns the transaction — mj sync push, between its own writes — is the
one case that has to see its uncommitted metadata rows, and it waits for this call, so
the read cannot race the commit.
OptionalproviderToUse: IMetadataProviderProtectedregisterRecords which entities compose this provider's metadata, from the loaded MJ_Metadata dataset result, and registers this instance with the static event fan-out so writes to any of them schedule a debounced metadata refresh. Called from GetAllMetadata on every successful load, so the set tracks the dataset definition as it changes.
Drop this instance's transaction handle. Must not close the shared pool.
ProtectedReloadReloads this provider's metadata. Single-flight: at most one reload runs at a time, and a request arriving mid-reload queues exactly one follow-up pass.
re-read the remote timestamps first and bypass every cache layer
OptionalproviderToUse: IMetadataProviderOptionalscopedFirst: booleantry the scoped reload of the saved-query family (see PartialRefreshMetadataItemCodes) before the full one. Defaults to true unless hardRefresh: an explicit Refresh() is always a full reload.
Removes all cached metadata from local storage. Clears both timestamps and metadata collections.
ProtectedRenderOptional replay form of a CREATE for the SQL log only (never executed). Dialects that record saves for migration replay (SQL Server's Metadata_Sync migrations) override this to emit a create-or-update guarded on the primary key, so replaying the recording on a database that already holds the row converges instead of failing (MemberJunction/MJ#4503). Default: no replay form, the plain save SQL is logged.
ProtectedRenderOptional replay form of an UPDATE for the SQL log only (never executed). Receives just the
fields this save CHANGED. A recording that carried the whole row would, on replay, overwrite
every column of the target row with the recording database's values — including settings a
deploying application tuned on purpose (an MJ metadata sync that only flipped
AllowUserSearchAPI on 'MJ: Content Items' also reset that consumer's
TrustServerCacheCompletely). Dialects whose update procs keep unpassed columns override
this. Default: no replay form, the plain save SQL is logged.
Protected AbstractRenderRenders the dialect-specific parameter binding for a save call.
Returns a discriminated SaveCallBinding the orchestrator treats as
opaque — the same provider's WrapSaveCallForResult /
WrapSaveCallWithRecordChange pattern-match the variant.
fieldValues is the post-coercion, post-encryption map from
GenerateSaveSQL. Iteration order matters for PostgreSQL positional
binding — Map iteration follows insertion order, which the
orchestrator preserves.
ProtectedRenderRenders the WHERE clause for a saved view, replacing template variables like {%UserView "viewId"%} with subquery SQL. Handles nested/recursive templates with circular reference detection.
Uses QuoteIdentifier/QuoteSchemaAndView for dialect-neutral SQL generation.
Drop a dead physical handle and reset depth/stack. Safe to call when already at depth 0. Does not go through RollbackTransaction (that would re-enter the mutex).
ProtectedresolveResolves a category path string to a QueryCategoryInfo ID.
ProtectedresolveAsync status-aware wrapper around GetEffectiveBaseView for the BASE-VIEW materialization case.
GetEffectiveBaseView name-swaps unconditionally, which (a) serves a Building/DriftHold/Disabled
snapshot — defeating "flag and hold" (§13/§17.2), since a base-view materialization reuses the source
entity and thus has no read-permission revoke to fall back on the way a minted query entity does — and
(b) hard-errors on a Materialized read of a non-materialized entity (missing view). This gates the swap
on an ACTIVE MaterializedResult and otherwise returns the LIVE base view (graceful fallback). The status
read uses BypassCache because DriftHold/Disabled are written by CodeGen via direct SQL (no BaseEntity
cache-invalidation event), so a cached status could otherwise be stale. Non-materialized reads and minted
query virtual entities (BaseView already materialized_vw…) skip the lookup entirely (no extra query).
OptionalcontextUser: UserInfoProtectedresolveResolves the cache TTL (in ms) for a RunView's entity, or undefined for MJ-DB entities
(which use event-based cache invalidation, not TTL). External-data-source entities MUST be
time-bounded — their remote data changes never emit BaseEntity events — so this returns the
data source's DefaultCacheTTLSeconds (via the external router) in ms, or 0 to signal "do
not cache" (TTL disabled on the source, or no router available to resolve one).
OptionalcontextUser: UserInfoProtectedResolveDecides whether a RunView returns the entity's binary fields, and records the decision on
params.IncludeBinaryFields so every later step (field widening, cache fingerprint,
transport) sees one answer.
Binary fields are returned when the caller sets IncludeBinaryFields, or names a binary
field in Fields — asking for a column by name is asking for it. Otherwise they are left
out, which keeps result sets, engine caches and the RunView caches free of large values
nobody reads. Must run before Fields is widened, since widening replaces the caller's list.
The view parameters; IncludeBinaryFields is set to true when applicable.
The entity being queried.
True when binary fields will be returned.
ProtectedResolveThe value to write into a dependent record's link column so it points at the surviving record of a merge.
The two kinds of link store their target differently, and writing the wrong one is silent: a
hard foreign key holds the bare primary key value, while a polymorphic RecordID column holds
the canonical ID|<guid> encoding produced by CompositeKey.ToRecordID. Writing a bare
value into a RecordID column leaves a pointer that resolves to nothing and re-introduces
the second encoding this work exists to eliminate - so it would corrupt exactly the rows the
merge was supposed to preserve.
Separated from MergeRecords so the choice is directly testable, since nothing about the
resulting row makes the mistake visible after the fact.
ProtectedResolveResolves any PlatformSQL values in RunViewParams to plain strings for the active platform. Mutates the params object in place so downstream InternalRunView implementations always receive plain string values for ExtraFilter and OrderBy.
ProtectedresolveResolves a query from RunQueryParams (by ID or Name+CategoryPath). Uses QueryEngine as the single source of truth for query metadata.
ProtectedResolveB45 override of the RunQuery cache-serve seam — enforce the SAME authorization on a
cache HIT that the miss path enforces. Resolution uses resolveQuery() (QueryEngine as
the single source of truth, with CategoryPath disambiguation of same-named queries — the
B46 pairing), and authorization is the FULL MJQueryEntityExtended.UserCanRun (roles +
entity CanRead + recursive composition) — exactly the check ValidateQueryForExecution
applies before a real execution. The base's roles-only check would let a user read cached
rows of a query whose underlying entities they cannot read.
Non-throwing by contract: any resolution error degrades to resolvable: false, which the
gate treats as "fall through to authorized execution" — one extra query run, never an
unauthorized serve and never a crashed cache path.
Optionaluser: UserInfoProtectedResolveThe entity a RunView targets, resolved with NO I/O — for gates that run on the result path,
where an async RunView.GetEntityNameFromRunViewParams (which issues a User Views query for
a bare ViewID) would be a query per result set.
Covers the two shapes every caller in this repository uses: an explicit EntityName, and a
loaded ViewEntity. A request carrying only ViewID/ViewName with neither is not
resolvable here and returns undefined — the same shape RunView's own row normalization
already declines to handle for the same reason, and the same one
cacheDeniedForViewOnlyRequest exists to fail closed for on the cache path.
ProtectedResolveResolves the compile-time vs. effective runtime SaveReturnContext for a save operation.
Centralizes the SaveReturnMode decision logic in GenericDatabaseProvider so dialect providers
(SQLServerDataProvider, PostgreSQLDataProvider, and future providers) never duplicate the
rules for:
spReturnMode: the return shape compiled into the stored procedure / function (EntityInfo.SaveReturnMode).effectiveReturnMode: the return shape requested for this call (options?.SaveReturnMode ?? spReturnMode).needsDatestampProjection: true when the compiled routine returns the full row (spReturnMode === 'full')
but the caller requested 'datestamps_only' at runtime, requiring the provider to project the result down
to datestampSelectColumnsSQL before wire transmission.Optionaloptions: EntitySaveOptionsResolves a PlatformSQL value to the appropriate SQL string for this provider's platform. If the value is a plain string, it is returned as-is (backward compatible). If the value is a PlatformSQL object, the platform-specific variant is used if available, otherwise the default variant is used.
Protected AbstractRollbackOutermost ROLLBACK. Release the driver object in finally even if rollback rejects.
Must not call public Begin/Commit/Rollback.
Rolls back the current transaction.
Routes a typed Remote Operation by key to its implementation (see IRemoteOperationProvider).
This is the public power-tool transport seam. Prefer the typed
BaseRemotableOperation.Execute() entry point in application code — RouteOperation is the
stringly-typed escape hatch for dynamic dispatch / generic tooling, not for building
significant systems. Server providers override InternalRouteOperation to execute the
operation in-process; the client (GraphQL) provider overrides it to marshal over the wire.
Only registered, active (and, when AI-authored, approved) operations are routable, and every
call is authorized on the server side.
Stable registry key of the operation (e.g. RecordProcess.RunNow).
The operation's typed input payload.
Optionaloptions: RemoteOpInvokeOptionsOptional invocation options (mode, progress callback, user, provider, fingerprint).
The operation result; never throws for logical failures — check Success/ErrorMessage.
Queue task until the outermost commit of the caller's transaction, or start it now when
there is none. See DatabaseProviderBase.RunAfterCommit for the contract. Work
registered by a caller outside the open transaction's scope (#4786) is not tied to that
transaction: it starts now, detached, on the pool, whether or not that transaction is open.
token, the task follows the transaction the token was captured in: it runs once
that transaction has committed; it is dropped if that transaction, or any savepoint the
token was captured in, rolled back — including a savepoint rolled back inside a transaction
that went on to commit — or if the token matches nothing this provider knows; otherwise it
is queued at the deepest captured frame that is still open. A token captured outside any
transaction always runs.BeginTransaction in the task's synchronous prefix stays in the task's own scope.
If the task begins a transaction while another is open on this instance, it joins that one as
a savepoint (logged) — a task that needs its own transaction belongs on an independent instance.Optionaltoken: PostCommitTokenProtectedRunAll-count_only batch: every view runs the normal per-view path (so every
security gate applies per view), but their COUNT queries are executed as ONE
UNION ALL statement — one database round trip for, e.g., every related-section
badge on a form. A view that fails keeps its own Success:false result; a connection
failure is thrown, as on the per-view path.
OptionalcontextUser: UserInfoProtectedrunRuns a differential query and returns only changes since the client's cached state. Includes updated/created rows and deleted record IDs. Falls back to full query if hidden deletes are detected.
OptionalcontextUser: UserInfoProtectedrunExecutes an external-data-source query with TTL-based result caching keyed off the data source's DefaultCacheTTLSeconds. External queries can't be event-invalidated (their data lives on a remote system), so a time-bounded cache is the read-cost mitigation the plan calls for — notably for warehouses like Snowflake. The data source's TTL is the source of truth (a TTL of <= 0 disables caching for the source); field-drift warnings still apply to freshly-fetched rows. Ad-hoc SQL (params.SQL) is never cached.
OptionalcontextUser: UserInfoProtectedrunRuns a full query and stores the result in the server's LocalCacheManager. Used by RunViewsWithCacheCheck to populate the server cache for future requests.
OptionalcontextUser: UserInfoProtectedrunRuns a full view query and returns results with cache metadata.
OptionalcontextUser: UserInfoProtectedrunRuns a full query and returns results with cache metadata.
OptionalcontextUser: UserInfoOptionalqueryId: stringOptionalvalidationStamp: { maxUpdatedAt?: string; rowCount?: number }ProtectedRunAwait one post-commit task, logging (never rethrowing) its failure. Shared by the immediate path of RunAfterCommit and by subclasses that drain a queue after commit.
ProtectedRunRuns all registered PostRunView hooks against a single result, returning the (possibly mutated) result.
Protected (not private) for the same reason as RunPreRunViewHooks above: a subclass pipeline that returns view rows WITHOUT passing through PostRunView/PostRunViews — e.g. the RunViewsWithCacheCheck smart-cache path — MUST apply these hooks to the rows it returns. PostRunView is the OUTPUT half of the enforcement seam (data masking / audit); a path that skips it returns rows the hooked paths would have masked.
OptionalcontextUser: UserInfoProtectedRunRuns all registered PreRunView hooks against a single RunViewParams, returning the (possibly mutated) params.
Protected (not private) on purpose: any subclass pipeline that executes view queries WITHOUT passing through PreRunView/PreRunViews — e.g. the RunViewsWithCacheCheck smart-cache path in GenericDatabaseProvider — MUST apply these hooks itself. Hooks are an enforcement seam (tenant scoping middleware injects filters here); a query path that skips them silently returns rows the hooked paths would have filtered out.
OptionalcontextUser: UserInfoRuns multiple queries based on the provided parameters. This method orchestrates the full execution flow for batch query operations.
Array of query parameters
OptionalcontextUser: UserInfoOptional user context for permissions (required server-side)
Array of query results
Smart cache validation for batch RunQueries. For each query, if cacheStatus is provided, checks CacheValidationSQL to determine staleness. Returns 'current' if cache is valid, 'stale' with fresh data, or 'no_validation' if the query has no CacheValidationSQL configured.
OptionalcontextUser: UserInfoOptionalcontextUser: UserInfoRuns a view based on the provided parameters. This method orchestrates the full execution flow: pre-processing, cache check, internal execution, post-processing, and cache storage.
The view parameters
OptionalcontextUser: UserInfoOptional user context for permissions (required server-side)
The view results
ProtectedrunSingle source of truth for whether a RunView call participates in the local cache (both READ and WRITE). Pre/Post hooks for the singular and batch paths must all use this predicate — historically each site recomputed it inline and they drifted (PostRunViews wrote BypassCache results into the cache, poisoning the Fields-agnostic superset slot with narrow rows).
Ineligible:
BypassCache — caller explicitly wants true DB state, no cache interactionAfterKey — keyset pages are single-use AND the fingerprint doesn't include
the seek key, so caching a page would poison the entity+filter slotResultType 'count_only' — returns no rows; caching its empty Results under
a fingerprint that excludes ResultType would poison row queriesDataSource: 'Materialized' — the snapshot is rebuilt OUT-OF-BAND by the scheduled refresh
(direct SQL, no BaseEntity save), so the entity's normal event-driven cache invalidation never
fires for it; a cached materialized result would be served indefinitely stale after a refresh.
Bypass caching entirely for materialized reads. (The ds:materialized fingerprint segment still
keeps the short-lived dedup/linger layer from cross-serving Live vs Materialized in-flight reads.)ProtectedrunWrite-side eligibility for the smart-cache-check (stamped) path. On the trusting SERVER this is exactly runViewCacheEligible — the server cache is kept fresh by BaseEntity events, so a server-cache-disallowed entity must never be slotted. On a CLIENT the slot is instead written with a maxUpdatedAt stamp and DB-revalidated per request, so the server Trust/event gate does NOT apply: a server-cache-disallowed entity (Trust=0 'MJ: Audit Logs', Record Changes, other caching-disabled) is still safely client-cacheable when stamped — folding runViewCacheEligible's server gate onto this path regressed that (integration check client-cache.C12). Materialized reads stay excluded on both (the out-of-band snapshot swap the stamp can't observe).
ProtectedRunThe body of InternalRunView. countExecutor, when supplied, replaces the
direct execution of a count_only view's COUNT query — InternalRunViews uses
it to coalesce an all-count_only batch into one statement (see countOnlyBatch.ts).
Everything before that step — permissions, RLS, filter screening — is unchanged.
OptionalcontextUser: UserInfoOptionalcountExecutor: (countSQL: string) => Promise<CountOnlyRow[]>Runs multiple views based on the provided parameters. Wraps the execution pipeline with request deduplication and a linger window so that concurrent (and near-sequential) identical calls share a single server round-trip. Every caller receives a shallow-copied Results array to protect against cross-caller mutations (push/sort/splice).
Array of view parameters
OptionalcontextUser: UserInfoOptional user context for permissions (required server-side)
Array of view results (shallow-copied Results per caller)
Smart cache validation for batch RunViews. For each view request, if cacheStatus is provided, checks if the cache is current by comparing MAX(__mj_UpdatedAt) and COUNT(*) with client's values. Returns 'current' if cache is valid (no data), 'stale' with fresh data, or 'differential' with only changed rows for entities that track record changes.
OptionalcontextUser: UserInfoSaves an entity record — the full orchestration flow shared by all DB providers.
Saves current metadata to local storage for caching. Serializes both timestamps and full metadata collections.
ProtectedSavepointProtectedscheduleSchedules this provider's metadata refresh after a write to a metadata member entity. Delay and re-arm semantics come from MetadataMemberRefreshDelayMs and MetadataMemberRefreshRearmsOnNewEvents — debounce on the server, long jittered coalescing window on clients. The refresh targets THIS instance — the provider that loaded the dataset owns the metadata built from it; short-lived per-request providers never load the dataset (they adopt the global's metadata as a shared shell), so on the server only the process-global provider ever gets here.
SECURITY — the full client-clause screen (ClientClauseScreen.AssertClauseUsesEntityBaseViews)
against this provider's entities and dialect, with contextUser as the acting user. With
entityInfo, the clause is screened in the form this provider runs a client clause for that
entity (TransformExternalSQLClause). Server code that hands client SQL text to RunView
outside the API entry points calls this: ad-hoc list sources (ListOperations) and the
saved-view save gate (MJUserViewEntityServer).
OptionalcontextUser: UserInfoOptionalentityInfo: EntityInfoProtectedScreenSECURITY — screens the ORDER BY a saved view builds from its stored SortState / GridState with the full client-clause screen. Any user who can save a view can write those.
ProtectedScreenSECURITY — screens a saved view's rendered WhereClause with the full client-clause screen.
Any user who can save a view can write its WhereClause, directly or through FilterState or a
Smart Filter. Exempt: a CustomWhereClause view loaded as an entity, whose clause only an Owner
can set (MJUserViewEntityServer). The flag on any other object, such as one built from a
request body, is not trusted. The exemption tests instanceof BaseEntity, so a second copy of
@memberjunction/core in the process makes loaded custom views fail closed, not open.
Batch form of SearchEntity. Fans the input list out to N
independent SearchEntity calls via Promise.all; result arrays come
back aligned by input order (result[i] holds the matches for params[i]).
On the server side, the per-entity passes are independent — running them
concurrently is a real wall-clock win when the caller wants results from
multiple entities. On the client side, GraphQLDataProvider overrides
this method to pack the whole batch into a single GraphQL round-trip
instead of issuing N parallel HTTP requests.
See IMetadataProvider.SearchEntities for the contract.
ProtectedsearchServer-side semantic ranking pass for ProviderBase.SearchEntity (and, by extension, the batched ProviderBase.SearchEntities).
The query embedding MUST be generated with the same model that produced
the indexed vectors — otherwise cosine scores compare apples to oranges
and rankings are garbage. We look up the EntityDocument's AIModelID via
AIEngine.Models to recover the driver class / APIName and call
EmbedText(model, text) directly. If the EntityDocument does not specify
a model (or the model isn't loaded) we fall back to
EmbedTextLocal (highest-power local model) only as a last resort.
Vector ranking runs against the in-process SimpleVectorServiceProvider,
which rehydrates the vector pool for entityDocumentId from
MJ: Entity Record Documents.VectorJSON rows.
Failures (no embedding model available, vector index miss) degrade to an empty result set so hybrid mode can still surface lexical matches.
Ranked search over one entity's records. See IMetadataProvider.SearchEntity for the contract and how this differs from EntityByName / FullTextSearch.
Implementation overview (concrete on ProviderBase, used as-is by every
server-side provider; GraphQLDataProvider overrides to proxy via GQL):
params.options.entityDocumentId
override or by looking up the active Search-category doc for the entity).IncludeInUserSearchAPI fields) and the semantic
pass (searchEntitiesSemanticPass, the protected template method
each concrete server provider implements).ComputeRRF() with optional per-list weights.Offers a record's name for later synchronous retrieval. BaseEntity calls it after every
Load(), LoadFromData() and Save(). Ignored here; see GetCachedRecordName.
Replaces the active local storage provider at runtime.
Use this to swap from the default in-memory provider to a Redis-backed provider (or any other ILocalStorageProvider) after reading application configuration.
The new storage provider to use for all caching operations.
import { RedisLocalStorageProvider } from '@memberjunction/redis-provider';
// During server startup, after config is loaded:
const redis = new RedisLocalStorageProvider({
url: process.env.REDIS_URL,
defaultTTLSeconds: 300
});
(Metadata.Provider as GenericDatabaseProvider).SetLocalStorageProvider(redis);
Creates or deletes a user favorite record for the specified entity record. Uses GetEntityObject and BaseEntity CRUD methods (no entity-specific type imports needed).
ProtectedshouldOptionalcontextUser: UserInfoProtectedShouldDialect-agnostic predicate: should we write a RecordChange entry for this entity? Excludes the Record Changes entity itself to prevent recursion. Provider implementations should call this before invoking BuildRecordChangePayload or constructing dialect SQL.
ProtectedStartCreates the initial merge log record at the start of a merge operation. Uses BaseEntity with .Set() calls (no typed entity subclass imports available in MJCore).
OptionalcontextUser: UserInfoCompares this provider's metadata with the database — but only for metadata whose entities have declared that their rows can change without firing an event.
Metadata staleness is otherwise event-driven: a BaseEntity write to one of the entities the
metadata is built from schedules a refresh, another process saving its snapshot publishes a
notice, and a tool that changed the database clears the shared cache. None of those fire for a
change made with raw SQL, so metadata edited directly in the database is never noticed by a
running process. This is the backstop for that case.
It is not an unconditional poll. Entity.TrustServerCacheCompletely = true
(the default) states that every mutation flows through BaseEntity, which the event paths
already hear; polling such an entity cannot discover anything, and a recurring query prevents a
serverless database from pausing. So the check reads the database only when at least one
metadata member entity declares false — typically because an operator writes that table
directly. On a stock installation no metadata entity declares it, and this costs one in-memory
pass over the membership set.
OptionalproviderToUse: IMetadataProviderProvider to compare against, for multi-provider clients.
Which entities declared drift, whether the database was consulted, and whether metadata was actually reloaded.
ProtectedTransformTransforms a user-provided SQL clause (ExtraFilter, OrderBy, etc.) for platform compatibility. PostgreSQL overrides to quote mixed-case identifiers and convert bracket notation. Default: returns the clause unchanged.
ProtectedTransformTransforms the result set from simple objects to entity objects if needed.
The RunViewParams used for the request
The RunViewResult returned from the request
OptionalcontextUser: UserInfoThe user context for permissions
ProtectedTrimTruncates a string value to a maximum length, appending trailing characters if truncated.
ProtectedtryPhase 2 (plan §5): resolve a materialized read plan for a query IF the caller opted into
DataSource:'Materialized' AND the query has a fresh, Active RowFilterBroad materialization whose
persisted spec fully covers the query's parameters. Returns null on ANY uncertainty → the caller runs
the live query (serving live is always correct — this is a transparent optimization, never a
correctness dependency).
OptionalcontextUser: UserInfoProtectedTryAttempts to fully decompose an RLS WHERE clause into the lower-cased column names it
references. Returns the columns ONLY when the whole clause is a conjunction of simple
column <op> literal terms — anything else (OR, parentheses beyond term grouping,
functions, subqueries, arithmetic, unrecognized syntax) returns null, meaning "not
fully understood, run the post-image check". Partial extraction is never returned:
a missed column reference here would silently skip a required authorization check.
ProtectedTryReloads only the saved-query family of MJ_Metadata and swaps it into the current metadata,
keeping every other array — and every object in them — as is. The dataset definition is
reused verbatim (same columns, where clauses and post-processing): every other item is
requested with a 1=0 filter, so it reads nothing. Always bypasses cache layers — the read
is small, and a cached answer is exactly what a staleness check cannot trust.
OptionalproviderToUse: IMetadataProvidertrue when the scoped reload was applied; false when it does not apply or any part of it failed — the caller then runs the full reload, so a failure never leaves stale queries.
ProtectedUpdateUpdates the local metadata cache with new data.
The new metadata to store locally
Returns true when CRUD sproc generation and call-construction for the
given entity + sproc verb should use a single JSON-arg shape (instead of
typed args + _Clear companions).
Convenience wrapper around the pure useJsonArgShape helper, applying
this provider's ProcedureParamLimit. CodeGen and runtime call-sites
can invoke this via the provider instance to keep sproc emit and sproc
invocation in lockstep.
ProtecteduserTrue when a field that will ACTUALLY PARTICIPATE in this search carries a
UserSearchParamFormatAPI. Such a format is admin-authored and may place {0} outside
quotes, so for those entities the search term is not guaranteed to land inside a literal
and still needs the SQL-fragment denylist (#4392).
deniedFields must be the same field-level-security exclusion set the predicate loop
applies. A field the caller may not read is skipped there, so its format never reaches the
SQL — screening the term on its behalf would refuse ordinary searches ("Union Pacific") for
exactly the users with the LEAST access. Participation, not mere configuration, is what
makes the denylist necessary.
OptionaldeniedFields: Set<string>ProtectedValidateValidates the result of a delete SQL execution by checking that the returned primary keys match the entity being deleted. SQL Server overrides to handle the multi-result-set case (CASCADE deletes).
ProtectedValidateValidates that a query can be executed by the given user. Checks both permissions and approval status. Permission failures throw an error. Non-approved status emits a console warning but allows execution to proceed, enabling query testing before formal approval.
The resolved MJQueryEntityExtended to validate
OptionalcontextUser: UserInfoThe user attempting to execute the query
ProtectedValidateValidates a user-provided SQL clause (WHERE, ORDER BY, etc.) to prevent SQL injection.
The clause is read with this provider's SQL dialect. It is rejected when it contains a
comment, text that cannot be read the way the database reads it, a forbidden keyword
(INSERT, UPDATE, DELETE, EXEC, DROP, UNION, etc.), a semicolon, or an xp_ name.
String literals and quoted identifiers are not checked for keywords.
The SQL clause to validate
true if the clause is safe, false if it contains forbidden patterns
ProtectedwarnProtectedWithProtected AbstractWrapWraps the bare SP-call binding with the dialect's result-capture
pattern. SQL Server emits DECLARE @ResultTable + INSERT INTO @ResultTable EXEC + SELECT * FROM @ResultTable;
PostgreSQL emits the bare SELECT * FROM schema."fn"(...).
Optionaloptions: EntitySaveOptionsOptionalreturnContext: SaveReturnContextProtected AbstractWrapWraps an already-result-captured save SQL with the dialect's
record-change emission. SQL Server inlines EXEC spCreateRecordChange_Internal
after the result table; PostgreSQL emits a CTE chain with
INSERT INTO "RecordChange" ... RETURNING.
Orchestrator only calls this when ShouldTrackRecordChanges is true
and BuildRecordChangePayload returned a non-null payload.
Optionaloptions: EntitySaveOptionsOptionalreturnContext: SaveReturnContextProtected StaticarrayConverts an ArrayBuffer to a base64-encoded string. Used for compressed metadata storage/retrieval.
Protected Staticbase64Converts a base64-encoded string to an ArrayBuffer. Used for compressed metadata storage/retrieval.
StaticbuildOptionalparamTypes?: Record<string, string>Declared parameter type per paramName (from MJ: Query Parameters). Drives type-faithful binding so a scalar value matches the live path's typed literal instead of a raw string the DB implicitly coerces.
StaticFieldThe single wording for "you cannot use this field," modeled on SQL Server's posture of never disclosing whether an object is missing or merely inaccessible.
Naming the field is safe — the caller supplied it, so it tells them nothing they did not already know. Naming the REASON is not: confirming "this field exists and is restricted" turns any predicate into an oracle for probing which columns a deployment considers sensitive. The ambiguity also keeps this message correct after #3485 tiers metadata and restricted fields stop shipping to clients at all, at which point "does not exist" becomes literally true from the client's vantage point.
StaticIsStatic convenience: checks all platforms' default-value functions. Prefer the instance method when you have a provider reference.
StaticIsStatic convenience: checks all platforms' UUID generation functions. Prefer the instance method when you have a provider reference.
StaticLogStatic method to log SQL statements from external sources like transaction groups. Gets the current provider instance from Metadata.Provider and delegates to the instance _logSqlStatement method.
The SQL query being executed
Optionalparameters: unknownParameters for the query
Optionaldescription: stringOptional description for this operation
Whether this is a data mutation operation
OptionalsimpleSQLFallback: stringOptional simple SQL to use for loggers with logRecordChangeMetadata=false
OptionalcontextUser: UserInfoOptional user context for session filtering
Protected StaticNamesTrue when fieldNames names at least one of the entity's binary fields (case-insensitive,
whitespace-trimmed).
The entity being queried.
Field names as a caller supplied them, e.g. RunViewParams.Fields.
StaticqueryInternalTrue if sql's top-level SELECT carries an ORDER BY. Used to refuse a materialized RowFilterBroad read:
buildMaterializedReadQuery emits no ORDER BY and the snapshot is built with the source's top-level
ORDER BY stripped, so an ordered query must be served LIVE (where its ordering — and therefore its
pagination under StartRow/MaxRows — is preserved) rather than from the unordered snapshot. Parse failure or
an un-reasoned statement shape returns true (refuse-to-live: treat unknown as ordered rather than risk
serving mis-ordered pages). Mirrors MaterializationRefresher.stripTopLevelOrderBy's AST detection.
Materialized-read ordering-fidelity gate — NOT part of this package's supported public API;
static only so it can be unit-tested. Do not call from outside @memberjunction/generic-database-provider.
StaticSaveFirst SaveCallVariableHashLength hex of sha1(${schema}.${table}|${pk values}).
Dialect-agnostic identity of a save call so sqlLogging recaptures of an unchanged tree
are byte-identical. A random uuidv4 slice (previously only in
SQLServerDataProvider.RenderSaveCallBinding) made every MetadataSync recapture a 250 MB
diff and, inside a batched TransactionGroup, collided under the birthday paradox at
~120k variables (loom #12 WP3 / F-D).
sha1 is an identity hash here, not a security primitive: it only has to be stable and
well distributed. Key values are normalized first so the same record hashes the same
wherever its key came from: UUID-shaped strings are lower-cased (SQL Server returns
upper-case, PostgreSQL and hand-authored JSON are usually lower-case), Dates use
ISO-8601, null/undefined are empty. A create with no client-side PK therefore hashes
schema.table|, a per-table constant, and inside a group the _n ordinal is what
tells those inserts apart.
Protected StaticUnionReturns the caller's requested fields (lowercased) unioned with the entity's
primary key field names. Platform contract: when Fields is explicitly
specified, results ALWAYS include the primary key(s) — the direct SQL path has
always done this, differential smart-cache merges require it, and entity
linking in UIs depends on it. Applying the same union at every projection site
keeps result shapes identical across cached, non-cached, and smart-cache paths.
A provider that screens client SQL clauses with ClientClauseScreen against its own entities and dialect.
GenericDatabaseProvideris one; server code that hands client SQL text to RunView outside the API entry points asks its provider through this.