the entities the provider knows
the user CurrentUser returns
Protected_Result from PreRunQueries hook containing cache status for batch operations
Protected_Result from PreRunQuery hook containing cache status and optional cached result
Protected_Result from PreRunView hook containing cache status and optional cached result
OptionalcachedResult?: RunViewResultOptionalcallerRequestedFields?: string[] | nullThe caller's original Fields list (lowercased), captured before PreRunView widened params.Fields to all entity fields for cache-superset storage. Non-null ONLY when that widening actually happened — PostRunView uses it to project cache-miss DB results back down to the requested shape.
Optionalfingerprint?: stringOptionaltelemetryEventId?: stringProtected_Result from PreRunViews hook containing cache status for batch operations
OptionalcachedResults?: RunViewResult[]OptionalcacheStatusMap?: Map<OptionalcallerFieldsMap?: Map<number, string[]>Per-param-index caller Fields lists (lowercased), captured before PreRunViews widened params.Fields to all entity fields for cache-superset storage. An index is present ONLY when that widening actually happened — PostRunViews uses it to project cache-miss DB results back down to the requested shape.
OptionalfingerprintMap?: Map<number, string>Per-param-index cache fingerprints computed during PreRunViews — carried forward so PostRunViews doesn't recompute the RLS where-clause and fingerprint string for every batch item.
OptionalpreResolvedCache?: Map<string, CachedRunViewResult | null>Pre-resolved local cache entries keyed by client cache fingerprint, populated during prepareSmartCacheCheckParams so executeSmartCacheCheck can reuse the already-parsed results for 'current' items without a second IndexedDB read.
OptionalsmartCacheCheckParams?: RunViewWithCacheCheckParams[]When CacheLocal is enabled, contains the cache check params to send to server
OptionaltelemetryEventId?: stringOptionaluncachedParams?: RunViewParams[]OptionaluseSmartCacheCheck?: booleanWhen CacheLocal is enabled, indicates we should use smart cache check
Protected Static_StaticCoalesceWhen enabled, concurrent RunViews calls arriving within the same microtask (or within CoalesceWindowMs) are merged into a single mega-batch before hitting the network. This dramatically reduces the number of HTTP round-trips during startup when multiple engines independently call RunViews in parallel.
Set to 0 to disable coalescing. Default 10ms — enough to capture all engines that fire in the same tick, without adding perceptible delay.
Static ReadonlyDatasetThe cache category every dataset read and write uses.
Every storage provider isolates by category — Redis keys are {prefix}:{category}:{key},
browser localStorage [mj]:[category]:[key], IndexedDB a dedicated object store, the
in-memory providers a nested map — so a read and a write that disagree about the category can
never meet. When the batched warm read in GetAndCacheDatasetByName named 'DatasetCache'
while every write and every other reader landed in default, the warm-serve path missed on
every transport and silently refetched from the server. Naming the category once is what keeps
all six call sites in agreement.
Static ReadonlyDatasetHow long a cached dataset lives.
Datasets have a category to themselves and it carries no category-wide expiry, so without a
per-write TTL every distinct filter set (GetDatasetCacheKey includes the filters) would
leave behind a blob nothing removes. An explicit TTL bounds that growth, and the store
prefers a per-write value over any category or global setting — which is also why giving
datasets their own category did not change how long one lives. Providers that do not
implement expiry ignore it.
Static ReadonlyDatasetHow long the _date key lives — deliberately SHORTER than the blob it vouches for.
It is written second, so with equal lifetimes it would usually be the one to survive, leaving
a freshness claim with nothing behind it: IsDatasetCached answers true and the blob is gone.
Expiring the claim first makes the pair fail in the safe direction — the cache reads as
absent and is refetched.
StaticDedupHow long (ms) a resolved RunViews result stays available for instant replay. Set to 0 to disable the linger window (in-flight dedup still applies). Default 5 000 ms.
StaticMaxSafety cap on the number of linger entries held simultaneously. The linger window is a latency optimization — under extreme churn (more distinct query keys than this resolving within one window) new resolutions skip lingering instead of accumulating result arrays in memory.
StaticMaxHow many consecutive windows a member-change refresh may wait on MetadataMemberRefreshMustWait before it is dropped. At the default 500ms window this is ten seconds — far longer than any transaction a Save or Delete holds.
StaticMetadataCoalescing window, in milliseconds, for metadata refreshes triggered by writes to the entities that compose this provider's metadata. A single administrative action produces a burst (enabling field security writes one permission row per field/role pair, each raising its own event) — one refresh per burst, not one per row. The window also gives an enclosing entity transaction time to COMMIT before the re-read: every event in the burst re-arms the timer, so the refresh runs no earlier than this long after the LAST write.
Static ReadonlyMetadataResult of SweepMetadataAgainstDatabase.
StaticMinMinimum interval (ms) between metadata refresh checks to prevent redundant network calls when Config()/RefreshIfNeeded() fire in quick succession (e.g., multiple engines during startup). The throttle is armed only by a check whose status request succeeded — a check that threw or got no timestamps proved nothing, so the next caller checks again. A caller arriving while a check is in flight is told no refresh is needed, as the in-flight owner acts on the answer (see CheckToSeeIfRefreshNeeded). Does NOT affect forced Refresh() calls. Default: 30 000 ms.
StaticPeerUpper bound, in milliseconds, of the random delay added to HandlePeerMetadataNotice's debounce. A tool that changes the schema removes the shared snapshot, which reaches every server at once; the spread lets the first server reload and republish, so the others adopt its snapshot instead of all querying the database together.
StaticPeerHow long a peer notice may be deferred by later notices before the check runs anyway. The debounce timer is reset by each arrival, so a fleet publishing steadily — exactly what a bulk import looks like — could postpone the check indefinitely.
StaticServerMaximum row count for auto-caching on the server side. When
TrustLocalCacheCompletely is true and a RunView result has no
ExtraFilter, no OrderBy, and the result count is at or below this
threshold, the result is automatically stored in LocalCacheManager
even without an explicit CacheLocal flag.
This captures small reference/lookup tables that are repeatedly queried by multiple clients while avoiding caching large ad-hoc result sets. Invalidation is handled by the standard BaseEntity event-driven upsert (safe because unfiltered caches can be updated in-place).
Set to 0 to disable auto-caching. Default 250.
Gets all explorer navigation items including inactive ones.
Array of all ExplorerNavigationItem objects
Returns the currently loaded local metadata from within the instance
ProtectedAllowDetermines if a refresh is currently allowed or not. Subclasses should return FALSE if they are performing operations that should prevent refreshes. This helps avoid metadata refreshes during critical operations.
Gets all application metadata in the system.
Array of ApplicationInfo objects representing all applications
Gets all audit log types defined for tracking system activities.
Array of AuditLogTypeInfo objects
Gets the flat collection of authorization-role assignments.
Consumed lazily by AuthorizationInfo.Roles — consumers should
prefer accessing roles through AuthorizationInfo.Roles rather than
filtering this array directly.
Array of AuthorizationRoleInfo join-table objects
Gets all authorization definitions in the system.
Array of AuthorizationInfo objects defining permissions
Gets the configuration data that was provided to the provider.
The provider configuration including schema filters
Gets the current user's information including roles and permissions.
UserInfo object for the authenticated user
For providers that have ProviderType==='Database', this property will return an object that represents the underlying database connection. For providers where ProviderType==='Network' this property will throw an exception. The type of object returned is provider-specific (e.g., SQL connection pool).
Gets all entity metadata in the system.
Array of EntityInfo objects representing all entities
Returns the filesystem provider for the current environment. Default implementation returns null (no filesystem access). Server-side providers should override this to return a NodeFileSystemProvider.
This property is implemented by each sub-class of ProviderBase and is intended to return a unique string that identifies the instance of the provider for the connection it is making. For example: for network connections, the URL including a TCP port would be a good connection string, whereas on database connections the database host url/instance/port would be a good connection string. This is used as part of cache keys to ensure different connections don't share cached data.
Why the most recent metadata download (GetAllMetadata) failed; null when it succeeded
or none has run. GetAllMetadata logs and returns undefined on failure so a background refresh
keeps the last good graph; a boot that ends with no metadata reads this to report the cause
(e.g. a user with no roles cannot read MJ: User Roles) instead of a generic "nothing loaded".
Gets the latest metadata timestamps from local cache. Used for comparison with remote timestamps.
Array of locally cached metadata timestamps
Gets the latest metadata timestamps from the remote server. Used to determine if local cache is out of date.
Array of metadata timestamp information
Gets all library definitions in the system.
Array of LibraryInfo objects representing code libraries
ProtectedLocalThis property will return the prefix to use for local storage keys. This is useful if you have multiple instances of a provider running in the same environment and you want to keep their local storage keys separate. The default implementation returns an empty string, but subclasses can override this to return a unique string based on the connection or other distinct identifier.
Gets the local storage provider implementation. Must be implemented by subclasses to provide environment-specific storage.
Local storage provider instance
ProtectedMetadataGets the metadata provider instance. Must be implemented by subclasses to provide access to metadata.
The metadata provider instance
ProtectedMetadataHow long a member-entity write waits before this provider's refresh runs. The base value is the short debounce window — right for the server, where the writer is the refresher and the delay only exists to coalesce a burst and let the enclosing transaction commit. Transport providers override this with a much longer, RANDOMIZED window: every browser receives every write broadcast, so the delay is what turns "N clients each re-pull the metadata graph within the same half-second of any member write" into "each client pays at most one staleness check per window, at a moment no other client shares".
ProtectedMetadataTrue while a member-change refresh must NOT run, e.g. the provider is inside an ambient transaction. The base never waits; a database provider overrides this so a timer-driven refresh cannot land inside a caller's transaction (see GenericDatabaseProvider, #4486).
ProtectedMetadataWhether a member-entity write arriving while the refresh timer is already armed RESTARTS the timer (debounce) or joins the pending window (coalesce/throttle).
The base is a true debounce (true): the server's refresh must run AFTER the last write
of the unit of work, so every event pushes the timer out — a burst costs one refresh, run
once the burst ends. Transport providers return false: with a long window, re-arming
would let steady org-wide write activity postpone the refresh indefinitely (starvation);
joining the armed window guarantees at most one refresh per window regardless of write
rate, which is the whole point of the window.
ProtectedMetadataTrue while RefreshWithinTransaction is running, so metadata-dataset reads join the ambient transaction instead of the pool.
Whether anything could read the metadata snapshot back, and therefore whether saving it is worth the cost.
The snapshot lets a cold process start from a cached copy of the metadata instead of querying
for it, which only works if the store outlives the writer. Where it does not, both halves of
the round trip are pure cost: the save serializes, gzips and base64-encodes the entire
metadata graph, and the load parses it and rebuilds every EntityInfo and EntityFieldInfo
from a copy of objects the heap already holds. On a large tenant that is expensive enough to
exhaust the heap.
A provider that does not declare ILocalStorageProvider.SupportsCrossProcessPersistence is treated as persistent, which fails in the safer direction.
Returns the database platform key for this provider. Override in subclasses to return the appropriate platform. Defaults to 'sqlserver' for backward compatibility.
ProtectedPreProtectedPreProtectedPreOptionalcachedResult?: RunViewResultOptionalcallerRequestedFields?: string[] | nullThe caller's original Fields list (lowercased), captured before PreRunView widened params.Fields to all entity fields for cache-superset storage. Non-null ONLY when that widening actually happened — PostRunView uses it to project cache-miss DB results back down to the requested shape.
Optionalfingerprint?: stringOptionaltelemetryEventId?: stringProtectedPreOptionalcachedResults?: RunViewResult[]OptionalcacheStatusMap?: Map<OptionalcallerFieldsMap?: Map<number, string[]>Per-param-index caller Fields lists (lowercased), captured before PreRunViews widened params.Fields to all entity fields for cache-superset storage. An index is present ONLY when that widening actually happened — PostRunViews uses it to project cache-miss DB results back down to the requested shape.
OptionalfingerprintMap?: Map<number, string>Per-param-index cache fingerprints computed during PreRunViews — carried forward so PostRunViews doesn't recompute the RLS where-clause and fingerprint string for every batch item.
OptionalpreResolvedCache?: Map<string, CachedRunViewResult | null>Pre-resolved local cache entries keyed by client cache fingerprint, populated during prepareSmartCacheCheckParams so executeSmartCacheCheck can reuse the already-parsed results for 'current' items without a second IndexedDB read.
OptionalsmartCacheCheckParams?: RunViewWithCacheCheckParams[]When CacheLocal is enabled, contains the cache check params to send to server
OptionaltelemetryEventId?: stringOptionaluncachedParams?: RunViewParams[]OptionaluseSmartCacheCheck?: booleanWhen CacheLocal is enabled, indicates we should use smart cache check
Returns the provider type for the instance. Identifies whether this is a Database or Network provider.
Gets all security roles defined in the system.
Array of RoleInfo objects representing all roles
Gets all row-level security filters defined in the system.
Array of RowLevelSecurityFilterInfo objects for data access control
ProtectedRunWhat, besides the request itself and the user, decides whether two identical RunViews may share one execution (in flight or lingering). Empty by default. A provider whose callers can see different data for the same query — a database provider whose open transaction belongs to one async scope only (#4786) — returns a value that differs between those callers, so a transaction member never awaits an outsider's pool read (missing its own writes, or on SQL Server waiting on its own row lock) and an outsider never receives uncommitted rows. Read synchronously while the dedup key is built, in the caller's async context.
Whether this provider can execute a multi-record unit of work atomically, in-process.
Defaults to false — the correct answer for every provider that is not talking directly to a
database, most importantly the client-side GraphQLDataProvider. DatabaseProviderBase
overrides this to true and supplies DatabaseProviderBase.BeginEntityTransaction.
BaseEntity reads this to decide whether a multi-node save graph runs locally inside a
transaction or is routed to the server as a single unit of work. Defaulting to false is the
safe direction: a provider that has not opted in never has non-atomic work mistaken for
atomic work.
ProtectedTrustWhen true, cached RunView/RunQuery results are returned immediately on a cache hit without any server-side validation round-trip.
Server-side providers (DatabaseProviderBase and its subclasses) override
this to return true because the cache is kept in perfect sync via
BaseEntity save/delete events and cross-server Redis pub/sub — the DB
validation query is unnecessary overhead.
Client-side providers (e.g. GraphQLDataProvider) keep the default false
so that the lightweight smart cache check (maxUpdatedAt + rowCount) is
still performed against the server before trusting the browser cache.
Gets only active explorer navigation items sorted by sequence. Results are cached for performance.
Array of active ExplorerNavigationItem objects
StaticPartialMJ_Metadata item codes that may be reloaded on their own, without rebuilding the rest of the metadata graph: the saved-query family. Those rows are written continuously by agents (a query is persisted per analysis step), and every write moves the timestamps of these items. Before this list existed, each such write cost a FULL reload — every EntityInfo and EntityFieldInfo rebuilt (hundreds of MB on a large schema) to pick up a few QueryInfo rows; on a busy server that was a fresh half-gigabyte graph per refresh window until the process hit its memory guard.
Membership rule: an item qualifies only when no OTHER metadata class embeds or memoizes its
rows at construction. The query classes reference each other lazily through the provider
(QueryInfo.Fields filters QueryFields on first use), so they can be swapped as a set;
entity metadata cannot (EntityInfo owns its fields, permissions and relationships).
The family is always reloaded together — see PartialRefreshItemCodes.
Intersected with AllMetadataArrays so the list is valid for whatever this build knows.
ProtectedApplyStrips fields the user cannot read from plain-object result rows.
This is the primary read-time control for list results, and it runs on BOTH cache hits and cache misses — the property the rest of the cache design is arranged around. Because it reads live metadata, a permission change takes effect on the next metadata refresh with no result-cache invalidation: the cached full-width superset stays valid and only the projection changes.
It composes with two per-request, never-cached narrowings that avoid pulling columns a
restricted service account has no business holding: the simple-path SELECT-list
intersection, and Load()'s allowed-column SELECT.
NEVER applied to entity_object results. Those become BaseEntity instances whose
fields round-trip through GenerateSaveSQL, which iterates ALL IsSPParameter fields
reading field.Value — not just dirty ones. A stripped field would therefore be written
back as a real NULL on the user's next save: silent data loss. Entity objects keep their
values in server memory exactly as encrypted fields do; the trust boundary is the API
output, which the GraphQL layer enforces separately.
OptionalcontextUser: UserInfoProtectedApplyApplies the PostRunView hook chain to a result that was served from cache, mutating
result in place so the caller's reference reflects the chain's output.
PostRunView is the OUTPUT half of the enforcement seam (data masking / audit). Hooks
receive contextUser, so masking is PER-USER, while the cache slot is shared across
users — there is no correct way to apply masking once at write time on behalf of a
reader who has not arrived yet. A hit that skips the chain therefore returns rows the
miss path would have masked.
This previously appeared to work by accident: PostRunView writes the cache BEFORE running the hooks, so a hook that masked rows in place was writing through into the cached objects — which both made later hits look masked and baked one user's masking decision into a shared slot. Freeze-on-write removes that write-through, which is what makes running the chain here necessary rather than merely tidier.
result in place is safeCache-hit results are FRESH wrapper objects built per hit by PreRunView/PreRunViews —
only .Results points at shared cache state. A hook that returns a replacement (the
required pattern now that rows are frozen) is copied onto that per-hit wrapper, so it
can never write back into the cache.
GetDataHooks is a memoized store read (~30ns), but await-ing the async chain costs
a microtask (~750ns) — comparable to the entire cache lookup this rides on. The
overwhelmingly common case is zero registered hooks, so check first and skip the await.
OptionalcontextUser: UserInfoProtectedApplyStrips or narrows the MJ: Record Changes payload columns a user may not read, using the
denied set of the entity each row is ABOUT rather than of Record Changes itself.
A sibling of ApplyFieldSecurityProjection rather than part of it, because that
method short-circuits on EnableFieldLevelSecurity for the entity named in the RunView
params — which here is MJ: Record Changes, whose flag is off in every default deployment.
Everything about the per-row denied set, the payload treatment, and the fail-closed decision
lives in RecordChangeFieldSecurityProjector; this is only the RunView wiring.
Runs at all four RunView projection points, matching the main projection: both cache-hit paths and both cache-miss paths. The cache-hit path is not optional — it is the exact path the original cross-user leak runs through, an unrestricted user warming a full-width slot that a restricted user then hits.
NEVER applied to entity_object results, for the reason the main projection is exempt plus
one specific to this one. The main reason transfers directly: an entity object's fields
round-trip through GenerateSaveSQL, which reads EVERY IsSPParameter field's value rather
than only dirty ones, so a withheld ChangesDescription would be written back as a real
NULL and a narrowed ChangesJSON as the narrowed payload — destroying audit history instead
of merely hiding it. Record Changes rows genuinely are saved through the entity layer
(replay writes Status/ErrorLog, users write Comments), so this is not hypothetical.
The additional reason is that the exemption cannot become a hole: the GraphQL RunView
resolver coerces entity_object to simple on the wire, so an entity_object Record
Changes result is by construction server-internal — and server-internal code holding full
values in memory is the documented trust boundary (FLS guide §3.4), exactly as for
encrypted fields.
OptionalcontextUser: UserInfoProtectedAssertRejects a RunView whose ExtraFilter, OrderBy, or Aggregates expressions reference a
field the user cannot read.
Output projection alone is security theater. A user denied Salary can send
ExtraFilter: "Salary > 200000" or OrderBy: "Salary DESC" and reconstruct the values
from which rows come back and in what order — the column never appears in a result, so
every output-stripping point reports "secure." Aggregates are the same channel in a purer
form: Aggregates: [{expression: 'MIN(Salary)'}] under a narrow filter returns a denied
field's exact values directly. Predicate validation is a first-class enforcement point,
not a belt-and-braces afterthought. Together these cover every caller-authored expression
surface (UserSearchString is handled by excluding denied fields from the searched set,
not by rejection — see below).
Lives at the provider layer rather than in the GraphQL resolver (where the plan first
placed it) because every RunView funnels through here — the batch path, server-internal
agents and actions running under a restricted contextUser, and the resolver alike. One
gate, no path left uncovered.
The error deliberately does not say whether the field is missing or merely forbidden — see ProviderBase.FieldSecurityDenialMessage.
OptionalcontextUser: UserInfoBackground validation for the stale-while-revalidate fast-start pattern. Checks if local metadata is still current; if stale, fetches fresh metadata and atomically swaps it in. The app continues operating on cached data during this process — no blocking.
OptionalproviderToUse: IMetadataProviderProtectedBuildBuilds dataset filters based on the provider configuration. Ensures MJ Core schema is always included and never excluded.
Array of filters to apply when loading metadata
Stores a dataset in the local cache. If itemFilters are provided, the combination of datasetName and the filters are used to build a key and determine a match in the cache
ProtectedcacheSECURITY — decide whether the shared cache must be BYPASSED for a RunView that targets
a saved VIEW rather than a named entity (no EntityName), under a context user.
The cache-hit path returns BEFORE the DB provider's read-permission gate
(CheckUserReadPermissions). The primary gate keys off the entity resolved from
params.EntityName, so a ViewID-/ViewName-only request (the Explorer-standard shape for a
saved view) yields no entity there and the gate is disarmed — a read-denied user could be
served rows a permitted user warmed for the same ViewID. The vw: fingerprint segment makes
the two users' requests collide on exactly one slot, so the leak is clean.
Returns true when the cache must be skipped for this call (fail-closed):
ViewEntity supplied and its entity resolves → apply the normal CanRead gate on it
(allow caching for a permitted user; deny for a read-denied one).ViewEntity absent/unresolvable but ViewID/ViewName present → fail closed: the view's
real entity (hence the user's permission) is only known after the async MJ: User Views
lookup that the cache-hit path deliberately skips, so we cannot safely consult the cache.
Returns false when there is no context user, when EntityName is set (the normal gate owns
that path), or when no view identifier is present at all (nothing to gate).OptionalcontextUser: UserInfoProtectedCancelCancels any pending debounced metadata refresh. Call during teardown (logout, provider disposal) so a timer armed just before teardown doesn't fire a refresh against a connection that no longer has a valid session.
Checks if local metadata is out of date and needs refreshing. Compares local timestamps with server timestamps.
OptionalproviderToUse: IMetadataProviderOptionalbypassMinCheckInterval: booleanWhen true, skips the MinRefreshCheckIntervalMs throttle. Event-driven callers pass true: they hold positive evidence that a metadata member entity was just written, and the throttle otherwise answers "fresh" for any check arriving within the window of the previous one — which would silently drop the second of two permission changes made less than the window apart. For the same reason a bypassing caller always runs its own check, even while another is in flight: that check may have started before the write.
The throttle is armed only when the status request succeeded (returned timestamps). A
check that throws or returns no timestamps leaves it unarmed, so a retried boot inside
the window really checks instead of reading "current" and loading nothing (#4887).
A non-bypassing caller arriving while a check is in flight is told no refresh is needed,
as the in-flight owner acts on the answer; that keeps N engines starting together down to
one request. It must not share the owner's answer: a shared true would send every caller
into its own full metadata reload (preValidateAndRefresh / backgroundValidateAndRefresh /
RefreshIfNeeded load outside Config's reload single-flight). For the same reason it gets
false, not the owner's rejection, when the owner's check throws.
True if refresh is needed, false otherwise
If the specified datasetName is cached, this method will clear the cache. If itemFilters are provided, the combination of datasetName and the filters are used to determine a match in the cache
OptionalitemFilters: DatasetItemFilterType[]ProtectedCloneThe reuse-global fast path now builds a shared shell instead — see CreateSharedMetadataShell. The metadata graph is immutable after Config, so re-instantiating every Info object (~1s of synchronous constructor work for a ~600-entity install) bought no isolation the shell doesn't already provide. Subclass OVERRIDES of this method are still honored on the fast path (see CopyMetadataFromGlobalProvider) for backward compatibility; new customizations should override CreateSharedMetadataShell instead.
ProtectedComputeThe CLIENT's field-security cache key: a canonical list of the fields this user MAY read.
Keyed on the ALLOWED set rather than the denied set for two reasons. Once metadata ships
to browsers filtered to what a user may see (#3485),
a denied field will not appear in the client's field list at all — so a denied-set key
would be empty and would silently stop segmenting. The allowed list also resolves the
f:* ambiguity in the client's projection segment, where "full width" means different
columns for different users.
Returns '' when the entity has field security off or the user is denied nothing, so unrestricted users keep byte-identical fingerprints and shared slots.
Only takes effect once the client's metadata refreshes. A client on stale metadata computes a stale key; the backstop is that the server strips denied columns from every fresh fetch regardless.
ProtectedComputeThe fetch-widening field list for a cache-eligible request: always every entity field. One slot per (entity, filter, order) serves every caller regardless of the field subset they asked for, and field security narrows per request at read time via ApplyFieldSecurityProjection.
Optionalparams: RunViewParamsProtectedComputeThe field-security segment for a LocalCacheManager RunView fingerprint — the one place the client/server asymmetry is decided, so the two tiers cannot drift.
SERVER → no segment. Its slots are full-width and shared by every user; per-request narrowing happens at read time in ApplyFieldSecurityProjection, which runs on every hit and every miss. A segment here would fragment one shared slot into one per permission class and protect nothing the projection does not already handle.
CLIENT → the allowed-list key. Its slots are stored exactly as the server returned
them (already narrowed on the wire) and are not projected on read, so slot identity has to
carry the field set. Without it, a user whose access is tightened keeps being served their
persisted IndexedDB slot: the currency check compares maxUpdatedAt and rowCount only,
neither of which notices a column, so the server answers "current" and the browser keeps
showing a column that was just taken away.
Empty/undefined for unrestricted users on both tiers, so their fingerprints stay byte-identical and keep sharing slots.
ProtectedComputeComputes the per-user Row-Level-Security WHERE clause that InternalRunView will append to this query's SQL for the given user, so it can be folded into the cache fingerprint. RLS-scoped reads return a different result set than unscoped reads of the same entity+filter; without including the RLS clause in the cache key, a scoped user could be served a cached unscoped result set (a data leak).
Returns '' when the user is exempt from RLS on this entity (the common case), which makes the resulting fingerprint byte-identical to the pre-RLS format — preserving normal cache sharing.
Uses this (the active provider) to resolve the entity, never the global Metadata, so the
correct per-provider/per-tenant metadata is consulted.
OptionalcontextUser: UserInfoConfigures the provider with the specified configuration data. Handles metadata refresh if needed and initializes the provider.
Configuration including schema filters and connection info
OptionalproviderToUse: IMetadataProviderTrue if configuration was successful
ProtectedConvertConverts dataset item filters into a unique string key for caching.
Array of filters to convert
JSON-formatted string representing the filters
ProtectedCopyAdopts the global provider's metadata for this instance without reloading it from the server: shares the (immutable post-Config) metadata arrays by reference via CreateSharedMetadataShell and builds this instance's entity lookup maps.
ProtectedCreateBuilds this instance's AllMetadata as a thin shell over another provider's already-loaded metadata: every metadata array is a PER-INSTANCE shallow copy whose elements are the SHARED Info object instances, and CurrentUser remains this instance's own.
Why sharing the instances is safe — and why this replaced the former deep clone (CloneAllMetadata) on the reuse-global fast path: the metadata graph is immutable after Config. Refreshes swap the WHOLE AllMetadata object (UpdateLocalMetadata), never mutate the Info objects in place, so the only per-instance datum inside the graph is CurrentUser — which this shell keeps independent. The deep clone cost ~1s of event-loop-blocking constructor work per provider on every server request (MemberJunction/MJ#3083); the shell is ~20 array-of-pointer copies (microseconds).
Why the array containers are copied rather than aliased: an in-place
.sort()/.push()/.splice() by request-scoped code then stays local to
that provider — matching the clone era's isolation for the common accidental
mutation class — instead of reordering the global graph for every other
in-flight request. Only the top-level AllMetadata collections get this
per-instance protection: everything below them is shared, including the
nested arrays owned by Info objects (entity.Fields,
entity.RelatedEntities, application.ApplicationEntities, ...) — an
in-place mutation of those is process-wide. Property writes on the shared
Info objects themselves are likewise visible process-wide (as they always
were on the client's global provider): treat Info objects and everything
they own as read-only; copy before sorting.
Owning provider: the shared Info objects deliberately keep the BaseInfo.OwningProvider of the provider that built them (the source global provider); the shell does NOT re-stamp them. Re-stamping a shared instance would re-point it for every other provider holding it. The sibling lookups it drives (IS-A children, RLS filters, query fields, ...) therefore read the source provider's graph — the same immutable graph this shell points into, so the answers are identical. A provider that needs its own graph (a different database) must load its own metadata, not adopt a shell.
Override precedence: if a subclass overrides BOTH this method and the deprecated CloneAllMetadata, the CloneAllMetadata override wins on the fast path (see CopyMetadataFromGlobalProvider) — the conservative back-compat choice, since pre-#3083 subclasses could only have customized adoption through CloneAllMetadata. Remove the CloneAllMetadata override to activate a CreateSharedMetadataShell override.
Creates a new transaction group for managing database transactions. Must be implemented by subclasses to provide transaction support.
A new transaction group instance
ProtectedDatasetCompares the per-entity row counts of a cached dataset against the server's status.
The timestamp comparison its callers run first catches new and updated rows; it cannot catch a
pure DELETE, which leaves LatestUpdateDate untouched while the count drops. This is the
check that catches those, and it is shared by GetAndCacheDatasetByName (which already holds
the blob) and IsDatasetCacheUpToDate (which reads it) so the two cannot drift apart again —
they had, and the differences were only ever masked by the warm path not running.
Two cases decide the shape of this, both load-bearing:
EntityUpdateDates never touches the blob
and is judged on its timestamp alone, so it answers true. Returning false here instead
looks strictly safer and is not: it makes such a dataset permanently stale, reloading on
every check (plan — the dataset-cache.DS2 regression)._date proxy expire independently and
a clear removes them in order, so the date can outlive the blob. Counts then cannot be
compared, which is false — refetch. A blob present but missing Results
answers false for the same reason rather than throwing, which is what the pre-unification
copy in the warm path would have done the moment that path started running.the cached dataset, which may be absent or incomplete
the server's status for the same dataset/filters combination
true when every entity the server reports matches the cache's row count for it
O(1) entity lookup by ID (UUID-normalized). Falls back to linear search if the internal Map hasn't been built yet.
O(1) entity lookup by name (case-insensitive, trimmed). Falls back to linear search if the internal Map hasn't been built yet.
ProtectedEntityUsed to check to see if the entity in question is active or not If it is not active, it will throw an exception or log a warning depending on the status of the entity being either Deprecated or Disabled.
OptionalcontextUser: UserInfoProtectedeventWhether the write described by entityEvent happened against the backend THIS provider's
metadata comes from. In a multi-provider process (a client connected to several MJ servers,
a server connected to several databases) a write on one backend must not refresh another's
metadata. Deliberately fails OPEN — when the event does not identify its provider, or a
connection string is unavailable, the answer is "yes": a spurious refresh is a bounded
cost, a suppressed one is a stale-permissions window.
Executes a query from a QueryExecutionSpec — the lower-layer interface-based entry point.
Runs the full pipeline: composition resolution → Nunjucks template processing → SQL execution.
Subclasses (GenericDatabaseProvider) provide the concrete implementation via InternalExecuteQueryFromSpec.
The execution spec describing the query, parameters, and inline dependencies
OptionalcontextUser: UserInfoOptional user context for permissions (required server-side)
Query results including data rows and execution metadata
ProtectedextractPerforms a full-text search across all entities that have FullTextSearchEnabled=true. Uses the existing RunView + UserSearchString infrastructure which routes through the database-native FTS capabilities (SQL Server FREETEXT functions, PostgreSQL tsvector).
This is the default implementation that works across all database providers. Each provider's createViewUserSearchSQL() method handles the platform-specific SQL generation.
OptionalcontextUser: UserInfoProtectedGetRetrieves all metadata from the server and constructs typed instances. Uses the MJ_Metadata dataset for efficient bulk loading.
OptionalproviderToUse: IMetadataProviderOptionalforceRefresh: booleanComplete metadata collection with all relationships
Gets a database by name, if required, and caches it in a format available to the client (e.g. IndexedDB, LocalStorage, File, etc). The cache method is Provider specific If itemFilters are provided, the combination of datasetName and the filters are used to determine a match in the cache
OptionalitemFilters: DatasetItemFilterType[]OptionalcontextUser: UserInfoOptionalproviderToUse: IMetadataProviderThis routine gets the local cached version of a given datasetName/itemFilters combination, it does NOT check the server status first and does not fall back on the server if there isn't a local cache version of this dataset/itemFilters combination
OptionalitemFilters: DatasetItemFilterType[]Returns a record's name for synchronous display code, or undefined when it is not at hand.
This provider keeps no record names, so without loadIfNeeded the answer is always
undefined. A provider that serves a single user (GraphQLDataProvider) overrides the
record-name methods with a cache; one shared by several users must not, since a name can be
withheld from some of them.
The name of the entity
The primary key value(s) for the record
OptionalloadIfNeeded: booleanLook the name up when it is not cached
The display name, or undefined if not cached and not looked up
A record's cached name, without ever starting a lookup. Always undefined here; see GetCachedRecordName.
ProtectedGetGets the current user information from the provider. Must be implemented by subclasses to return user-specific data.
Current user information including roles and permissions
Retrieves a dataset by name. When forceRefresh is true, bypasses any in-memory or local cache
and fetches directly from the database. When false (default), server-side providers may serve
from LocalCacheManager if TrustLocalCacheCompletely is true.
Creates a unique key for the given datasetName and itemFilters combination coupled with the instance connection string to ensure uniqueness when 2+ connections exist
OptionalitemFilters: DatasetItemFilterType[]Retrieves the date status information for a dataset and all its items from the server. This method will match the datasetName and itemFilters to the server's dataset and item filters to determine a match
Returns a list of entity dependencies, basically metadata that tells you the links to this entity from all other entities.
Creates a new instance of a BaseEntity subclass for the specified entity and automatically calls NewRecord() to initialize it. This method serves as the core implementation for entity instantiation in the MemberJunction framework.
The name of the entity to create (must exist in metadata)
OptionalcontextUser: UserInfoOptional user context for permissions and audit tracking
Promise resolving to the newly created entity instance with NewRecord() called
Creates a new instance of a BaseEntity subclass and loads an existing record using the provided key. This overload provides a convenient way to instantiate and load in a single operation.
The name of the entity to create (must exist in metadata)
CompositeKey containing the primary key value(s) for the record to load
OptionalcontextUser: UserInfoOptional user context for permissions and audit tracking
Promise resolving to the entity instance with the specified record loaded
Gets the display name for a single entity record. Uses the entity's IsNameField or falls back to 'Name' field if available.
The name of the entity
The primary key value(s) for the record
OptionalcontextUser: UserInfoThe acting user; field- and row-level security are applied for them
OptionalforceRefresh: booleanBypass any cache. This provider has none, so it always looks up.
The display name of the record, or an empty string if not found or not readable
Gets display names for multiple entity records in a single operation.
Array of entity/key pairs to lookup
OptionalcontextUser: UserInfoThe acting user; field- and row-level security are applied for them
OptionalforceRefresh: booleanBypass any cache. This provider has none, so it always looks up.
Array of results with names and status for each requested record
ProtectedGetRetrieves the latest metadata update timestamps from the server.
OptionalproviderToUse: IMetadataProviderArray of metadata update information
Returns the timestamp of the local cached version of a given datasetName or null if there is no local cache for the specified dataset
the name of the dataset to check
OptionalitemFilters: DatasetItemFilterType[]optional filters to apply to the dataset
Returns a list of dependencies - records that are linked to the specified Entity/RecordID combination. A dependency is as defined by the relationships in the database. The MemberJunction metadata that is used for this simply reflects the foreign key relationships that exist in the database. The CodeGen tool is what detects all of the relationships and generates the metadata that is used by MemberJunction. The metadata in question is within the EntityField table and specifically the RelatedEntity and RelatedEntityField columns. In turn, this method uses that metadata and queries the database to determine the dependencies. To get the list of entity dependencies you can use the utility method GetEntityDependencies(), which doesn't check for dependencies on a specific record, but rather gets the metadata in one shot that can be used for dependency checking.
Returns a list of record IDs that are possible duplicates of the specified record.
Checks if a specific record is marked as a favorite by the user.
True if the record is a favorite, false otherwise
ProtectedhandleStatic fan-out callback: a BaseEntity save/delete (or a remote-invalidate from another
server) touched lowerEntityName. If that entity is one of the entities this provider's
metadata is BUILT FROM, the metadata this provider is serving — and, on the server, the
metadata every per-request provider adopts from it — is now stale, so schedule a debounced
refresh. Permission metadata is the load-bearing case: field-level security is enforced
FROM metadata at every enforcement point, so a rule an administrator just tightened is
simply not enforced until this re-read happens.
Reacts to another server saving its metadata snapshot: after a short debounce, runs the normal staleness check (RefreshIfNeeded, bypassing the check throttle). The check compares this process's timestamps with the database and adopts the shared snapshot, so a metadata change made on one server reaches the others in seconds instead of at the next poll. A server that is already current does no refresh and saves nothing, so notices do not echo around the fleet.
true when the event was a metadata notice (and a check was scheduled)
Whether a record's name is cached for synchronous retrieval. Always false here; see GetCachedRecordName.
ProtectedInternalInternal implementation for spec-based query execution. Subclasses must provide the concrete pipeline (composition → templates → execute).
ProtectedInternalInternal provider-specific implementation to get a single entity record name from database. Subclasses must implement this to query the database.
The display name of the record or null if not found
ProtectedInternalInternal provider-specific implementation to get multiple entity record names from database. Subclasses must implement this to query the database in batch.
Array of results with names and status for each requested record
ProtectedInternalProvider-specific transport for a Remote Operation. The default implementation reports that the provider does not support remote operations; concrete providers override it — server providers resolve and execute the operation in-process, the client provider marshals it over GraphQL. Kept as an overridable (non-abstract) hook so existing providers remain source- compatible until they opt in.
Trimmed, non-empty operation key (validated by RouteOperation).
ProtectedInternalInternal implementation of RunQueries that subclasses must provide. This method should ONLY contain the batch query execution logic - no pre/post processing. The base class handles all orchestration (telemetry, caching).
ProtectedInternalInternal implementation of RunQuery that subclasses must provide. This method should ONLY contain the query execution logic - no pre/post processing. The base class handles all orchestration (telemetry, caching).
ProtectedInternalInternal implementation of RunView that subclasses must provide. This method should ONLY contain the data fetching logic - no pre/post processing. The base class handles all orchestration (telemetry, caching, transformation).
ProtectedInternalInternal implementation of RunViews that subclasses must provide. This method should ONLY contain the batch data fetching logic - no pre/post processing. The base class handles all orchestration (telemetry, caching, transformation).
ProtectedinvalidateDrops every in-flight/lingered RunView entry whose params touch the given entity (lowercased name). Called on BaseEntity save/delete/remote-invalidate.
Drops this provider's in-flight/lingered RunView dedup entries so the next identical RunView goes to the server instead of replaying a result from the last DedupLingerMs.
Saves and deletes made through a BaseEntity in this process already invalidate the entries
for their entity automatically. Call this after anything that can write on the SERVER
without such an event reaching this process — a Remote Operation, an Action, an agent run,
any other mutation. RouteOperation and the GraphQL client's mutation transport call
it for you.
Dropping an in-flight entry never affects the callers already waiting on it; it only stops later callers from joining (and stops the result from lingering), since that query may have read pre-write data.
OptionalentityNames: readonly string[]entities known to have been written. Only entries touching them are dropped. Omit (or pass an empty list) when the writes are unknown: every entry is dropped.
Determines if a given datasetName/itemFilters combination is cached locally or not
OptionalitemFilters: DatasetItemFilterType[]This routine checks to see if the local cache version of a given datasetName/itemFilters combination is up to date with the server or not
OptionalitemFilters: DatasetItemFilterType[]ProtectedIsWhether a saved query is bound to an external data source. The base returns false; providers that support external data sources override this (consulting query metadata) so the outer RunQuery CacheLocal layer can defer to InternalRunQuery's own external TTL caching. Synchronous + non-throwing: resolves from cached metadata only.
ProtectedisTrue when the entity is a CodeGen materialized-query wrapper (materialized_vw*) whose snapshot is refreshed out-of-band — the same one IsServerCacheAllowedForEntity excludes from the server cache.
True when a shared-cache event says the metadata snapshot changed: another process saved one
(set), or a tool that changed the database removed it (removed, e.g. mj migrate). Only
the timestamps key counts — it is written last (see SaveLocalMetadataToStorage) and
removed last — so each save or removal is one notice.
Whether entityName is one of the entities this provider's metadata is built from.
False when the membership set has not been loaded yet — a caller must not treat "unknown"
as "reload". Names are compared case-insensitively, matching the set recorded by
registerMetadataDatasetMembership.
ProtectedIsChecks whether server-side caching is allowed for the entity in the given RunViewParams. Returns false for entities that have TrustServerCacheCompletely = false, or for Record Changes which is always exempt (rows are created via raw SQL side-effects, not BaseEntity.Save(), so cache invalidation events never fire).
ProtectedLoadLoads metadata from local storage if available. Deserializes and reconstructs typed metadata objects.
Checks if local metadata is obsolete compared to remote metadata. Compares timestamps and row counts to detect changes.
Optionaltype: stringOptional specific metadata type to check
True if local metadata is out of date
ProtectedmergeMerges cached and fresh results for RunViews, maintaining original order.
The pre-processing result with cache info
OptionalcachedResults?: RunViewResult[]OptionalcacheStatusMap?: Map<OptionalcallerFieldsMap?: Map<number, string[]>Per-param-index caller Fields lists (lowercased), captured before PreRunViews widened params.Fields to all entity fields for cache-superset storage. An index is present ONLY when that widening actually happened — PostRunViews uses it to project cache-miss DB results back down to the requested shape.
OptionalfingerprintMap?: Map<number, string>Per-param-index cache fingerprints computed during PreRunViews — carried forward so PostRunViews doesn't recompute the RLS where-clause and fingerprint string for every batch item.
OptionalpreResolvedCache?: Map<string, CachedRunViewResult | null>Pre-resolved local cache entries keyed by client cache fingerprint, populated during prepareSmartCacheCheckParams so executeSmartCacheCheck can reuse the already-parsed results for 'current' items without a second IndexedDB read.
OptionalsmartCacheCheckParams?: RunViewWithCacheCheckParams[]When CacheLocal is enabled, contains the cache check params to send to server
OptionaltelemetryEventId?: stringOptionaluncachedParams?: RunViewParams[]OptionaluseSmartCacheCheck?: booleanWhen CacheLocal is enabled, indicates we should use smart cache check
The fresh results from InternalRunViews
Combined results in original order
ProtectedmergeMerges cached and fresh results for RunQueries, maintaining original order.
The pre-processing result with cache info
The fresh results from InternalRunQueries
Combined results in original order
This method will merge two or more records based on the request provided. The RecordMergeRequest type you pass in specifies the record that will survive the merge, the records to merge into the surviving record, and an optional field map that can update values in the surviving record, if desired. The process followed is:
The return value from this method contains detailed information about the execution of the process. In addition, all attempted merges are logged in the RecordMergeLog and RecordMergeDeletionLog tables.
The entities this provider's metadata is built from that declare they can change without
firing an event (Entity.TrustServerCacheCompletely === false).
Empty — the usual case — means a metadata sweep has nothing to look for, so a caller can skip the work (and any lease) without asking the database anything.
ProtectedNormalizeNormalizes non-entity ('simple') result rows so Date and numeric columns hold real
Dates and numbers on EVERY tier, matching what the generated entity types declare.
Before this existed, the value a simple read returned for a DATETIME column depended on
where the code happened to run: a fresh server-side query yields real Date objects (the
driver parses them and AdjustDatetimeFields timezone-adjusts them), a server-side Redis
cache hit yields ISO strings (JSON.parse with no reviver), and a browser client over
GraphQL yields ISO strings (rows are JSON.stringify'd on the wire). Same call, three
shapes. MJ's contract is a unified programming interface on both sides of the wire, so the
one representation the platform's own generated types declare — Date — is enforced here,
at the one choke point every provider's RunView pipeline flows through.
It makes date and number VALUES match the generated types; it does not make a caller's T
honest in general. A Status column typed as a closed union still holds whatever string the
database held, and plain rows never have entity methods. If you need the type to be fully
true, use ResultType: 'entity_object'.
The field-key lists are computed once per view from EntityInfo, not per cell. Rows already
in the right shape — the common server-side case, where the driver returned Dates — are
detected and the ORIGINAL array is kept untouched: same array identity, same row objects,
zero copying. A row is shallow-copied only when a cell actually converts, and that copy is
load-bearing: on a cache hit the rows handed back can be the cache's OWN objects (the
in-memory server store holds them by reference), so converting in place would write Dates
into the cache entry itself and corrupt it for serialization and for later readers.
Per-cell rules:
Date instances pass through untouched, so the pass is idempotent on every path.NULL/undefined cells are left alone rather than becoming epoch-1970 dates.Invalid Date, which renders
as that literal string and destroys the evidence of what the database actually held.Number.MAX_SAFE_INTEGER stays a string: the PostgreSQL
provider deliberately returns unsafe-range BIGINTs as strings to avoid precision loss,
and Number('9007199254740993') "succeeds" while silently corrupting the value.View-based runs (ViewID/ViewName with neither EntityName nor a loaded ViewEntity)
skip normalization: resolving the entity would take an async User Views read this late in
the pipeline. Pass EntityName alongside the view identifier to get normalized rows.
ProtectedPartialDecides whether the current staleness can be served by a scoped reload of the saved-query family. Compares local and remote timestamps item by item (the same test as LocalMetadataObsolete) and maps each stale entity back to its MJ_Metadata item code.
the family's item codes (the whole family — the query classes are swapped as a set) when every stale item belongs to it; null when anything else is stale, nothing is, or the inputs needed to attribute the change are missing — the full reload is then the answer.
ProtectedPostOptionalorganicKeys: OrganicKeyMetadataRow[]OptionalorganicKeyRelatedEntities: OrganicKeyRelatedEntityMetadataRow[]OptionalfieldPermissions: EntityFieldPermissionMetadataRow[]ProtectedPostBase class post-processor that all sub-classes should call after they finish their RunView process
OptionalcontextUser: UserInfoProtectedPostBase class utilty method that should be called after each sub-class handles its internal RunViews() process before returning results This handles the optional conversion of simple objects to entity objects for each requested view depending on if the params requests a result_type === 'entity_object'
OptionalcontextUser: UserInfoProtectedPostPost-processing hook for RunQueries (batch). Handles telemetry end.
Array of query results
Array of query parameters
The pre-processing result
OptionalcontextUser: UserInfoOptional user context
ProtectedPostPost-processing hook for RunQuery. Handles cache storage and telemetry end.
The query result
The query parameters
The pre-processing result
OptionalcontextUser: UserInfoOptional user context
ProtectedPostPost-processing hook for RunView. Handles result transformation, cache storage, and telemetry end.
The view result
The view parameters
The pre-processing result
OptionalcachedResult?: RunViewResultOptionalcallerRequestedFields?: string[] | nullThe caller's original Fields list (lowercased), captured before PreRunView widened params.Fields to all entity fields for cache-superset storage. Non-null ONLY when that widening actually happened — PostRunView uses it to project cache-miss DB results back down to the requested shape.
Optionalfingerprint?: stringOptionaltelemetryEventId?: stringOptionalcontextUser: UserInfoOptional user context
ProtectedPostPost-processing hook for RunViews (batch). Handles result transformation, cache storage, and telemetry end.
Array of view results
Array of view parameters
The pre-processing result
OptionalcachedResults?: RunViewResult[]OptionalcacheStatusMap?: Map<OptionalcallerFieldsMap?: Map<number, string[]>Per-param-index caller Fields lists (lowercased), captured before PreRunViews widened params.Fields to all entity fields for cache-superset storage. An index is present ONLY when that widening actually happened — PostRunViews uses it to project cache-miss DB results back down to the requested shape.
OptionalfingerprintMap?: Map<number, string>Per-param-index cache fingerprints computed during PreRunViews — carried forward so PostRunViews doesn't recompute the RLS where-clause and fingerprint string for every batch item.
OptionalpreResolvedCache?: Map<string, CachedRunViewResult | null>Pre-resolved local cache entries keyed by client cache fingerprint, populated during prepareSmartCacheCheckParams so executeSmartCacheCheck can reuse the already-parsed results for 'current' items without a second IndexedDB read.
OptionalsmartCacheCheckParams?: RunViewWithCacheCheckParams[]When CacheLocal is enabled, contains the cache check params to send to server
OptionaltelemetryEventId?: stringOptionaluncachedParams?: RunViewParams[]OptionaluseSmartCacheCheck?: booleanWhen CacheLocal is enabled, indicates we should use smart cache check
OptionalcontextUser: UserInfoOptional user context
ProtectedPreOptionalcontextUser: UserInfoProtectedPreBase class implementation for handling pre-processing of RunViews() each sub-class should call this within their RunViews() method implementation
OptionalcontextUser: UserInfoProtectedPrePre-processing hook for RunQueries (batch). Handles telemetry for batch query operations.
Array of query parameters
OptionalcontextUser: UserInfoOptional user context
Pre-processing result
ProtectedPrePre-processing hook for RunQuery. Handles telemetry and cache lookup.
The query parameters
OptionalcontextUser: UserInfoOptional user context
Pre-processing result with cache status and optional cached result
ProtectedPreOptionalcontextUser: UserInfoProtectedPrePre-processing hook for RunViews (batch). Handles telemetry, validation, and cache lookup for multiple views.
Array of view parameters
OptionalcontextUser: UserInfoOptional user context
Pre-processing result with cache status for each view
Synchronous pre-validation of cached metadata before engine startup.
On a warm load we serve the metadata graph from IndexedDB so the app can boot without pulling MBs of metadata from the server. Before engines run, this method makes one batched timestamp round-trip to confirm the snapshot is still current:
Cost on the warm-current path is one batched timestamp fetch (~50–200 ms depending on RTT). On the warm-stale path we additionally pay the full metadata fetch but avoid serving stale data to the UI in the first place.
On a cold boot, where Config() just loaded the graph (and the current user) from the server, the first call after that load returns immediately — no check, no current-user re-fetch. Later calls, and calls after a background refresh, check as usual.
Caller contract: invoke this before StartupManager.Startup().
OptionalproviderToUse: IMetadataProviderProtectedRebuildRebuilds the O(1) entity lookup Maps from the current AllEntities array. Called automatically from UpdateLocalMetadata().
Refreshes all metadata from the server. Respects the AllowRefresh flag from subclasses.
OptionalproviderToUse: IMetadataProviderTrue if refresh was initiated or allowed
ProtectedRefreshHow this provider refreshes after a metadata member entity changed. The base behavior is a hard Refresh — correct for database providers, where the process that PERFORMED the write is the one refreshing, so re-checking staleness first is wasted work and the re-read must bypass every cache layer. Transport providers (GraphQL) override this with a staleness check so a browser doesn't re-pull the full metadata graph for a change the server-side timestamp comparison can disconfirm.
Refreshes the CurrentUser from the server and updates local metadata in place. Useful on warm boot or when user roles/permissions change dynamically without entity schema changes.
Refreshes metadata only if needed based on timestamp comparison. Combines check and refresh into a single operation.
OptionalproviderToUse: IMetadataProviderOptionalbypassMinCheckInterval: booleanPassed through to CheckToSeeIfRefreshNeeded; event-driven callers set true so the throttle cannot eat a check they have positive evidence for.
True if refresh was successful or not needed
Refreshes the remote metadata timestamps from the server. Updates the internal cache of remote timestamps.
OptionalproviderToUse: IMetadataProviderTrue if timestamps were successfully refreshed
The same hard reload as Refresh, but the metadata dataset's reads join the ambient transaction for the duration of the call (MJ#4836).
A timer-driven refresh must stay on the pool so it cannot land beside COMMIT (#4514).
The caller that owns the transaction — mj sync push, between its own writes — is the
one case that has to see its uncommitted metadata rows, and it waits for this call, so
the read cannot race the commit.
OptionalproviderToUse: IMetadataProviderProtectedregisterRecords which entities compose this provider's metadata, from the loaded MJ_Metadata dataset result, and registers this instance with the static event fan-out so writes to any of them schedule a debounced metadata refresh. Called from GetAllMetadata on every successful load, so the set tracks the dataset definition as it changes.
ProtectedReloadReloads this provider's metadata. Single-flight: at most one reload runs at a time, and a request arriving mid-reload queues exactly one follow-up pass.
re-read the remote timestamps first and bypass every cache layer
OptionalproviderToUse: IMetadataProviderOptionalscopedFirst: booleantry the scoped reload of the saved-query family (see PartialRefreshMetadataItemCodes) before the full one. Defaults to true unless hardRefresh: an explicit Refresh() is always a full reload.
Removes all cached metadata from local storage. Clears both timestamps and metadata collections.
ProtectedResolveDecides whether a RunView returns the entity's binary fields, and records the decision on
params.IncludeBinaryFields so every later step (field widening, cache fingerprint,
transport) sees one answer.
Binary fields are returned when the caller sets IncludeBinaryFields, or names a binary
field in Fields — asking for a column by name is asking for it. Otherwise they are left
out, which keeps result sets, engine caches and the RunView caches free of large values
nobody reads. Must run before Fields is widened, since widening replaces the caller's list.
The view parameters; IncludeBinaryFields is set to true when applicable.
The entity being queried.
True when binary fields will be returned.
ProtectedResolveResolves any PlatformSQL values in RunViewParams to plain strings for the active platform. Mutates the params object in place so downstream InternalRunView implementations always receive plain string values for ExtraFilter and OrderBy.
ProtectedResolveThe RunQuery cache-serve seam (B45/B46) — resolves a RunQuery request against this provider's query metadata and answers, in ONE computation performed BEFORE fingerprinting:
categoryPath: the RESOLVED query's canonical full category path. This becomes a
distinguishing fingerprint segment (B46) so two same-named queries in different
categories can never collide onto one cache slot. When the request is unresolvable the
caller falls back to the CALLER-STATED params.CategoryPath (still distinguishing,
just not canonicalized).resolvable: whether metadata could resolve the request at all. Runtime-created
queries are typically NOT resolvable from the base metadata cache (it does not refresh
in-process) — the gate then applies the warmer tie-break instead.authorized: whether user may run the resolved query. Meaningful only when
resolvable is true.The BASE implementation resolves from the metadata Queries cache and enforces the
ROLES-ONLY QueryInfo.UserCanRun — the strongest check available at this layer.
Providers with richer query metadata MUST override this to enforce the SAME authorization
their miss path enforces (GenericDatabaseProvider overrides with
MJQueryEntityExtended.UserCanRun, which adds entity CanRead + recursive composition
checks — the exact check ValidateQueryForExecution applies on a cache miss). The
invariant this seam exists to hold: a cache HIT must never be easier to read than a
cache MISS (B45 was precisely that asymmetry — the TTL gate checked roles only while
the miss path also checked entity read permissions).
Optionaluser: UserInfoProtectedResolveThe entity a RunView targets, resolved with NO I/O — for gates that run on the result path,
where an async RunView.GetEntityNameFromRunViewParams (which issues a User Views query for
a bare ViewID) would be a query per result set.
Covers the two shapes every caller in this repository uses: an explicit EntityName, and a
loaded ViewEntity. A request carrying only ViewID/ViewName with neither is not
resolvable here and returns undefined — the same shape RunView's own row normalization
already declines to handle for the same reason, and the same one
cacheDeniedForViewOnlyRequest exists to fail closed for on the cache path.
Resolves a PlatformSQL value to the appropriate SQL string for this provider's platform. If the value is a plain string, it is returned as-is (backward compatible). If the value is a PlatformSQL object, the platform-specific variant is used if available, otherwise the default variant is used.
Routes a typed Remote Operation by key to its implementation (see IRemoteOperationProvider).
This is the public power-tool transport seam. Prefer the typed
BaseRemotableOperation.Execute() entry point in application code — RouteOperation is the
stringly-typed escape hatch for dynamic dispatch / generic tooling, not for building
significant systems. Server providers override InternalRouteOperation to execute the
operation in-process; the client (GraphQL) provider overrides it to marshal over the wire.
Only registered, active (and, when AI-authored, approved) operations are routable, and every
call is authorized on the server side.
Stable registry key of the operation (e.g. RecordProcess.RunNow).
The operation's typed input payload.
Optionaloptions: RemoteOpInvokeOptionsOptional invocation options (mode, progress callback, user, provider, fingerprint).
The operation result; never throws for logical failures — check Success/ErrorMessage.
ProtectedRunRuns all registered PostRunView hooks against a single result, returning the (possibly mutated) result.
Protected (not private) for the same reason as RunPreRunViewHooks above: a subclass pipeline that returns view rows WITHOUT passing through PostRunView/PostRunViews — e.g. the RunViewsWithCacheCheck smart-cache path — MUST apply these hooks to the rows it returns. PostRunView is the OUTPUT half of the enforcement seam (data masking / audit); a path that skips it returns rows the hooked paths would have masked.
OptionalcontextUser: UserInfoProtectedRunRuns all registered PreRunView hooks against a single RunViewParams, returning the (possibly mutated) params.
Protected (not private) on purpose: any subclass pipeline that executes view queries WITHOUT passing through PreRunView/PreRunViews — e.g. the RunViewsWithCacheCheck smart-cache path in GenericDatabaseProvider — MUST apply these hooks itself. Hooks are an enforcement seam (tenant scoping middleware injects filters here); a query path that skips them silently returns rows the hooked paths would have filtered out.
OptionalcontextUser: UserInfoRuns multiple queries based on the provided parameters. This method orchestrates the full execution flow for batch query operations.
Array of query parameters
OptionalcontextUser: UserInfoOptional user context for permissions (required server-side)
Array of query results
OptionalcontextUser: UserInfoRuns a view based on the provided parameters. This method orchestrates the full execution flow: pre-processing, cache check, internal execution, post-processing, and cache storage.
The view parameters
OptionalcontextUser: UserInfoOptional user context for permissions (required server-side)
The view results
ProtectedrunSingle source of truth for whether a RunView call participates in the local cache (both READ and WRITE). Pre/Post hooks for the singular and batch paths must all use this predicate — historically each site recomputed it inline and they drifted (PostRunViews wrote BypassCache results into the cache, poisoning the Fields-agnostic superset slot with narrow rows).
Ineligible:
BypassCache — caller explicitly wants true DB state, no cache interactionAfterKey — keyset pages are single-use AND the fingerprint doesn't include
the seek key, so caching a page would poison the entity+filter slotResultType 'count_only' — returns no rows; caching its empty Results under
a fingerprint that excludes ResultType would poison row queriesDataSource: 'Materialized' — the snapshot is rebuilt OUT-OF-BAND by the scheduled refresh
(direct SQL, no BaseEntity save), so the entity's normal event-driven cache invalidation never
fires for it; a cached materialized result would be served indefinitely stale after a refresh.
Bypass caching entirely for materialized reads. (The ds:materialized fingerprint segment still
keeps the short-lived dedup/linger layer from cross-serving Live vs Materialized in-flight reads.)ProtectedrunWrite-side eligibility for the smart-cache-check (stamped) path. On the trusting SERVER this is exactly runViewCacheEligible — the server cache is kept fresh by BaseEntity events, so a server-cache-disallowed entity must never be slotted. On a CLIENT the slot is instead written with a maxUpdatedAt stamp and DB-revalidated per request, so the server Trust/event gate does NOT apply: a server-cache-disallowed entity (Trust=0 'MJ: Audit Logs', Record Changes, other caching-disabled) is still safely client-cacheable when stamped — folding runViewCacheEligible's server gate onto this path regressed that (integration check client-cache.C12). Materialized reads stay excluded on both (the out-of-band snapshot swap the stamp can't observe).
Runs multiple views based on the provided parameters. Wraps the execution pipeline with request deduplication and a linger window so that concurrent (and near-sequential) identical calls share a single server round-trip. Every caller receives a shallow-copied Results array to protect against cross-caller mutations (push/sort/splice).
Array of view parameters
OptionalcontextUser: UserInfoOptional user context for permissions (required server-side)
Array of view results (shallow-copied Results per caller)
Saves current metadata to local storage for caching. Serializes both timestamps and full metadata collections.
ProtectedscheduleSchedules this provider's metadata refresh after a write to a metadata member entity. Delay and re-arm semantics come from MetadataMemberRefreshDelayMs and MetadataMemberRefreshRearmsOnNewEvents — debounce on the server, long jittered coalescing window on clients. The refresh targets THIS instance — the provider that loaded the dataset owns the metadata built from it; short-lived per-request providers never load the dataset (they adopt the global's metadata as a shared shell), so on the server only the process-global provider ever gets here.
Batch form of SearchEntity. Fans the input list out to N
independent SearchEntity calls via Promise.all; result arrays come
back aligned by input order (result[i] holds the matches for params[i]).
On the server side, the per-entity passes are independent — running them
concurrently is a real wall-clock win when the caller wants results from
multiple entities. On the client side, GraphQLDataProvider overrides
this method to pack the whole batch into a single GraphQL round-trip
instead of issuing N parallel HTTP requests.
See IMetadataProvider.SearchEntities for the contract.
ProtectedsearchRun the semantic ranking pass for SearchEntity. Each concrete
ProviderBase subclass supplies its own implementation: server-side
providers (GenericDatabaseProvider) embed the query text and query
an in-process vector pool directly; client-side providers
(GraphQLDataProvider) override SearchEntity / SearchEntities
outright to proxy via GraphQL and never reach this method.
Ranked array of ScoredCandidate whose ID is the parent
entity's record ID and Metadata.entityRecordDocumentId
carries the EntityRecordDocument PK.
Ranked search over one entity's records. See IMetadataProvider.SearchEntity for the contract and how this differs from EntityByName / FullTextSearch.
Implementation overview (concrete on ProviderBase, used as-is by every
server-side provider; GraphQLDataProvider overrides to proxy via GQL):
params.options.entityDocumentId
override or by looking up the active Search-category doc for the entity).IncludeInUserSearchAPI fields) and the semantic
pass (searchEntitiesSemanticPass, the protected template method
each concrete server provider implements).ComputeRRF() with optional per-list weights.Offers a record's name for later synchronous retrieval. BaseEntity calls it after every
Load(), LoadFromData() and Save(). Ignored here; see GetCachedRecordName.
Sets or removes a record's favorite status for a user.
ProtectedshouldOptionalcontextUser: UserInfoCompares this provider's metadata with the database — but only for metadata whose entities have declared that their rows can change without firing an event.
Metadata staleness is otherwise event-driven: a BaseEntity write to one of the entities the
metadata is built from schedules a refresh, another process saving its snapshot publishes a
notice, and a tool that changed the database clears the shared cache. None of those fire for a
change made with raw SQL, so metadata edited directly in the database is never noticed by a
running process. This is the backstop for that case.
It is not an unconditional poll. Entity.TrustServerCacheCompletely = true
(the default) states that every mutation flows through BaseEntity, which the event paths
already hear; polling such an entity cannot discover anything, and a recurring query prevents a
serverless database from pausing. So the check reads the database only when at least one
metadata member entity declares false — typically because an operator writes that table
directly. On a stock installation no metadata entity declares it, and this costs one in-memory
pass over the membership set.
OptionalproviderToUse: IMetadataProviderProvider to compare against, for multi-provider clients.
Which entities declared drift, whether the database was consulted, and whether metadata was actually reloaded.
ProtectedTransformTransforms the result set from simple objects to entity objects if needed.
The RunViewParams used for the request
The RunViewResult returned from the request
OptionalcontextUser: UserInfoThe user context for permissions
ProtectedTryReloads only the saved-query family of MJ_Metadata and swaps it into the current metadata,
keeping every other array — and every object in them — as is. The dataset definition is
reused verbatim (same columns, where clauses and post-processing): every other item is
requested with a 1=0 filter, so it reads nothing. Always bypasses cache layers — the read
is small, and a cached answer is exactly what a staleness check cannot trust.
OptionalproviderToUse: IMetadataProvidertrue when the scoped reload was applied; false when it does not apply or any part of it failed — the caller then runs the full reload, so a failure never leaves stale queries.
ProtectedUpdateUpdates the local metadata cache with new data.
The new metadata to store locally
Protected StaticarrayConverts an ArrayBuffer to a base64-encoded string. Used for compressed metadata storage/retrieval.
Protected Staticbase64Converts a base64-encoded string to an ArrayBuffer. Used for compressed metadata storage/retrieval.
StaticFieldThe single wording for "you cannot use this field," modeled on SQL Server's posture of never disclosing whether an object is missing or merely inaccessible.
Naming the field is safe — the caller supplied it, so it tells them nothing they did not already know. Naming the REASON is not: confirming "this field exists and is restricted" turns any predicate into an oracle for probing which columns a deployment considers sensitive. The ambiguity also keeps this message correct after #3485 tiers metadata and restricted fields stop shipping to clients at all, at which point "does not exist" becomes literally true from the client's vantage point.
Protected StaticNamesTrue when fieldNames names at least one of the entity's binary fields (case-insensitive,
whitespace-trimmed).
The entity being queried.
Field names as a caller supplied them, e.g. RunViewParams.Fields.
Protected StaticUnionReturns the caller's requested fields (lowercased) unioned with the entity's
primary key field names. Platform contract: when Fields is explicitly
specified, results ALWAYS include the primary key(s) — the direct SQL path has
always done this, differential smart-cache merges require it, and entity
linking in UIs depends on it. Applying the same union at every projection site
keeps result shapes identical across cached, non-cached, and smart-cache paths.
A real
ProviderBasethat knows only the entities it is given: a typed metadata-provider double for unit tests, so a test hands code anIMetadataProviderwithout a cast. Entity lookups (Entities,EntityByID) andCurrentUserwork; every data operation throws. Spy onRunView/RunViews(vi.spyOn(provider, 'RunView')) to serve rows.