Class ConversationEngine

Subclasses can override this method to perform additional loading tasks

Applies an immediate array mutation based on the entity event type. This is faster than running a full view refresh for simple add/update/delete operations.

On save, the cached entry is a clone owned by this engine's provider — not the saver's entity instance. Storing the saver's instance would pin the saver's provider (often a per-request provider) inside the engine's cache for the engine's full lifetime, which leaks the provider and all its associated state.

Removes a deleted record from the engine's in-memory arrays using the primary key values from the remote-invalidate event payload. No server round-trip needed.

Applies record data from a remote-invalidate event directly to the engine's in-memory arrays. Creates a BaseEntity instance, loads the JSON data into it, then updates the matching config arrays — same as applyImmediateMutation but from serialized data instead of a live entity.

Opens a new full-refresh "generation" for a property and returns its token. Each call bumps the property's monotonic counter, so a token is the latest iff no later refresh for that property has begun since. See _configRefreshGeneration.

Builds the RunViewParams for an engine config. Used by LoadSingleEntityConfig, LoadMultipleEntityConfigs, RegisterCacheChangeCallbacks, and syncLocalCacheForConfig to ensure the fingerprint-affecting params (EntityName, ExtraFilter, OrderBy, IgnoreMaxRows, IncludeBinaryFields) are always consistent — preventing cache key mismatches that break cross-server invalidation via Redis pub/sub and local cache upsert/remove operations.

Row-level security is not part of the fingerprints built from these params. The engine builds them with GenerateRunViewFingerprint(params, connection), without the RLS clause the provider adds for the reading user. On a server that is correct: engines load as the system user, whose RLS clause is empty, so the provider's fingerprint for the same read matches. An engine loaded by an RLS-scoped user (possible on a client) would compute an unscoped fingerprint for a slot the provider stored under a scoped one, so its cache callbacks and syncLocalCacheForConfig would target a different slot. Pass the RLS clause here if such an engine is ever needed.

Determines if an immediate array mutation can be used instead of running a full view refresh. Immediate mutations are only safe when:

1. The config has no Filter (no server-side filtering that might exclude the entity)
2. The config has no OrderBy (no server-side ordering that would need to be maintained)
3. The config does not use ResultType='simple' (immediate-mutation pushes BaseEntity instances; arrays loaded as plain objects would become type-mixed)
4. The subclass has not overridden AdditionalLoading (no post-processing that depends on full data)

All-or-nothing permission gate: checks CanRead on every entity config. If ANY entity is denied, ALL configs are skipped — the engine is marked permission-constrained and its data arrays are set to empty []. This prevents noisy permission-denied errors and endless retry loops for users with limited permissions (e.g., org-scoped SaaS roles).

On the server side with a system user (who has all permissions), this method returns the original configs unchanged — no behavior change for privileged users.

Classifies a single entity event against a single config's backing array:

• 'refresh' — the array does not yet reflect this event's changes; a refresh (or immediate mutation) is required.
• 'notify' — the array already reflects the change (in-place save of the array's own cached instance, or a manually-pushed create); observers still need a notification.
• 'silent' — a delete of a row absent from the array; "already spliced" is indistinguishable from "never matched the Filter", so no notification is emitted (manual-splice engine code owns that notification).

For deletes, the by-key membership check uses the event payload's pre-delete OldValues snapshot — BaseEntity.Delete() calls NewRecord() right after raising the event, which wipes field values and REGENERATES the primary key, so the live entity's key can never match the deleted row by the time the debounced handler runs.

Creates a fresh BaseEntity owned by this engine's provider and populates it from the given source entity's field values. Used by applyImmediateMutation to avoid pinning the source entity's provider inside this engine's cache.

The cache fingerprint of a config's slot, as RegisterCacheChangeCallbacks computes it.

True when the config's last load attempt left it in a successfully-loaded state. Reads the same map entry HandleSingleViewResult writes — a transient failure (network, server restart) records loadedSuccessfully=false; a permission denial is recorded as loaded-empty (true) and is deliberately NOT retryable.

Returns false ONLY when we can positively determine that the effective user lacks Read permission on entityName. Unknown cases (no entity name, no resolvable user, entity not in metadata) return true — so the default is to treat a failure as transient/retryable and server-side system-user loads (full access) are unaffected.

Used by HandleSingleViewResult to classify a FAILED config load: a load that failed because the user can't read the entity is a PERMANENT condition (a retry will never succeed for this role), so the engine should load that property empty rather than loop on "not marking as loaded" — which is what hangs the Explorer shell for a restricted / app-scoped user (e.g. a magic-link guest). Security is unaffected: the user still receives no data.

This is a classifier consulted AFTER a failure, never a predictive pre-skip — so a readable entity is always actually queried, and stale/late client permission metadata can never cause a readable entity to be silently skipped.

This method handles the debouncing process, by default using the EntityEventDebounceTime property to set the debounce time. Debouncing is done on a per-entity basis, meaning that if the debounce time passes for a specific entity name, the events will be processed. This is done to prevent multiple events from being processed in quick succession for a single entity which would cause a lot of wasted processing.

ALL events raised during the debounce window are buffered and delivered as one batch to ProcessEntityEvents — not just the last one. The refresh-vs-skip decision must be an OR over every coalesced event: judging only the last event would let an already-applied write (e.g., an engine method's in-place save of a cached instance) mask an earlier fresh-instance save the array has never seen.

Override this method if you want to change how debouncing works, such as having variable debounce times per-entity, etc.

Notifies subscribers of ObserveProperty(propertyName) that the array has changed. No-op if no one has ever observed this property (BehaviorSubject not created). Called from the array mutation sites in BaseEngine.

Finds an entity in the array by matching all primary key columns. Supports composite primary keys by comparing all PrimaryKey fields from EntityInfo.

Retrieves engine-loaded data for a config property by name. This is the canonical accessor for engine getter properties — it checks the data map for permission denial and throws PermissionConstrainedError with the specific denied entity name(s) if the config was skipped.

Subclasses should use this in every getter that exposes engine-loaded data:

public get Models(): MJAIModelEntityExtended[] {
    return this.GetConfigData<MJAIModelEntityExtended>('_models');
}
Copy

Counts that describe the state a subclass derives in AdditionalLoading — grouped child collections, lookup maps. Override to expose them; the default reports nothing. Must be read-only and must not throw.

Subclasses of BaseEngine can override this method to handle individual MJGlobal events. This is typically done to optimize the way refreshes are done when a BaseEntity is updated. If you are interested in only BaseEntity events, override the HandleIndividualBaseEntityEvent method instead as this method primarily serves to filter all the events we get from MJGlobal and only pass on BaseEntity events to HandleIndividualBaseEntityEvent.

Handles remote-invalidate events from cross-server cache invalidation. These events are fired by GraphQLDataProvider when it receives a cache invalidation notification via GraphQL subscription (originating from Redis pub/sub on another server).

When the event payload includes recordData (the saved entity as JSON), the engine applies the change directly to its in-memory array — no server round-trip needed. For delete events or events without recordData, falls back to LoadSingleConfig.

Handles the result of a single view load.

Checks if the current instance has overridden the AdditionalLoading method. We do this by comparing the method to the base class's method.

Checks if the exact entity object reference is already in the config's data array. Used to skip unnecessary refreshes for UPDATE events where the object was mutated in place.

Checks if an entity is in the config's data array by object reference OR by primary key match. Used for DELETE events where we need to know if the entity still exists in the array.

For deletes, pass preDeleteValues (the event payload's OldValues snapshot): by the time the debounced handler runs, BaseEntity.Delete() has already called NewRecord(), which wipes the entity's fields and regenerates its primary key — so a by-key check against the live entity can never match the deleted row. Same hazard (and same OldValues workaround) as LocalCacheManager.HandleBaseEntityEvent.

True when generation is still the most recent token handed out by beginConfigRefresh for propertyName — i.e. no newer full refresh for this property has started since. A refresh whose token is stale must NOT commit its results: a newer refresh was initiated afterward and read a more-recent state.

This method should be called by sub-classes to load up their specific metadata requirements. For more complex metadata loading or for post-processing of metadata loading done here, overide the AdditionalLoading method to add your logic.

Loads the specified metadata configurations.

Handles the process of loading multiple entity configs in a single network call via RunViews()

Loads a single metadata configuration.

Handles the process of loading a single config of type 'dataset'.

Handles the process of loading a single config of type 'entity'.

Records a config as successfully loaded with an EMPTY result set. Used when a load failed permanently because the context user lacks Read on the entity: the engine exposes an empty array (not a hang) and is marked loaded so shell boot can complete for a restricted role.

Emits change notifications for a config whose backing array ALREADY reflects the entity event — e.g., engine code saved the array's own cached instance in place, or manually pushed a newly created entity after Save. In those cases ProcessEntityEvent safely skips the redundant refresh, but the notification must NOT be skipped: without it, DataChange$ and ObserveProperty subscribers (and anything derived from them downstream) never learn the array changed and are stranded on stale state.

Engine subclasses that manually SPLICE a deleted row out of a config's array must call this themselves ('delete') right after splicing — the debounced event handler cannot distinguish "already spliced" from "never matched this config's Filter", so it stays silent for absent rows.

Deliberately does not run AdditionalLoading — the skip paths never did, and engines that maintain their arrays manually own any derived-data updates themselves.

Notify listeners that a data property has changed. Called automatically by HandleSingleViewResult after data refresh and by applyImmediateMutation for array operations. Subclasses can also call this manually when modifying data arrays directly.

Called when another server instance updates cached data that this engine is tracking. Default behavior: reload the affected config from the database.

Engines can override this for custom behavior (e.g., incremental update using the event's CacheChangedEvent.Data payload).

True when payloadRows are the same row versions this config currently holds: the same number of rows, and the same multiset of (primary key, __mj_UpdatedAt) pairs.

Identity is derived from what the engine holds right now, never remembered, so no other assignment path has to keep anything in step with it. (An earlier version remembered the last payload applied and went stale whenever another path replaced the property.) Comparing rows, not the writer's maxUpdatedAt, keeps it independent of how the writer stamped the slot, and catches a delete paired with an insert.

Returns false — apply the payload — whenever identity cannot be established: the config is not loaded, the entity has no __mj_UpdatedAt column, or a timestamp cannot be read. Known limit: a change that does not bump __mj_UpdatedAt (direct SQL) is invisible here, as it is to the client smart-cache check.

Back-compat single-event wrapper around ProcessEntityEvents. The debounced pipeline delivers full batches to ProcessEntityEvents — override THAT method to change event-processing behavior; this wrapper exists for subclasses/tests that process one event at a time.

Does the actual work of processing all entity events coalesced into one debounce window. Not called directly from the event handler because we first debounce the events, which also introduces a delay that is usually desirable so processing happens outside the scope of any transaction processing that originated the events.

Per matching config, the decision is an OR over the whole batch:

• If ANY event's changes are not yet reflected in the config's array, run the refresh (or apply each such event via immediate mutation when the config allows it). A single full refresh covers every event in the window.
• Else, if any event's changes were already applied (in-place save of a cached instance, manual push after create), notify observers once — the refresh is redundant but the notification is not.
• Deletes of rows absent from the array stay silent: "already spliced by engine code" (that code owns the notification, see notifyAlreadyAppliedMutation) is indistinguishable from "never matched this config's Filter", and notifying would assert phantom deletes to filtered configs' observers.

A transiently-failed refresh schedules a bounded retry via scheduleEventRefreshRetry — without it, the consumed debounce event would leave observers permanently stale until an unrelated event arrived.

This is the best method to override if you want to change the actual processing of entity events but do NOT want to modify the debouncing behavior.

Rebuilds derived state after a cache-change event, one rebuild at a time and without letting a failure escape.

AdditionalLoading is subclass-supplied and may perform database I/O — some overrides call Config() on another engine, which issues queries. Cache-change events are dispatched per fingerprint from a fire-and-forget pub/sub callback, so a burst of them would otherwise run several of those overlapping on the same connection. Serializing them keeps the rebuild off the critical path of whatever else that connection is doing.

Failures are logged rather than propagated: this runs from an event callback with no caller able to handle them, an unhandled rejection there would be lost anyway, and one engine's rebuild failing must not stop the next event from being applied.

Registers cross-server cache change callbacks for entity configs. When another server instance updates cached data for an entity this engine tracks, the engine will automatically reload the affected config.

This enables multi-server deployments to keep engine in-memory arrays synchronized without polling. Requires a Redis-backed storage provider with pub/sub enabled (via RedisLocalStorageProvider.StartListening).

Resolves BaseEnginePropertyConfig.IncludeBinaryFields for this engine's provider: 'DatabaseProviderOnly' becomes true on a database provider and false on any other.

Decides which identity this engine acts as and stores it in _contextUser.

Server-side (ProviderType.Database), an engine acts as the MJ system user — full stop. Engine data is infrastructure: the cache is process-wide and shared by every user of the process, so its contents must not depend on whichever caller reached Config() first. A caller carrying entity-permission denials, RLS row scoping, or field-level denials would otherwise seal a partial, empty, or permission-constrained cache that then serves everyone until the process restarts. Restricting what any given user may SEE stays where it belongs — at the point data is served to them, not at the point it is cached.

There is deliberately only ONE identity here rather than a separate load-user: every load, refresh, hydration and attribution path reads _contextUser, so a single field makes the rule structurally impossible to violate — a refresh path added later cannot forget to use the right one. The cost is that ContextUser reports the system user on a server; see its docs.

Client-side (ProviderType.Network) this is a plain assignment: a browser hosts one signed-in principal, and the server already strips whatever that user may not see.

Degrades, never fails. When no system user can be resolved — no server-side WellKnownUserSource registered, no such row in this database, a failed lookup — the engine simply acts as the caller, exactly as it did before this existed.

Schedules a bounded, backed-off retry of a config refresh that failed transiently during entity-event processing. Without this, one failed RunView after a save would permanently strand every observer on stale data — the debounced event is already consumed, so nothing else re-runs the refresh until an unrelated event for the same entity arrives.

At most one retry is pending per property at a time; a retry that succeeds notifies observers through the normal HandleSingleViewResult → NotifyDataChange path. Permission denials never reach here (HandleSingleViewResult marks them loaded-empty).

Internal method to set the provider when an engine is loaded. Once this engine instance has a provider bound, subsequent calls are no-ops — preventing transient per-request providers from displacing the persistent provider that first bound to this connection. The cache key is the connection (not the object), so the first persistent provider to load an engine for a connection "owns" the engine for that connection's lifetime.

This method is responsible for registering for MJGlobal events and listening for BaseEntity events where those BaseEntity are related to the engine's configuration metadata. The idea is to auto-refresh the releated configs when the BaseEntity is updated.

Syncs an entity change to the LocalCacheManager for a config with CacheLocal enabled. This ensures that IndexedDB/localStorage stays in sync with the engine's in-memory array.

Only called for configs WITHOUT Filter/OrderBy (immediate mutation path). Filtered/sorted configs use debounced refresh which handles its own caching.

Helper method for sub-classes to have a single line of code that will make sure the data is loaded before proceeding and will throw an error if not loaded.

Utility method to upgrade an object to a BaseEnginePropertyConfig object.