Class SearchEngine

Access the cached provider metadata from SearchEngineBase

Resolver for a scope's declared Search Context dimensions. Overridable so a host can supply additional derivation sources (e.g. an external signal) without forking the engine.

Resolve the metadata provider via SearchEngineBase (which extends BaseEngine and tracks ProviderToUse).

Enforce a lane's RequiredMetadataKeys contract (Phase E).

The rendered filter must mention every key the author declared. This is the only guard that catches a filter which rendered partially — where an optional {% if %} clause disappeared because its dimension was absent or discarded, leaving a non-empty filter that passes every other check while restricting on strictly less than intended.

Fail a search CLOSED when a scope field that RESTRICTS was authored but did not render usably (see CheckRenderedTemplate).

Throwing rather than dropping the offending row is deliberate. Dropping one lane of several would quietly change what the scope covers, and a scope's ONLY lane dropped used to collapse to undefined — "unscoped" to every provider. (A non-global scope now hands providers an empty list, which means "nothing", but a silently narrowed scope is still a wrong answer.) A broken restricting template is a misconfiguration and should be loud and actionable, never silently degraded.

Build a stable cache key for a search.

The key must include EVERY input that can change the result set, or the cache will serve one caller's results to another. Two of those inputs were previously missing and both are tenancy/authorization-relevant:

• SearchContext — carries PrimaryScopeRecordID (the TENANT) and the SecondaryScopes dimensions. Omitting it meant a user with access to two tenants could be served the other tenant's results for up to the cache TTL, and that two searches differing only by dimension (channel, skill, …) collided. This is the reason for the fix.
• ScopeIDs — determines which corpora are searched at all.

Also folded in: Mode, FusionWeightsOverride and PermissionOverfetchFactor (all change ranking or the candidate pool, so they change results) and AIAgentID (conservative: agent identity participates in scope resolution and per-agent overrides upstream; including it can only cost a miss, never leak).

ScopeIDs order is deliberately NOT sorted: it is behaviourally significant, because cross-scope reranker config and budget are taken from the first scope in the array that supplies one. Two different orderings can therefore produce different results and must not share a key.

The whole projection is emitted through stableStringify so key-order variation in SecondaryScopes doesn't fragment the cache. The user ID stays as a readable prefix for debuggability.

Note this runs AFTER scope resolution in searchInternal, so nothing here is circular. Entitlement is resolved by the CALLERS (__Scoped_Search, the GraphQL resolvers), which deny before reaching the engine; the engine therefore never caches across an allow/deny boundary.

Resolve entitlement for the dry run, including the skill and tenant principals. protected, like principalsFrom, so a test can drive it without weak-typed casts.

Filter search results by entity-level and row-level security permissions, then by whether the user may read the record each content result was derived from.

Steps 1–4 are a safety net. Providers are expected to do per-provider permission push-down (Section 3.6 of plans/search-scopes-rag-plus.md). If those steps remove more than a handful of results in practice, the responsible provider's push-down is incomplete and should be fixed. Step 5 is not: it drops by design, because no provider can push an origin record's own row filters down into a content index. Its removals are counted into stats.OriginGateRemoved, so the late-filter count keeps its meaning.

Groups results by entity for efficient permission checking:

1. Unknown entities are excluded (fail closed).
2. If the user lacks entity-level CanRead, all results for that entity are dropped.
3. If the user is exempt from RLS, all results pass through.
4. If RLS applies, a RunView validates which record IDs the user can read.
5. Content items and chunks are kept only when the user may read the record they were derived from (VerifyOriginRecords).

storage-file results take their own path (filterStorageResults): they are re-checked against the storage-account permission model for this user, never passed through on the strength of their type.

Keep only the results every reader in params.Audience may read, on top of the caller's own filter. Each reader gets the same safety net the caller did (filterByPermissions: entity read, row filters, ownership), so a shared conversation's results are the intersection of what every participant may see — the asker's reach is the ceiling, each other reader's reach lowers it. Readers are checked concurrently; a reader with the caller's own ID, or listed twice, is checked once. Expects a validated audience (searchInternal validates it before doing any work).

Two things the pass cannot do, both failing closed:

• Storage hits are refused under an audience. Their permission model is the storage account's; filterByPermissions re-checks a storage-file hit's account for the user it is given, but this pass does not yet combine those per-reader answers for storage, so a storage hit is not shown to the room.
• A reader with no roles (UserRoles: []) reads nothing. Permissions and row filters are evaluated from UserInfo.UserRoles, so such a reader empties the result. A reader with no UserRoles array at all never gets here: validation refuses it.

Identity contract. A result survives when every reader's filterByPermissions returns it. Survivors are matched to the input by EntityName|RecordID (the record the result names), falling back to object identity for a result that names no record. An override of filterByPermissions may therefore return copies, but must not rewrite EntityName or RecordID.

Protected so a host can change how an audience combines (a host that materializes a shared reach in one query can replace the per-reader pass). It receives the whole params, so an override can see the anchor (SearchContext) and the scopes. The default is the intersection.

Load an agent or skill principal by ID; null when no ID was supplied. protected so a test can substitute principal loading; see explainEntitlement.

Build the principal set a dimension's expansion query may bind.

Exists so the real search path and the ExplainScope dry run cannot construct principals differently. They already did once: ExplainScope passed the agent and the search path passed nothing, so any scope deriving its bound from AgentID previewed one bound and searched with another. A single conversion site makes that class of drift unrepresentable rather than merely fixed.

Accepts anything carrying the two principal IDs, which both SearchParams and ExplainScopeInput do.

The configured provider whose engine-stamped ProviderId item carries, or undefined when it carries none or names no configured provider. The engine overwrites ProviderId on every result a provider returns, before fusion.

Protected so a probe can map a hand-built hit to a provider instance without configuring the engine. It decides which results skip ownership verification and which skip the lane ExtraFilter re-check, so an override must only ever return the provider that actually produced the result.

For each origin entity, those of its record ids the user may read under that entity's own permissions and row filters — the check verifyOwnershipAndRowFilters gives a result's own entity. All entities are asked in one RunViews batch. An entity the provider does not know, the user may not read at all, or whose view fails gets no entry, so only the rows derived from it are dropped.

Protected so a host extending VerifyOriginRecords to another derived-content family can reuse it.

The per-provider over-fetch multiplier for this search, in priority order: the caller's SearchParams.PermissionOverfetchFactor; else the largest of each resolved scope's declared ScopeConfig.permissionOverfetchFactor (a scope that declares none counts as the engine default); else the engine default. Clamped to [1, MAX_OVERFETCH_FACTOR].

The largest wins across scopes, not the first: over-fetch exists to compensate for late permission filtering, and a lane trimmed heavily by it needs the extra candidates whichever scope it belongs to. A larger factor never changes which results the caller gets (the final list is still trimmed to MaxResults), but it does cost more: every provider returns more candidates, dedup, the content exclusion and the permission passes do more work, a re-ranker is fed up to its inputTopN from a bigger pool, and streamSearch's per-provider partial events (capped to the caller's MaxResults) draw from it. This is why a scope may declare it at all: the scope's author knows how sparse its lanes are after permissions, and every caller shouldn't have to.

Deterministically serialize a value with object keys sorted, so that two logically-identical inputs always produce the same string.

JSON.stringify preserves insertion order, which means a caller that builds SecondaryScopes by spreading (a common pattern) can emit the same dimensions in different orders across calls. Left unsorted that causes avoidable cache misses; sorted, identity is stable. Array order is PRESERVED — see buildCacheKey for why ScopeIDs order is significant.

Second gate, after a result is known to be a readable row of its own entity: may the user also read the record the row was DERIVED from?

Row-level security on a content entity settles who may read the content table. It cannot say whether the person may read the file, task or conversation that content was extracted from — that permission lives on the origin entity, under its own row filters. Without this gate, a chunk of a document the user may not open passes the first gate and is quoted anyway.

For MJ: Content Items and MJ: Content Item Chunks (and any IS-A subtype of either) the default follows chunk → item (→ root item, for a split child) → MJ: Entity Record Documents → the origin record, and keeps a result only when the origin is a row the user may read, verified the way verifyOwnershipAndRowFilters verifies the result's own entity. A content row with no Entity Record Document (crawled, uploaded, external) has no origin and passes unchanged, as does every result of a non-content entity.

Every lookup runs as the user, and that is a NEW requirement on who may see content hits: read on the base MJ: Content Items for anyone who sees chunk hits (the chunk → item hop reads it), and read on MJ: Entity Record Documents for anyone who sees hits derived from a record (the item → document hop reads it). An app that indexes documents behind its own permissions grants those, under its own row filters. Without them the affected rows are dropped: every row of the group for the chunk and item hops, only the document-bearing rows for the document hop.

The origin's own permissions are checked once; an origin that is itself a content row is not walked again. Protected so a host can extend the rule to another derived-content family. Fails closed: a lookup that reports failure, or returns a row without a column it asked for (field-level security), drops the rows that depended on it; a lookup that throws drops the whole group. It never widens.