Class EntityVectorSyncer

Returns the active metadata provider — explicit override (via this.Provider = ...) if set, otherwise the global default. Provided as ProviderToUse for backward compatibility with code paths that referenced this name before the base class standardized on the Provider getter/setter.

Adds configured display fields with type-aware storage: numeric SQL types stay numeric, date fields can convert to epoch seconds/milliseconds via storeAs, uniqueidentifier values are lowercase-normalized, and long strings are truncated.

Adds EntityIcon: opt-out by default, opt-in under the 'explicit' strategy.

Adds the system-injected RecordID / Entity / TemplateID keys — omitted entirely under the 'explicit' strategy, where metadata contains only configured fields.

Adds __mj_UpdatedAt for recency sorting: opt-out by default, opt-in under the 'explicit' strategy.

Refuse a batch whose vectors are not the width the index accepts, before a single record is upserted.

The Dimensions argument threaded into RunEmbedding is a request hint, not a contract: only some providers honour it (OpenAI's text-embedding-3-* family does), a local ONNX model is free to ignore it entirely, and nothing downstream ever compared what came back against what the index accepts. The consequences split by provider and both are bad:

• A provider that enforces width rejects the upsert. Pinecone's own driver already knows this failure by name — see the comment on its CreateRecords catch block quoting "Vector dimension 1536 does not match the dimension of the index 512" — but it only learns it after the API round trip, per batch, for the whole run.
• A provider that does not enforce width accepts them, and the index silently ends up holding vectors of two widths, which makes similarity search quietly meaningless.

Throwing is deliberate and not per-record: every vector in the run comes from the same model, so a first-batch mismatch is a run-level misconfiguration. Continuing would spend the embedding budget on vectors that cannot be stored.

Skipped when the index declares no width — see ResolveIndexDimensions for when that is legitimate.

Build the SQL filter to select records that are in the given list. For single PK entities, uses a simple IN clause. For composite PK entities, uses an EXISTS clause that concatenates PK columns to match the RecordID format.

The record-id string persisted to MJ: Entity Record Documents.RecordID and carried through the pipeline as __mj_recordID: the entity's actual primary key column(s) read off the row, in compact URL-segment form — the bare value for a single-column key (whatever that column is called), F1|v1||F2|v2 for a composite key. CompositeKey.FromURLSegment reads either form back, so a customer entity keyed by individual_id or (OrderID, LineNo) round-trips instead of being reduced to whichever column happens to be listed first.

The filter on the records a run vectorizes: the list's members when the run is for a list, and the entity document's record filter (Configuration.recordFilter.extraFilter) when one is set, ANDed together. Undefined when neither applies.

Build the vector metadata object for one embedding item.

Under 'all' / 'include' / 'exclude' strategies (the historical behavior), the system keys RecordID / Entity / TemplateID are always present, and EntityIcon / __mj_UpdatedAt default ON (opt-out via includeEntityIcon / includeUpdatedAt).

Under the 'explicit' strategy, metadata contains EXACTLY the configured fields: no system keys, and EntityIcon / __mj_UpdatedAt flip to opt-in (default false).

protected and decomposed into focused add*Metadata steps (each also protected) so a subclass can override just the piece it cares about — e.g. a custom EntityIcon resolution, or different chunk-aware display-field handling — without having to reimplement this whole method.

Embeds one batch of rendered texts on the entity document's model. The ModelID pin keeps the vector index single-model: unpinned, the runner may answer from any Embeddings model. embeddingDimensions is the index's MJ: Vector Indexes.Dimensions (undefined for the model's native size).

The related-entity rows a page of records' templates render; see EntityDocumentTemplateDataBuilder.LoadRelatedData.

The data one record's template renders. Built by EntityDocumentTemplateDataBuilder, which duplicate detection renders with too, so the text it queries with matches the text stored here.

Resolve the embedding driver and vector-DB driver for an Entity Document.

Pick the embedding model: the index's when we have an index, the document's otherwise.

When both are set and they disagree, the index wins and the disagreement is logged. The disagreement is a genuine misconfiguration — one of the two is going to produce vectors the index cannot hold — but it is the operator's to fix, and failing the run would strand every tenant whose document and index drifted apart while still embedding correctly.

The width this index accepts, or null when the index legitimately has none.

Order of preference:

1. VectorIndex.Dimensions when it is set. This is the operator's declaration, or the width MJVectorIndexEntityServer learned by probing the embedding model at create time and wrote back once the provider accepted the index. It is what the embedding request honours.
2. Fallback only: the provider's own answer, via ListIndexes(), written back onto the row. This covers rows whose Dimensions is still null: indexes created before the create-time probe existed (the Knowledge Hub create-index form never set it, and the old write-back never ran when the provider call failed), and a row read before its detached provisioning has finished writing back. Refusing those rows outright would strand them, so the provider is asked first and the row is repaired.
3. Refuse. An index whose width nobody can state cannot have embeddings checked against it, and that is the exact state the check exists to prevent.

null is returned only for a driver that owns no index object (VectorDBBase.ManagesIndexes === false): for SimpleVectorServiceProvider the "index" is a logical pairing and the vectors live in MJ: Entity Record Documents.VectorJSON, a JSON column with no fixed width. There is nothing to check against and nothing to refuse.

Resolves the API key for a vector database provider. Checks the Credential Engine first (if VectorDatabase.CredentialID is set), then falls back to the legacy environment variable AI_VENDOR_API_KEY__.

Creates or updates Entity Record Documents for a batch of source records.

Reads are batched + parallel, saves are per-record. Previously each record did its own RunView to find its existing ERD — an N+1 read storm (~1 query per source record) that dominated the upsert phase and tripped the sequential/duplicate-RunView telemetry. Here we group by (EntityID, EntityDocumentID) and issue the existence checks for ALL groups in a SINGLE RunViews batch (one round trip, run in parallel) with a RecordID IN (…) filter per group, map the results by RecordID, then find-or-create + Save() each record (the per-record Save() is inherent to the BaseEntity contract). Grouping is usually a single read within one entity's pipeline; a mixed batch produces several views, now executed together.

Creates or updates the Entity Record Document for a single source record. Thin back-compat wrapper over UpsertEntityRecordDocumentBatch; the pipeline uses the batch path directly to avoid an N+1 read per record.

This method is resposnible for determining if the template(s) given have aligned parameters, meaning they don't have overlapping parameter names that have different meanings. It is okay for scenarios where there are > 1 template in use for a message to have different parameter names, but if they have the SAME parameter names they must not have different settings.