Interface IContentSourceClassificationConfiguration

How chunk text + vectors are stored for this source. Default 'alwaysChunk'.

• 'alwaysChunk' (default): every content item gets at least one ContentItemChunk row holding its text — even items small enough to fit in a single chunk — and ContentItem.VectorRecordID is never set. The ContentItemChunk table is always the single source of truth for vectors.
• 'mixed': items that fit in a single chunk keep their text and vector id on the ContentItem (no chunk row); only larger items are split into ContentItemChunk rows.

Options passed to the cleaning strategy named by CleanerKey. Selector rules are per-source because the right selector is a property of the site's template.

Enable vectorization for this source. Default true

Maximum cumulative cost (USD) the run may incur before pausing. NULL/unset = unlimited.

Maximum number of content items the autotagger may PROCESS (hand to the LLM) per run before the run is paused via the existing CancellationRequested machinery. Does not include items skipped by change-detection — those are free. NULL/unset = unlimited.

Most intuitive "do at most N this run, do the rest next time" knob. When checking budgets after a batch, this is evaluated FIRST (before tag / token / cost caps) because it is the most user-facing and not tied to a specific model's pricing or tokenization.

Pause is graceful — the next invocation re-crawls, change-detection skips the items already tagged in DB, and the remaining items get processed.

Maximum number of new tags the autotagger may auto-create for a single ContentItem. Once reached, further free-text tags from that item are routed to MJ:Tag Suggestions with Reason='MaxItemTagsExceeded' instead of being created. NULL/unset = unlimited.

Maximum number of new tags the autotagger may auto-create across an entire run before the run is paused via the existing CancellationRequested machinery. NULL/unset = unlimited. Pause is graceful — the run resumes from LastProcessedOffset when restarted.

Maximum cumulative LLM tokens (prompt + completion) the run may consume before pausing. Reads from ContentProcessRunDetail.TotalTokensUsed rollup. NULL/unset = unlimited.

Options passed to the segmentation strategy named by SegmenterKey.

Sizing note: TargetTokens should be driven by the shape of your QUERIES, not by the embedding model's context window. The window is an upper bound; a good chunk is about as much content as a good answer, so that a matching chunk is mostly signal.

Whether to share existing tag taxonomy with the LLM during autotagging. Default true

Source-type-specific configuration values. The keys here correspond to the RequiredFields[].Key values defined on the parent ContentSourceType's Configuration.

Examples:

• Entity type: { EntityID: "uuid", EntityDocumentID: "uuid" }
• RSS Feed: { URL: "https://example.com/feed.xml" }
• Cloud Storage: { FileStorageProviderKey: "Azure Blob Storage", PathPrefix: "/documents" }
• Local File System: { Path: "/var/data/documents" }
• Website: { URL: "https://example.com" } — see Website sub-object below for crawl knobs

Lower confidence band (0.0-1.0) that routes a semantic match into the human-in-the-loop MJ:Tag Suggestions queue instead of auto-applying or auto-creating. A score s is routed as: s >= TagMatchThreshold → apply; SuggestThreshold <= s < TagMatchThreshold → enqueue suggestion (Reason='BelowThreshold'); s < SuggestThreshold → fall through to handleNoMatch (governed by TagTaxonomyMode). When unset, defaults to TagMatchThreshold - 0.05 at runtime.

Similarity threshold (0.0-1.0) for matching ContentItemTags to formal Tags. Default 0.9

Root Tag ID for constrained/auto-grow modes — limits taxonomy operations to this subtree

Tag taxonomy matching mode: constrained (only match within subtree), auto-grow (match or create within subtree), free-flow (match or create anywhere)

Vector-database record-id strategy for this source's chunks. Default 'recordId'.

• 'recordId' (default, recommended): each ContentItemChunk's unique RecordID is used as its vector-DB record id. Safe with the soft-delete + PurgeDeletedChunks flow — a re-chunk mints new rows with new ids, so a superseded (soft-deleted) chunk and its replacement never share a vector id, and purging the old one can't orphan the live chunk's vector.
• 'hash': a deterministic hash of the parent content item id (5.49 EntityDocument parity). NOT safe with re-chunking + purge — a replacement chunk reuses the superseded chunk's id, so purging the old chunk would delete the live chunk's vector. Use only for sources that are never re-chunked or purged.

Controls what goes into each vector's metadata. Vector-store metadata has real storage + performance cost, so this lets a source keep it minimal. Falls back to the ContentType's default, then 'default'.

Website-crawler settings — only meaningful for content sources whose ContentSourceType is "Website". Replaces the legacy per-key ContentSourceParam rows; AutotagWebsite reads from this sub-object first and falls back to ContentSourceParam rows for sources configured before this field existed.

In the future, source-type-specific knobs like these may move to a pluggable per-source-type sub-interface scheme (one named property per source type). This is the first opt-in implementation; other source types will follow the same pattern as their knobs grow.