The registration key for this segmenter. Must match the key passed to
@RegisterClass so metadata-driven resolution round-trips.
Modalities this segmenter can produce. Used to validate configuration.
ProtectedresolveFill in defaults for any option the caller left unset.
Optionaloptions: SegmentationOptionsSegment content into embeddable units.
Never throws for content-shaped problems — inspect Success/ErrorMessage.
ProtectedSegmentProduce the raw, un-normalized segments for this content.
Implementations should focus purely on where the boundaries are; the base
class enforces the token ceiling afterwards, so returning a segment that is
too large is acceptable (it will be split, preserving Title and offsets).
ProtectedtokensStaticResolveResolve a registered segmenter by key via the MJ class factory. Returns null when no segmenter is registered under that key.
Uses TryCreateInstance rather than CreateInstance deliberately: the latter never returns
null for an unregistered key — it falls back to new BaseSegmenter(), a hollow object whose
abstract Key/SegmentCore are undefined. That failure stays invisible until something calls
it, so an unresolvable key must be reported as such here.
Emits one segment per page of a paginated source, preserving
PageNumber.Page boundaries are authored boundaries — an author decided where the page broke — which makes them a better split point than any inferred one, and they give citation-grade provenance: a retrieved chunk resolves to "page 14 of this PDF" rather than a character offset nobody can act on.
Each page may carry text, a media reference, or both. The media case is what enables embedding a PDF page as an image with a multimodal model — preserving tables, charts, and layout that text extraction flattens or loses entirely — while the extracted text rides along for lexical search and agent reasoning.
Pages are supplied by the caller via
SegmentationParams.Pages; this segmenter does no PDF parsing itself, keeping document-format dependencies out of the segmentation layer.