Member Junction
    Preparing search index...

    Interface MJContentSourceEntity_IContentCleaningOptions

    Options for the content-cleaning strategy that runs before segmentation.

    Garbage that survives cleaning is expensive: it gets embedded, stored, retrieved, and shown to a user or an agent. Navigation chrome repeated across a thousand pages produces a thousand near-identical vectors that crowd out real answers.

    interface MJContentSourceEntity_IContentCleaningOptions {
        ExcludeSelectors?: string[];
        IncludeImageAltText?: boolean;
        IncludeSelectors?: string[];
        MaxLength?: number;
        NormalizeWhitespace?: boolean;
        ReplaceDefaultExcludes?: boolean;
    }
    Index

    Properties

    ExcludeSelectors? IncludeImageAltText? IncludeSelectors? MaxLength? NormalizeWhitespace? ReplaceDefaultExcludes?

    Properties

    ExcludeSelectors?: string[]

    CSS selectors to remove, applied after IncludeSelectors (inline ad slots, share widgets).

    IncludeImageAltText?: boolean

    Html cleaner: keep alt text from images as content.

    IncludeSelectors?: string[]

    CSS selectors whose content is the ONLY content to keep. The highest-leverage knob: naming the element that holds the article (e.g. '.article-body', 'main') discards navigation, sidebars, and advertising without enumerating what to drop.

    MaxLength?: number

    Maximum characters to retain after cleaning.

    NormalizeWhitespace?: boolean

    Collapse runs of whitespace and blank lines. Default true.

    ReplaceDefaultExcludes?: boolean

    Html cleaner: replace the built-in exclusion list rather than appending to it.

    Settings

    Member Visibility

    On This Page

    Properties