Member Junction
    Preparing search index...

    Interface MJContentProcessRunEntity_IContentProcessRunConfiguration

    Pipeline configuration stored on ContentProcessRun.Configuration. Controls batch size, rate limiting, error thresholds, and duplicate detection.

    interface MJContentProcessRunEntity_IContentProcessRunConfiguration {
        Pipeline?: {
            BatchSize?: number;
            DelayBetweenBatchesMs?: number;
            ErrorThresholdPercent?: number;
            MaxConcurrentBatches?: number;
            ResumeFromLastBatch?: boolean;
        };
        RateLimits?: {
            Embedding?: { RequestsPerMinute?: number; TokensPerMinute?: number };
            LLM?: { RequestsPerMinute?: number; TokensPerMinute?: number };
            VectorDB?: { RequestsPerMinute?: number };
        };
    }
    Index

    Properties

    Pipeline? RateLimits?

    Properties

    Pipeline?: {
        BatchSize?: number;
        DelayBetweenBatchesMs?: number;
        ErrorThresholdPercent?: number;
        MaxConcurrentBatches?: number;
        ResumeFromLastBatch?: boolean;
    }

    Batch processing settings

    Type Declaration

    • OptionalBatchSize?: number

      Number of content items per batch. Default: 100

    • OptionalDelayBetweenBatchesMs?: number

      Delay between batches in milliseconds. Default: 200

    • OptionalErrorThresholdPercent?: number

      Error rate percentage that triggers the circuit breaker (0-100). Default: 20

    • OptionalMaxConcurrentBatches?: number

      Maximum concurrent batches. Default: 1

    • OptionalResumeFromLastBatch?: boolean

      If true, resume from the last completed batch on restart. Default: true

    RateLimits?: {
        Embedding?: { RequestsPerMinute?: number; TokensPerMinute?: number };
        LLM?: { RequestsPerMinute?: number; TokensPerMinute?: number };
        VectorDB?: { RequestsPerMinute?: number };
    }

    API rate limiting per provider

    Type Declaration

    • OptionalEmbedding?: { RequestsPerMinute?: number; TokensPerMinute?: number }

      Embedding rate limits

      • OptionalRequestsPerMinute?: number

        Maximum embedding requests per minute

      • OptionalTokensPerMinute?: number

        Maximum embedding tokens per minute

    • OptionalLLM?: { RequestsPerMinute?: number; TokensPerMinute?: number }

      LLM (tagging) rate limits

      • OptionalRequestsPerMinute?: number

        Maximum LLM requests per minute

      • OptionalTokensPerMinute?: number

        Maximum LLM tokens per minute

    • OptionalVectorDB?: { RequestsPerMinute?: number }

      Vector database rate limits

      • OptionalRequestsPerMinute?: number

        Maximum vector DB requests per minute

    Settings

    Member Visibility

    On This Page

    Properties