Member Junction
    Preparing search index...

    Interface RubricEvaluatorSettings

    Settings for one evaluation. They come from an agent-rubric link's EvaluatorConfig, a test's evaluator block, or the caller. Each evaluator reads the fields it understands and ignores the rest.

    interface RubricEvaluatorSettings {
        AgentID?: string;
        CriterionPromptID?: string;
        CriterionPromptName?: string;
        Extensions?: Record<string, RubricJsonValue>;
        Mode?: RubricPromptMode;
        ModelID?: string;
        ModelSelection?: RubricModelSelection;
        PromptID?: string;
        PromptName?: string;
        Samples?: number;
        SystemPromptID?: string;
        SystemPromptName?: string;
    }
    Index

    Properties

    AgentID? CriterionPromptID? CriterionPromptName? Extensions? Mode? ModelID? ModelSelection? PromptID? PromptName? Samples? SystemPromptID? SystemPromptName?

    Properties

    AgentID?: string

    The agent an Agent evaluator runs. Recorded with the evaluation.

    CriterionPromptID?: string

    The prompt that renders one criterion as text, for LLM and Decision. Default Rubric Criterion.

    CriterionPromptName?: string

    The criterion prompt by name.

    Extensions?: Record<string, RubricJsonValue>

    Settings for a custom evaluator, keyed by its evaluator name.

    Mode?: RubricPromptMode

    LLM only. SinglePass when omitted.

    ModelID?: string

    Pins the model. Otherwise the choosing prompt's model bindings choose.

    ModelSelection?: RubricModelSelection

    LLM only. System (default): the evaluator prompt's bindings choose the model. Judge: the judge's do.

    PromptID?: string

    The swappable prompt, by MJ: AI Prompts ID. Wins over PromptName. For LLM it is the judge: the child prompt rendered into the evaluator prompt's judgePrompt slot (default Rubric Evaluator - Default Judge). For Decision it is the decision prompt whose bindings choose the model (default Default Decision).

    PromptName?: string

    The swappable prompt by name, when no PromptID is set.

    Samples?: number

    LLM only. Runs the rubric this many times and keeps each criterion's median level.

    SystemPromptID?: string

    LLM only. The parent evaluator prompt, which owns the reply contract the engine parses. Default Rubric Evaluator. Replace it only with a prompt that returns the same JSON.

    SystemPromptName?: string

    LLM only. The parent evaluator prompt by name.

    Settings

    Member Visibility

    On This Page

    Properties