The class factory passes no runner. EvaluateRubric builds one from the prompt service.
Optionalrunner: RubricPromptRunnerThe registered name. Stored on the evaluation as EvaluatorName.
Stored on the evaluation as EvaluatorType. A custom evaluator that is not an AI prompt or agent is External.
False for an evaluator a person completes, such as Human. The engine does not run those.
Scores the candidates against the version and returns the normalized score, the combined rationale, and the evidence refs.
Runs the prompt and scores the accepted answers with RubricScoring. criteria is the template
data, already carrying each criterion's rendered Text; it defaults to data with no Text.
Renders the criteria, runs the evaluator prompt with its judge, and records each prompt run.
Runs the rubric n times. Each criterion keeps the median level by normalized value, and the result records every level that appeared. Scoring runs once, on those median answers.
Scores a rubric with a chat model, through prompts stored as metadata. Nothing here writes prompt text: the engine builds template data and the prompts render it.
Rubric Criterion): renders each leaf intoCriterion.Text. The Decision evaluator uses the same text, so both describe a criterion identically.Rubric Evaluator, the parent): the rubric, the reply contract the engine parses, and ajudgePromptslot. Swap it only with a prompt that returns the same JSON.Rubric Evaluator - Default Judge, the child): how to judge. This is the one a link swaps with PromptID or PromptName, for example a judge written for one agent.The parent and judge render into one system message, the way a loop agent's system prompt and agent prompt do, in one model call with one prompt run. The subject is a separate user message inside a nonce delimiter, so it never passes through a template.
SinglePass asks once for the whole rubric. PerCriterion asks once per leaf, and confidence is the probability of the chosen level. Samples runs the rubric several times and keeps each criterion's median level. Unknown keys are dropped and counted. Levels map by label. A numeric value outside the scale throws before scoring. A quote that is not in the subject text is dropped.