The registered name. Stored on the evaluation as EvaluatorName.
Stored on the evaluation as EvaluatorType. A custom evaluator that is not an AI prompt or agent is External.
False for an evaluator a person completes, such as Human. The engine does not run those.
Scores the candidates against the version and returns the normalized score, the combined rationale, and the evidence refs.
Scores one subject. Throw to fail the evaluation; the message is stored on it.
Scores a rubric with a Decision-type model, such as Jev, or LLM Decision for a chat model run as a decision. Every leaf with a level scale becomes a typed Score question, and all of them go in one call, so the model answers with a calibrated probability per level instead of free text.
Each question's instructions are the criterion as the criterion prompt (
Rubric Criterion, or CriterionPromptID / CriterionPromptName) renders it, the same text the LLM evaluator shows. The decision prompt is the settings' PromptID or PromptName, else Default Decision, and its model bindings choose the model. ModelID pins one.A decision model returns no rationale and no quotes. Each answer's rationale states the chosen level and its probability. A leaf that requires evidence is refused before any call, because the submit would reject it. A leaf with a numeric scale cannot be asked as a Score question; it is left unanswered and listed in the run metadata as UnaskedCriteria.