Member Junction
    Preparing search index...

    Class LLMJudgeOracle

    Interface for oracle implementations.

    An oracle is a source of truth that determines whether test output is correct. Oracles can be deterministic (schema validation, exact match) or heuristic (LLM judge).

    Each oracle type evaluates a specific aspect of test output:

    • SchemaValidator: Structural correctness
    • TraceValidator: Execution safety (no errors)
    • LLMJudge: Semantic quality
    • ExactMatch: Deterministic output comparison
    • SQLValidator: Database state verification

    IOracle

    export class SchemaValidatorOracle implements IOracle {
        readonly type = 'schema-validate';

        async evaluate(input: OracleInput, config: OracleConfig): Promise<OracleResult> {
            // Validation logic
            return { oracleType: this.type, passed: true, score: 1.0, message: 'Valid' };
        }
    }

    Implements

    Index

    Constructors

    constructor

    Properties

    type

    Methods

    evaluate

    Constructors

    Properties

    type: "llm-judge" = 'llm-judge'

    Unique type identifier for this oracle. Used in test configuration to specify which oracles to run.

    Methods

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties
    Methods