Member Junction
    Preparing search index...

    Interface ComputerUseExpectedOutcomes

    Expected outcomes for a Computer Use test. Stored as JSON in TestEntity.ExpectedOutcomes.

    interface ComputerUseExpectedOutcomes {
        checkpoints?: CheckpointDef[];
        finalUrlPattern?: string;
        goalCompleted?: boolean;
        judgeValidationCriteria?: string[];
        maxSteps?: number;
        minConfidence?: number;
    }
    Index

    Properties

    checkpoints? finalUrlPattern? goalCompleted? judgeValidationCriteria? maxSteps? minConfidence?

    Properties

    checkpoints?: CheckpointDef[]

    Ordered tour checkpoints. When set, the test is verified section-by-section: it passes iff every checkpoint is reached (URL/element assertions latch for free every step; visual criteria via the judge), rather than on a single final-frame judge. Mapped to RunComputerUseParams.Checkpoints. See plans/regression-testing/checkpoint-tours-design.md.

    finalUrlPattern?: string

    Regex pattern that the final URL must match. Tested with new RegExp(pattern).test(finalUrl).

    goalCompleted?: boolean

    Whether the goal should be completed (default: true)

    judgeValidationCriteria?: string[]

    Custom validation criteria for LLM judge oracles

    maxSteps?: number

    Maximum number of steps allowed (exceeding = fail)

    minConfidence?: number

    Minimum judge confidence for a passing result (0.0 to 1.0)

    Settings

    Member Visibility

    On This Page

    Properties