Member Junction
    Preparing search index...

    Interface TestRunOptions

    Options for running a single test

    interface TestRunOptions {
        agentEvaluationRubricId?: string;
        agentVersion?: string;
        configOverride?: Record<string, unknown>;
        dryRun?: boolean;
        environment?: string;
        gitCommit?: string;
        logCallback?: (message: TestLogMessage) => void;
        progressCallback?: (progress: TestProgress) => void;
        publishedRubricVersionId?: string;
        repeatCountOverride?: number;
        rubricId?: string;
        rubricVersionId?: string;
        tags?: string;
        variables?: TestRunVariables;
        verbose?: boolean;
    }

    Hierarchy (View Summary)

    Index

    Properties

    agentEvaluationRubricId? agentVersion? configOverride? dryRun? environment? gitCommit? logCallback? progressCallback? publishedRubricVersionId? repeatCountOverride? rubricId? rubricVersionId? tags? variables? verbose?

    Properties

    agentEvaluationRubricId?: string

    The target agent's default Evaluation rubric, used when nothing earlier names one.

    agentVersion?: string

    Agent/system version being tested

    configOverride?: Record<string, unknown>

    Override test configuration

    dryRun?: boolean

    Validate configuration without executing

    environment?: string

    Environment context (dev, staging, prod)

    gitCommit?: string

    Git commit SHA for versioning

    logCallback?: (message: TestLogMessage) => void

    Log callback for streaming execution details to the test run log

    progressCallback?: (progress: TestProgress) => void

    Progress callback for real-time updates

    publishedRubricVersionId?: string

    Latest published version observed at suite-run start, used to pin the suite.

    repeatCountOverride?: number

    Override the test's RepeatCount field at runtime. Useful for flaky-test detection: forces every test in the run to execute N times so the suite runner can compute score variance. If undefined, the test's own RepeatCount is used.

    rubricId?: string

    Experiment override for the rubric that judges this run. A version here is explicit and is not replaced by the suite pin.

    rubricVersionId?: string

    Explicit rubric version id. When omitted, the suite run pins the latest published version.

    tags?: string

    Tags to apply to the test run (JSON string array)

    variables?: TestRunVariables

    Variable values to use for this run. These values take highest priority in the resolution order: run > suite > test > type

    verbose?: boolean

    Verbose logging

    Settings

    Member Visibility

    On This Page

    Properties