Member Junction
    Preparing search index...

    Class AgentEvalDriver

    Abstract base class for test driver implementations.

    Each TestType in the database has a corresponding DriverClass that extends this base. The driver is responsible for:

    • Parsing test-specific configuration from Configuration JSON
    • Executing the test with appropriate logic
    • Running oracles to evaluate results
    • Calculating scores and determining pass/fail status
    • Returning structured results

    BaseTestDriver handles common functionality:

    • Configuration parsing
    • Score calculation
    • Status determination
    • Logging
    • Error handling

    Follows pattern from BaseScheduledJob and BaseAgent.

    @RegisterClass(BaseTestDriver, 'AgentEvalDriver')
    export class AgentEvalDriver extends BaseTestDriver {
        async Execute(context: DriverExecutionContext): Promise<DriverExecutionResult> {
            const config = this.parseConfig<AgentEvalConfig>(context.test);
            // Execute test logic
            return result;
        }
    }

    Hierarchy (View Summary)

    Index

    Constructors

    constructor

    Properties

    _provider

    Accessors

    _metadata Provider

    Methods

    calculateScore createLogMessage determineStatus Execute getEffectiveTimeout LoadAgentEvaluationRubric LoadSuites log logError logToTestRun LookupLatestPublished LookupVersionLabel parseConfig parseExpectedOutcomes parseInputDefinition ReadMany ReadOne SetProvider SetupSuite supportsCancellation TeardownSuite Validate WithResolvedRubric

    Constructors

    Properties

    Protected_provider

    _provider: IMetadataProvider | null = null

    Metadata provider used by the driver for entity access. Set explicitly via the Provider setter; falls back to the global Metadata.Provider when not set. The engine (or test harness) should set this to thread a transaction-scoped provider.

    Accessors

    Protected_metadata

    Methods

    • Protected

      Get the effective timeout for a test.

      Priority (highest to lowest):

      1. Configuration JSON maxExecutionTime field (backward compatibility)
      2. Test.MaxExecutionTimeMS column
      3. DEFAULT_TEST_TIMEOUT_MS constant (5 minutes)

      Parameters

      • test: MJTestEntity

        The test entity

      • Optionalconfig: { maxExecutionTime?: number }

        Parsed configuration object (optional)

      Returns number

      Timeout in milliseconds

    • Suite-scoped teardown. GUARANTEED by TestEngine.RunSuite via a finally, so it runs on pass, fail, a thrown Execute, and after a 'Timeout' result. MUST be best-effort: never throw (the engine logs and ignores any throw), only clean up. Delete what SetupSuite created (the context.CreatedRecords list and any driver-specific payload under context.Data).

      Default is a no-op. Does NOT fire for the standalone mj test run path.

      Parameters

      Returns Promise<void>

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties
    Accessors
    Methods