Class ComputerUseTestDriver

Test driver for Computer Use browser automation tests.

Orchestrates:

Parsing test configuration → MJRunComputerUseParams
Executing MJComputerUseEngine.Run() with timeout
Running oracles (built-in + global registry)
Calculating score and status
Returning DriverExecutionResult

Hierarchy (View Summary)

BaseTestDriver
- ComputerUseTestDriver

Index

Constructors

constructor

new ComputerUseTestDriver(): ComputerUseTestDriver
Returns ComputerUseTestDriver
Inherited from BaseTestDriver.constructor

Properties

`Protected`_provider

_provider: IMetadataProvider | null

Metadata provider used by the driver for entity access. Set explicitly via the Provider setter; falls back to the global Metadata.Provider when not set. The engine (or test harness) should set this to thread a transaction-scoped provider.

Accessors

`Protected`_metadata

get _metadata(): Metadata
Returns Metadata
Deprecated
Use _provider (IMetadataProvider) instead. Retained for backward compatibility with subclasses that referenced the previous _metadata: Metadata field directly. Returns the same underlying provider cast to the legacy Metadata shape.
Inherited from BaseTestDriver._metadata
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:68

Provider

get Provider(): IMetadataProvider
The metadata provider this driver uses. Falls back to the global Metadata.Provider when the engine hasn't injected an explicit provider — multi-tenant servers should always set this so each driver run binds to the right database connection.

Returns IMetadataProvider
Inherited from BaseTestDriver.Provider
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:57
set Provider(value: IMetadataProvider | null): void
Parameters
- value: IMetadataProvider | null
Returns void
Inherited from BaseTestDriver.Provider
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:58

Methods

`Protected`calculateScore

calculateScore(oracleResults: OracleResult[], weights?: ScoringWeights): number
Protected
Calculate overall score from oracle results.

If weights are provided, calculates weighted average. Otherwise, calculates simple average.
Parameters
- oracleResults: OracleResult[]
  Results from oracle evaluations
- Optionalweights: ScoringWeights
  Optional scoring weights by oracle type
Returns number
Overall score from 0.0 to 1.0
Inherited from BaseTestDriver.calculateScore
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:110

`Protected`createLogMessage

createLogMessage(
    level: "info" | "warn" | "error" | "debug",
    message: string,
    metadata?: Record<string, unknown>,
): TestLogMessage
Protected
Create a log message for the test execution log.
Parameters
- level: "info" | "warn" | "error" | "debug"
  Log level
- message: string
  Log message
- Optionalmetadata: Record<string, unknown>
  Optional metadata
Returns TestLogMessage
TestLogMessage object
Inherited from BaseTestDriver.createLogMessage
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:235

`Protected`determineStatus

determineStatus(oracleResults: OracleResult[]): "Failed" | "Passed"
Protected
Determine overall test status from oracle results.

Test passes only if ALL oracles pass.
Parameters
- oracleResults: OracleResult[]
  Results from oracle evaluations
Returns "Failed" | "Passed"
'Passed' if all oracles passed, 'Failed' otherwise
Inherited from BaseTestDriver.determineStatus
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:120

Execute

Execute(context: DriverExecutionContext): Promise<DriverExecutionResult>
Execute a Computer Use test.

Steps:
1. Parse Configuration, InputDefinition, ExpectedOutcomes
2. Build MJRunComputerUseParams from parsed data
3. Execute MJComputerUseEngine.Run() with timeout
4. Build actual output from ComputerUseResult
5. Run oracles (built-in + from context registry)
6. Calculate score, determine status
7. Return DriverExecutionResult
Parameters
- context: DriverExecutionContext
Returns Promise<DriverExecutionResult>
Overrides BaseTestDriver.Execute
- Defined in AI/MJComputerUse/src/test-driver/ComputerUseTestDriver.ts:134

`Protected`getEffectiveTimeout

getEffectiveTimeout(
test: MJTestEntity,
config?: { maxExecutionTime?: number },
): number
Protected
Get the effective timeout for a test.

Priority (highest to lowest):
1. Configuration JSON maxExecutionTime field (backward compatibility)
2. Test.MaxExecutionTimeMS column
3. DEFAULT_TEST_TIMEOUT_MS constant (5 minutes)
Parameters
- test: MJTestEntity
  The test entity
- Optionalconfig: { maxExecutionTime?: number }
  Parsed configuration object (optional)
Returns number
Timeout in milliseconds
Inherited from BaseTestDriver.getEffectiveTimeout
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:223

`Protected`log

log(message: string, verboseOnly?: boolean): void
Protected
Log execution progress.
Parameters
- message: string
  Log message
- OptionalverboseOnly: boolean
  Whether to only log in verbose mode (default: false)
Returns void
Inherited from BaseTestDriver.log
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:161

`Protected`logError

logError(message: string, error?: Error): void
Protected
Log errors.
Parameters
- message: string
  Error message
- Optionalerror: Error
  Optional error object
Returns void
Inherited from BaseTestDriver.logError
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:169

`Protected`logToTestRun

logToTestRun(
    context: DriverExecutionContext,
    level: "info" | "warn" | "error" | "debug",
    message: string,
    metadata?: Record<string, unknown>,
): void
Protected
Log a message to both the console (if verbose) and accumulate for test run log.
Parameters
- context: DriverExecutionContext
  Driver execution context
- level: "info" | "warn" | "error" | "debug"
  Log level
- message: string
  Log message
- Optionalmetadata: Record<string, unknown>
  Optional metadata
Returns void
Inherited from BaseTestDriver.logToTestRun
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:245

`Protected`parseConfig

parseConfig<T>(test: MJTestEntity): T
Protected
Parse and validate Configuration JSON.

Helper method for drivers to parse their configuration with type safety. Throws if configuration is invalid.
Type Parameters
- T
  The configuration type
Parameters
- test: MJTestEntity
  The test containing the configuration
Returns T
Parsed configuration

Throws
Error if configuration is missing or invalid
Inherited from BaseTestDriver.parseConfig
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:133

`Protected`parseExpectedOutcomes

parseExpectedOutcomes<T>(test: MJTestEntity): T
Protected
Parse and validate ExpectedOutcomes JSON.
Type Parameters
- T
  The expected outcomes type
Parameters
- test: MJTestEntity
  The test containing the expected outcomes
Returns T
Parsed expected outcomes

Throws
Error if expected outcomes is missing or invalid
Inherited from BaseTestDriver.parseExpectedOutcomes
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:153

`Protected`parseInputDefinition

parseInputDefinition<T>(test: MJTestEntity): T
Protected
Parse and validate InputDefinition JSON.
Type Parameters
- T
  The input definition type
Parameters
- test: MJTestEntity
  The test containing the input definition
Returns T
Parsed input definition

Throws
Error if input definition is missing or invalid
Inherited from BaseTestDriver.parseInputDefinition
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:143

SetProvider

SetProvider(provider: IMetadataProvider): void
Parameters
- provider: IMetadataProvider
Returns void
Deprecated
Use the Provider setter instead. Retained for backward compatibility.
Inherited from BaseTestDriver.SetProvider
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:62

SetupSuite

SetupSuite(context: SuiteFixtureContext, contextUser: UserInfo): Promise<void>
Suite-scoped setup. Runs ONCE after the suite run is created and before the first test's Execute, when the test runs inside a suite (mj test suite). Provision suite-shared fixtures here — discover/create users, create Query/Category rows, refresh engine caches — and stash them on context.Data / context.CreatedRecords so every Execute of the suite (which receives this same context via DriverExecutionContext.fixtures) can read them, and TeardownSuite can clean them up.

Default is a no-op, so existing drivers (AgentEval, Computer Use) are unaffected. Does NOT fire for the standalone mj test run path (no suite).
Parameters
- context: SuiteFixtureContext
  Per-suite-run fixture bag, keyed by SuiteRunID
- contextUser: UserInfo
  User context for data access
Returns Promise<void>
Inherited from BaseTestDriver.SetupSuite
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:196

supportsCancellation

supportsCancellation(): boolean
Returns true — this driver supports cancellation via engine.Stop().

Returns boolean
Overrides BaseTestDriver.supportsCancellation
- Defined in AI/MJComputerUse/src/test-driver/ComputerUseTestDriver.ts:118

TeardownSuite

TeardownSuite(
context: SuiteFixtureContext,
contextUser: UserInfo,
): Promise<void>
Suite-scoped teardown. GUARANTEED by TestEngine.RunSuite via a finally, so it runs on pass, fail, a thrown Execute, and after a 'Timeout' result. MUST be best-effort: never throw (the engine logs and ignores any throw), only clean up. Delete what SetupSuite created (the context.CreatedRecords list and any driver-specific payload under context.Data).

Default is a no-op. Does NOT fire for the standalone mj test run path.
Parameters
- context: SuiteFixtureContext
  The same per-suite-run fixture bag passed to SetupSuite
- contextUser: UserInfo
  User context for data access
Returns Promise<void>
Inherited from BaseTestDriver.TeardownSuite
- Defined in TestingFramework/Engine/dist/drivers/BaseTestDriver.d.ts:209

Validate

Validate(test: MJTestEntity): Promise<ValidationResult>
Validate Computer Use test configuration.

Checks:
- Base JSON validation
- InputDefinition has a goal
- Oracle types are valid
- URL patterns are valid regexes
- Scoring weights sum to ~1.0
Parameters
- test: MJTestEntity
Returns Promise<ValidationResult>
Overrides BaseTestDriver.Validate
- Defined in AI/MJComputerUse/src/test-driver/ComputerUseTestDriver.ts:266

Class ComputerUseTestDriver

Hierarchy (View Summary)

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Returns ComputerUseTestDriver

Properties

Protected_provider

Accessors

Protected_metadata

Returns Metadata

Deprecated

Provider

Returns IMetadataProvider

Parameters

Returns void

Methods

ProtectedcalculateScore

Parameters

Returns number

ProtectedcreateLogMessage

Parameters

Returns TestLogMessage

ProtecteddetermineStatus

Parameters

Returns "Failed" | "Passed"

Execute

Parameters

Returns Promise<DriverExecutionResult>

ProtectedgetEffectiveTimeout

Parameters

Returns number

Protectedlog

Parameters

Returns void

ProtectedlogError

Parameters

Returns void

ProtectedlogToTestRun

Parameters

Returns void

ProtectedparseConfig

Type Parameters

Parameters

Returns T

Throws

ProtectedparseExpectedOutcomes

Type Parameters

Parameters

Returns T

Throws

ProtectedparseInputDefinition

Type Parameters

Parameters

Returns T

Throws

SetProvider

Parameters

Returns void

Deprecated

SetupSuite

Parameters

Returns Promise<void>

supportsCancellation

Returns boolean

TeardownSuite

Parameters

Returns Promise<void>

Validate

Parameters

Returns Promise<ValidationResult>

Settings

On This Page

`Protected`_provider

`Protected`_metadata

`Protected`calculateScore

`Protected`createLogMessage

`Protected`determineStatus

`Protected`getEffectiveTimeout

`Protected`log

`Protected`logError

`Protected`logToTestRun

`Protected`parseConfig

`Protected`parseExpectedOutcomes

`Protected`parseInputDefinition