ReadonlytypeUnique type identifier for this oracle. Used in test configuration to specify which oracles to run.
Evaluate the test output.
Evaluation input (test, expected/actual outputs, target entity, user)
Oracle-specific configuration
Promise resolving to oracle result
discovery-label-match: scores an agent-discovery run against its label, for either arm.agentlabel is correct when the chosen agent is the labelled agent: the Choice's pick for the decision, whatever its confidence, or the first rowFind Candidate Agentswould return for thesemantic-searchbaseline.nonelabel is correct when discovery would not inject a suggestion (the run'sWouldInject): production's own verdict,DecisionDiscoveryFromResult, not confident once the answers are calibrated for the answering model, or that model having no calibration, or the discovery finishing after production's timeout (decision); or no candidate reaching the action's similarity floor (baseline).correct: null.detailscarry{ arm, label, kind, expectedAgentId, chosenAgentId, confidence, anyApplies, wouldInject, topScore, correct }, so the scorecard can aggregate without re-running anything. It takes no configuration.