Function EstimateDiscoveryEvalRun

EstimateDiscoveryEvalRun(
    cases: readonly LabelledDiscoveryRequest[],
    cells: readonly {
        baseline?: "semantic-search";
        failover?: boolean;
        inputPricePer1M?: number;
        label: string;
        modelId?: string;
        outputPricePer1M?: number;
        promptName?: string;
        seed?: number;
        temperature?: number;
        vendorId?: string;
    }[],
    reps: number,
    catalog: readonly { Description: string; ID: string; Name: string }[],
): DecisionEvalRunEstimate
Estimates the run's tokens and cost. A decision call's prompt is the request and the questions over the catalog snapshot's agents (at most DECISION_DISCOVERY_MAX_OPTIONS), at four characters a token, plus a fixed allowance. A baseline cell makes no model call. A floor, not a ceiling.
Parameters
- cases: readonly LabelledDiscoveryRequest[]
  The labelled requests.
- cells: readonly {
      baseline?: "semantic-search";
      failover?: boolean;
      inputPricePer1M?: number;
      label: string;
      modelId?: string;
      outputPricePer1M?: number;
      promptName?: string;
      seed?: number;
      temperature?: number;
      vendorId?: string;
  }[]
  The matrix cells.
- reps: number
  Repeats per decision record.
- catalog: readonly { Description: string; ID: string; Name: string }[]
  The agents the corpus was generated from (agents.json), or none.
Returns DecisionEvalRunEstimate
- Defined in packages/TestingFramework/Engine/src/decision-eval/discovery-suite.ts:181

Function EstimateDiscoveryEvalRun

Parameters

Returns DecisionEvalRunEstimate

Settings