Member Junction
    Preparing search index...

    Configuration for RunLLMConformanceSuite. The Script* functions program the provider test file's vendor-SDK mock; the suite tells the adapter WHAT the vendor should do, the adapter translates that into the vendor's native wire shapes.

    interface LLMConformanceSuiteConfig {
        CreateLLM: () => BaseLLM;
        KnownDeviations?: LLMConformanceDeviation[];
        NonStreamingFailureMode: "throws" | "failedResult";
        ProviderName: string;
        ScriptFailure: (
            error: Error,
            at: FailureSite,
            options?: { ChunkBeforeError?: string },
        ) => void;
        ScriptNonStreamingSuccess: (
            content: string,
            usage: ExpectedUsageCounts,
        ) => void;
        ScriptStreamingCancellation: (
            chunksBeforeAbort: string[],
            controller: AbortController,
        ) => void;
        ScriptStreamingSuccess: (
            chunks: string[],
            usage: ExpectedUsageCounts,
        ) => void;
        SupportsStreaming: boolean;
    }
    Index

    Properties

    CreateLLM: () => BaseLLM

    Construct a fresh driver instance wired to the scriptable vendor mock. Called in beforeEach, so every test gets a clean instance (and clean streaming state).

    KnownDeviations?: LLMConformanceDeviation[]

    Documented deviations from the ideal contract — see LLMConformanceDeviation.

    NonStreamingFailureMode: "throws" | "failedResult"

    How the driver surfaces NON-cancellation vendor failures on the NON-streaming path. BaseLLM imposes no rule here (ChatCompletions, plural, is the normalizing layer):

    • 'throws': the vendor error propagates out of ChatCompletion (OpenAI family, Groq, Cerebras, Mistral).
    • 'failedResult': the driver catches and resolves ChatResult{success:false} (Ollama, Azure).
    ProviderName: string

    Human-readable provider name, used in describe() labels.

    ScriptFailure: (
        error: Error,
        at: FailureSite,
        options?: { ChunkBeforeError?: string },
    ) => void

    Program the vendor mock: the next call fails with error at the given site. For 'midStream', the stream must first emit options.ChunkBeforeError (as a normal text chunk) and then throw, so the suite can prove partial content is discarded.

    ScriptNonStreamingSuccess: (content: string, usage: ExpectedUsageCounts) => void

    Program the vendor mock: the next non-streaming call succeeds with this content + usage.

    ScriptStreamingCancellation: (
        chunksBeforeAbort: string[],
        controller: AbortController,
    ) => void

    Program the vendor mock for a mid-stream cancellation: emit chunksBeforeAbort as normal text chunks, then abort controller, then end the stream the way the real SDK ends an aborted stream (throw its abort error, or end iteration silently — whichever is faithful).

    ScriptStreamingSuccess: (chunks: string[], usage: ExpectedUsageCounts) => void

    Program the vendor mock: the next streaming call emits these text chunks (one vendor chunk per entry, in order) and then completes, reporting usage in the vendor's native place for stream usage (final chunk / usage event) even if the driver is known to drop it.

    SupportsStreaming: boolean

    Whether the driver declares streaming support (asserted against SupportsStreaming).