Protected_Protected property to store additional provider-specific settings
ProtectedthinkingState tracking for streaming thinking extraction Providers should initialize this if they support thinking models
Get the current additional settings
ProtectedapiOnly sub-classes can access the API key
Bedrock supports assistant prefill for Claude (anthropic.*) models. The provider implementation handles per-model-prefix gating internally.
Amazon Bedrock supports streaming for most models
Whether this driver implements native tool/function calling — i.e. whether it maps
ChatParams.tools onto its SDK and normalizes tool calls back into
ChatCompletionMessage.toolCalls.
This is a CODE-level capability ("has the mapping been written for this driver?"), distinct
from the METADATA-level capability ModelConfiguration.LLM.SupportsNativeToolCalling
("does this model, on this vendor, support tools at all?"). Both must hold for native mode.
A driver returning false ignores any tools passed to it and records the fact in the
result's modelSpecificResponseDetails — the prompt runner's gate should keep that from
happening, but the layer is safe standalone.
ProtectedaddAdd thinking content to a chat completion message
OptionalthinkingContent: stringProtectedbuildBuilds the Bedrock Anthropic request body, separating system messages and placing
{ cachePoint: { type: 'default' } } checkpoints on the system prompt and stable history prefix.
ProtectedBuildThe result of a cancelled request, the same for every driver: a failed result with no choices,
statusText 'cancelled', and an errorInfo that says Cancelled (Fatal, never failed over) —
or, when the signal was aborted by a timeout, a retriable NetworkError.
Optionaloptions: { error?: unknown; message?: string; signal?: AbortSignal }Optionalerror?: unknownThe error the abort raised, kept on exception.
Optionalmessage?: stringThe message to report; defaults to one naming the signal's reason.
Optionalsignal?: AbortSignalThe request's abort signal, read for why it was aborted.
Process a chat completion request. If streaming is enabled and supported, this will route to the streaming implementation.
Process multiple chat completion requests in parallel. This is useful for:
Array of chat completion parameter objects
Optionalcallbacks: ParallelChatCompletionsCallbacksOptional callbacks for progress and individual completions
Promise resolving to an array of ChatResults in the same order as the input params
Clear all additional settings This is useful for resetting the state of the provider or when switching between different configurations.
ProtectedcreateProtectedextractExtract thinking content from non-streaming content This method handles case-insensitive extraction of thinking blocks
ProtectedfinalizeCreate the final response from streaming results for Bedrock
ProtectedflushReturns (and clears) any user-visible content the thinking-tag stripper is still holding back at
the end of a stream. Mid-stream, processStreamChunkWithThinking holds back a trailing
fragment that could be the start of a <think>/</think> tag so a split tag never leaks as
partial text; once the stream ends, such a fragment is real content and must be emitted (bug A5).
Only flushes when NOT inside a thinking block: an unterminated <think> block's buffered text is
reasoning, not answer, and is left held back (never surfaced as visible content). Returns ''
when thinking extraction isn't active (no state) or there's nothing to flush.
ProtectedformatFormats a message content payload into Bedrock content blocks and appends a Bedrock
{ cachePoint: { type: 'default' } } block when enableCaching is true.
ProtectedformatFormats non-system conversation messages for Bedrock Claude models with prompt caching.
When the conversation ends with a trailing <mj-runtime-state> message (optionally followed
by an assistant prefill), splitTrailingVolatileState places the { cachePoint: { type: 'default' } }
on the last message of split.head (the stable history prefix) and leaves split.tail uncached,
inserting an { role: 'assistant', content: [{ type: 'text', text: 'OK' }] } bridge turn if needed
to preserve strict user/assistant role alternation.
ProtectedformatFormats system messages into a flattened array of Bedrock content blocks and appends a
trailing { cachePoint: { type: 'default' } } checkpoint when enableCaching is true.
Returns the native file input capabilities of this LLM driver, or null if the driver does not support file attachments. Subclasses that accept files (PDFs, images, etc.) should override this method.
ProtectedgetGet the thinking tag format for this provider Providers can override this to customize the thinking tag format
ProtectedhandleTemplate method for handling streaming chat completion This implements the common pattern across providers while delegating provider-specific logic to abstract methods.
ProtectedinitializeInitialize thinking stream state for streaming extraction
ProtectedisReturns true if the given Bedrock model ID is an Anthropic Claude model
(either direct anthropic.* or cross-region inference profile us.anthropic.*, eu.anthropic.*, etc.).
ProtectedIsWhether a request ended because it was aborted: its signal is aborted, the error is an abort by
name (AbortError, APIUserAbortError, RequestAbortedError), or IsProviderAbortError
recognizes it.
Optionalsignal: AbortSignalProtectedIsRecognizes an abort error the shared names miss. A driver whose SDK raises its own class for an aborted request overrides this. Default: false.
ProtectedisDetects whether a message is a trailing volatile state message (<mj-runtime-state> or
<mj-agent-specialization>), either via metadata.volatileState === true (via BaseLLM)
or via the XML tag prefix on string or block content.
ProtectednonImplementation of non-streaming chat completion for Amazon Bedrock
ProtectedprocessProcess streaming chunk with thinking extraction This method handles case-insensitive extraction across chunk boundaries
ProtectedprocessProcess a streaming chunk from Bedrock
ProtectedresetHook invoked at the start AND end (in finally) of every streaming chat
completion to reset per-request streaming state. Default is a no-op;
providers that maintain instance-level streaming state (e.g., Anthropic /
OpenAI thinking-block accumulators) MUST override this. See audit R2-C5
for context — without this, state from a prior request bleeds into the
next one and accumulated buffers grow unbounded across the singleton's
lifetime.
ProtectedresetReset thinking stream state
Set additional provider-specific settings Subclasses should override this method to validate required settings
Provider-specific settings
ProtectedsplitSplits trailing volatile state (and optional trailing assistant prefill) from the stable conversation history prefix so prompt cache breakpoints stay on the stable prefix.
ProtectedsupportsReturns true if the given Bedrock model ID supports Bedrock prompt caching (cachePoint).
ProtectedsupportsCheck if the provider supports thinking models Providers should override this to return true if they support thinking extraction
ProtectedtrailingIndex of the trailing volatile-state message in an outgoing request, or -1 when there is none. The fragment is the LAST message, or the one before it when an assistant prefill has been appended after it. -1 is also returned when nothing precedes the fragment, because then there is no stable history for a provider to cache ahead of it.
Base class for all LLM sub-class implementations. Not all sub-classes will support all methods. If a method is not supported an exception will be thrown.