Class SummarizeParams

Optional text to prefill the assistant's response. When set, the model will behave as if it has already started generating this text and will continue from where the prefill ends.

This is useful for:

• Forcing structured output formats (e.g., prefill with "```json" to get raw JSON)
• Skipping preamble (e.g., prefill with "{" to get a JSON object directly)
• Guiding the model's response style or format

Best used in combination with stopSequences — for example, prefill with "json\n" and set stopSequences to [""] to extract clean JSON without any markdown fencing.

Provider support: Anthropic, Mistral, Groq, Bedrock (Claude models), Ollama, and OpenRouter natively support prefill. For providers that don't support it, this parameter is silently ignored.

Note: The prefill text is NOT included in the response — the model's output begins immediately after the prefill. If you need the prefill text in your final result, prepend it yourself after receiving the response.

Optional cancellation token to abort the chat completion request. When this signal is aborted, the provider should cancel the request and return a cancelled result as gracefully as possible.

If the model supports effort levels, this parameter can be used to specify the effort level.

Whether to enable caching for this request. Implementation depends on the specific provider (the below are examples, many other providers exist):

• For Anthropic: Uses Anthropic's ephemeral cache control to cache system prompt and last user message.
• For OpenAI: Uses automatic caching (provider handles it).
• For other providers: May be a no-op if caching isn't supported.

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

Not all AI providers support this feature. When supported, this parameter indicates if logprobs are requested. Logprobs provide information about the likelihood of each token in the response. This can be useful for debugging or understanding the model's behavior. When models support this and this property is set to true, the model will return logprobs for the tokens in the

Model max output response tokens, optional.

Array of messages, allows full control over the order and content of the conversation.

Minimum probability threshold for token sampling (0-1). Tokens with probability below this threshold are filtered out before sampling. This is a newer parameter not yet widely supported.

Model name, required.

The standard response formats may not be sufficient for all models. This field allows for a model-specific response format to be specified. For this field to be used, responseFormat must be set to 'ModelSpecific'.

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

Model max budget tokens that we may use for reasoning in reasoning models, optional.

Specifies the format that the model should output. Not all models support all formats. If not specified, the default is 'Any'.

Optional seed for reproducible outputs. Not all models support seeding, but when supported, using the same seed with the same inputs should produce identical outputs.

Optional array of sequences where the model will stop generating further tokens. The returned text will not contain the stop sequence.

Whether to use streaming for this request. If true and the provider supports streaming, responses will be streamed. If true but the provider doesn't support streaming, the request will fall back to non-streaming.

Callbacks for streaming responses. Only used when streaming is true.

Model temperature, optional.

Top-k sampling parameter. Limits the model to only sample from the top K most likely tokens at each step. For example, k=50 means the model will only consider the 50 most likely next tokens. Not supported by all providers (e.g., OpenAI doesn't support this).

Number of top log probabilities to return per token. Only used when includeLogProbs is true. Typically ranges from 2-20, depending on the provider.

Top-p (nucleus) sampling parameter (0-1). An alternative to temperature sampling that considers the smallest set of tokens whose cumulative probability exceeds the probability p. For example, 0.1 means only the tokens comprising the top 10% probability mass are considered. Generally, use either temperature OR top-p, not both.