Creates a new parallel execution coordinator with default configuration.
ProtectedDefaultKeeps this runner's uncategorized errors logged under AIPromptRunner, as before the base class existed.
ProtectedDefaultEngine-level default model-call timeout, in milliseconds, applied when the caller supplies no
AIPromptParams.timeoutMS. undefined (the default) means NO implicit bound — a prompt run
with neither a timeout nor a cancellation token stays unbounded, exactly as before, so this
change is behavior-preserving for existing callers.
Subclasses (or a host application's runner subclass) can override this to impose a global safety ceiling on every prompt call.
Access the underlying AIModelRunner for embedding and other non-LLM model calls. Use this when you need tracked embedding execution with AIPromptRun record creation.
ProtectedParallelLazily resolves the parallel execution coordinator.
The coordinator is a SUBCLASS of AIPromptRunner — it inherits executeModel and the rest
of the battle-tested execution path so there is a single source of truth for credential / driver
/ ChatParams / streaming resolution. That subclass relationship means the base cannot statically
new it without a hard circular import, so we resolve it through the ClassFactory instead (the
coordinator self-registers via @RegisterClass(AIPromptRunner, PARALLEL_COORDINATOR_KEY)).
Created once per runner and reused; the runner's Provider override is propagated. Throws a clear
error if the coordinator class was never loaded/registered — otherwise ClassFactory would silently
fall back to a plain AIPromptRunner that lacks the parallel methods.
Optional metadata provider override. Callers should set
instance.Provider = providerToUse before invoking run methods
in multi-provider contexts. Falls back to the global default provider when unset.
The model type this runner requires. Returns 'LLM' for AIPromptRunner.
ProtectedApplyReturns the template data with _NATIVE_TOOL_CALLING and _NATIVE_CONTROL_FLOW set.
With a decision, both carry the gate's real answer. Without one (a run that declares no tools), they are set to the envelope path's values unless the caller already supplied them. The Loop system prompt reads both unconditionally, so template parameter extraction marks them required, and a run that left them unset failed to render.
The caller's template data, if any
The resolved native tool-calling decision, or null when no tools were declared
ProtectedApplyOptionalsuggestedDelaySeconds: numberProtectedApplyRecords one call's usage on its run row, for runners whose calls are not chat calls: tokens;
continuous units (seconds, characters, images) with the MJ: AI Usage Types row that names
their measure; and a cost only when the driver gave one. With no usage nothing is recorded, so
the cost stays empty and the row's save prices it from the model's cost rows, or declines to.
Call it from the FinalizeRunRecord callback.
ProtectedAssertAsserts that the prompt's configured model type matches the runner's required model type.
If prompt.AIModelTypeID is null or undefined, does not throw (the runner's required type applies).
If prompt.AIModelTypeID is set and does not match RequiredModelTypeID, throws a descriptive
error naming the prompt, the prompt's configured type (resolved to name if possible), and the runner's
required type.
ProtectedBuildThe error for a model call the caller cancelled. It is classified Cancelled — Fatal and not
eligible for failover — so ErrorAnalyzer, the failover loop and the validation retries all stop
on it rather than treating it as an unknown model error.
Optionalmessage: stringThe message to keep, when a driver already reported the cancellation.
ProtectedBuildBuilds a unified, ordered list of model-vendor candidates based on all selection criteria. Uses a 3-phase approach to properly handle SelectionStrategy='Specific' with AIPromptModel priorities.
Phase 1: Handle explicit model ID (highest priority) Phase 2: Check if SelectionStrategy='Specific' with AIPromptModel entries - use ONLY those with AIPromptModel priorities Phase 3: Use general selection strategy (fallback) - blended priorities from legacy behavior
The AI prompt with selection criteria
OptionalexplicitModelId: stringExplicitly specified model ID (highest priority)
OptionalconfigurationId: stringConfiguration ID for filtering
OptionalpreferredVendorId: stringPreferred vendor ID
Optionalverbose: booleanOrdered array of model-vendor candidates (highest priority first)
ProtectedcalculateCalculates the delay before the next failover attempt.
The current attempt number (1-based)
The base delay in seconds from configuration
Delay in milliseconds before the next attempt
ProtectedcreateComposes the caller-supplied cancellation token with the resolved model-call timeout into a single AbortSignal that bounds one model call. NEITHER bound is discarded:
Signal is undefined and the call runs unbounded (legacy behavior)Implemented with an AbortController + relay listener rather than AbortSignal.any() so it works
on Node 18 (where AbortSignal.any does not exist — it landed in Node 20.3).
OptionalcancellationToken: AbortSignalProtectedCreateCreates the MJ: AI Prompt Runs record for one model call, fills the fields every runner shares,
lets the subclass add its own request fields, then queues the INSERT (fire-and-forget).
For prompt-based runners: the record is always tied to an AIPrompt and its AIPromptParams.
OptionalvendorId: stringOptionalmodelSelectionInfo: AIModelSelectionInfoOptionalapplyRequestFields: (promptRun: MJAIPromptRunEntityExtended) => voidSets the runner's own request columns. It runs once, after the shared
fields are set and before the INSERT is queued, so everything it sets is on the inserted row.
It must be synchronous. An async callback type-checks against this signature, but any
field it sets after its first await races the INSERT and can miss the row.
ProtectedexecuteOptionalconversationMessages: ChatMessage[]OptionalcancellationToken: AbortSignalOptionalvendorDriverClass: stringOptionalvendorApiName: stringOptionalvendorSupportsEffortLevel: booleanOptionalmodelEffortLevel: numberOptionalpromptModelConfiguration: AIConfigurationSectionsProtectedexecuteExecutes the AI model with failover support
OptionalconversationMessages: ChatMessage[]OptionalcancellationToken: AbortSignalOptionalallCandidates: ModelVendorCandidate[]OptionalpromptRun: MJAIPromptRunEntityExtendedOptionalvendorDriverClass: stringOptionalvendorApiName: stringOptionalvendorSupportsEffortLevel: booleanOptionalmodelEffortLevel: numberOptionalcredentialAvailability: Map<string, boolean>OptionalpromptModelConfiguration: AIConfigurationSectionsThis method wraps the core executeModel functionality with intelligent failover capabilities. It will attempt to execute with different models/vendors according to the configured failover strategy when errors occur.
Candidates come from model selection (allCandidates), already filtered to the prompt's
model type by ID. When failover applies, the loop itself is
BaseModelRunner.ExecuteWithFailover: this method supplies the chat call on each candidate
(executeModel with that candidate's model, vendor, driver, effort level and prompt-model
configuration) and the final error result (createFailoverErrorResult). The base records
failover success or failure on the prompt run.
Executes an AI prompt with full support for templates, model selection, and validation.
Parameters for prompt execution
Promise<AIPromptRunResult
// Execute with specific result type
interface AnalysisResult {
sentiment: string;
score: number;
keywords: string[];
}
const result = await promptRunner.ExecutePrompt<AnalysisResult>({
prompt: sentimentPrompt,
data: { text: "Customer feedback text" }
});
if (result.success && result.result) {
// result.result is typed as AnalysisResult
console.log(`Sentiment: ${result.result.sentiment}, Score: ${result.result.score}`);
}
Executes multiple tasks in parallel according to their execution groups.
Tasks are grouped by their executionGroup property and executed sequentially by group. Within each group, tasks are executed in parallel up to the concurrency limit.
Array of execution tasks to process
Optionalconfig: Partial<ParallelExecutionConfig>Optional configuration to override defaults
OptionalparentPromptRunId: stringOptional parent prompt run ID for hierarchical logging
OptionalcancellationToken: AbortSignalOptional cancellation token to abort execution
OptionalprogressCallbacks: ProgressCallbacksInterfaceOptional callbacks for progress tracking
Promise
ProtectedExecuteRuns one model call across the failover candidates, in priority order: skips candidates with no credentials, lets processFailoverError decide retry / next candidate / stop, and records failover success or failure on the prompt run. The model call itself and the final error result are supplied by the subclass, so the loop works for any result type that extends BaseResult.
For prompt-based runners: it takes the AIPrompt and AIPromptParams the call is for, and
candidates that carry prompt-model fields (effort level, the AIPromptModel configuration).
Makes the call on one candidate. It must use the candidate's own model, vendor, driver and prompt-model fields, not those of the first candidate.
Builds the result returned when every candidate has failed.
OptionalpromptRun: MJAIPromptRunEntityExtendedOptionalcredentialAvailability: Map<string, boolean>ProtectedFinalizeFinalizes a prompt-run record in memory immediately AND queues a fire-and-forget UPDATE on
_promptRunQueue that re-applies the finalized state after any in-flight 'Running' INSERT
and its finalizeSave reload complete.
Applying the completion fields synchronously in memory ensures callers that inspect
result.promptRun right after ExecutePrompt (such as BaseAgent.calculateTokenStats() reading
TokensPromptRollup, TokensCompletionRollup, TotalCost, and Status) see the finalized values
immediately without blocking on the database round-trip, while re-applying the captured snapshot
inside _promptRunQueue.Update(promptRun, ...) guarantees an in-flight INSERT reload can never
revert the persisted row back to 'Running'.
In order, it sets: the completion timing (CompletedAt, ExecutionTimeMS); the outcome (Success,
and Status = 'Completed' or 'Failed' from it); any pending failover fields; the subclass's
result fields (applyResultFields, invoked once); and finally the token rollups and TotalCost.
Whether the call succeeded, as the runner judges it (the chat runner also requires its output to pass validation).
Sets the runner's own result columns. It can read the timing and outcome
already set, and may refine them (for example ErrorDetails or Status = 'Cancelled').
It must be synchronous.
Awaits all in-flight MJAIPromptRunEntityExtended INSERT and UPDATE operations queued on this
runner's internal BaseEntitySaveQueue (_promptRunQueue) and clears the in-flight parent
lookup table.
Normal intra-step prompt execution is non-blocking (fire-and-forget); callers that own the outer
lifecycle (such as BaseAgent at run completion, failure, cancellation, or compaction) call
FlushPendingSaves() to guarantee every AIPromptRun row is durably settled before returning.
ProtectedgetResolves the per-model-call timeout: the caller's AIPromptParams.timeoutMS, else the runner's
DefaultPromptTimeoutMS. Non-positive / non-numeric values mean "no timeout".
The bound is applied PER MODEL CALL (not per prompt execution), which mirrors the parallel
path's existing taskTimeoutMS semantics: each failover candidate / validation retry gets a
fresh budget rather than sharing one wall-clock window.
NOTE (issue #3064): there is deliberately NO prompt-entity source here yet — the AIPrompt
table has no TimeoutMS column today. Once a migration adds one and CodeGen regenerates the
entity, this becomes prompt.TimeoutMS ?? params.timeoutMS ?? this.DefaultPromptTimeoutMS
and every bound below starts honoring the per-prompt configuration with no other change.
ProtectedgetRetrieves failover configuration from the prompt entity.
The AI prompt entity containing failover settings
FailoverConfiguration object with strategy and settings
ProtectedGetThe server default template-data capture mode (aiPromptRuns.templateDataCapture in mj.config).
ProtectedHasChecks if credentials are available for a given model-vendor combination. This is a pre-flight check used during model selection to determine which candidates have valid authentication configured.
Checks the credential hierarchy:
Each tier counts only if CredentialScopeAllows the run's scope to use its source: under
'RuntimeOnly' only 1 and 6 count, so a candidate the caller has no key for is unavailable —
which is what keeps failover on the caller's keys.
The driver class name (e.g., 'OpenAILLM')
The prompt ID for looking up AIPromptModel bindings
The model ID for looking up AIPromptModel and AIModelVendor bindings
The vendor ID for looking up AIModelVendor and AIVendor bindings
Optionalparams: AIPromptParamsThe prompt execution parameters
true if credentials are available, false otherwise
ProtectedIsWhether a failure was caused by the caller cancelling the request: its classification is
Cancelled, or the older shape drivers built before that type existed.
ProtectedIsChecks if a model vendor is configured as an inference provider. Delegates to the memoized AIEngine.IsInferenceProvider helper so the "Inference Provider" vendor-type lookup happens once per engine load rather than on every candidate in every selection pass.
The model vendor to check
true if the vendor is an inference provider
ProtectedlogHelper method for enhanced error logging with metadata
Optionaloptions: {ProtectedlogLogs a failover attempt for tracking and debugging.
The ID of the prompt being executed
The failover attempt details
Whether another attempt will be made
ProtectedlogInternal logging helper that wraps LogStatusEx with verbose control
The message to log
Whether this is a verbose-only message
Optionalparams: AIPromptParamsOptional prompt parameters for custom verbose check
ProtectedRaceWaits for a model call, or rejects as soon as its bound aborts: with the bound's AIPromptTimeoutError when the timeout fired, else with a cancellation error. The call the runner stops waiting for is torn down only if its driver was handed the bound's signal.
For any runner whose driver call is not already bounded by the driver itself.
Render a set of child prompt templates WITHOUT executing anything, returning the rendered text keyed by each child's parent placeholder — exactly what the hierarchical execution path embeds into the parent template.
Exposed for callers that need a child's rendered text before the run: the loop agent uses it to
relocate a volatile specialization into the trailing runtime-state message (see
ResolveSpecializationPlacement in @memberjunction/ai-agents) while the system prompt renders a
stub in its place. Rendering is deterministic for the same inputs, so a subsequent execution of
the same params reproduces the same text.
The child prompt params, as they would be passed in AIPromptParams.childPrompts.
The parent params (context user, data, template data) the children render against.
OptionalcancellationToken: AbortSignalOptional abort signal.
ProtectedRequiredResolves the runner's RequiredModelType name against AIEngine.Instance.ModelTypes
(case-insensitive, trimmed) and returns its UUID. Callers compare models' AIModelTypeID with it
using UUIDsEqual.
Throws a descriptive error if the model type is not found in the engine metadata.
ProtectedResolveResolves credentials for AI model execution using a hierarchical resolution system.
Resolution priority (highest to lowest):
IMPORTANT: When ANY credential ID is found (priorities 1-4), the system uses the Credentials path and ignores legacy methods (priorities 5-6).
Each tier is consulted only if CredentialScopeAllows the run's
AIPromptParams.CredentialScope to use its source. Under 'RuntimeOnly' that leaves
priority 1 and the apiKeys entry for this driver class; bindings, the vendor default and the
environment are skipped.
The driver class name (e.g., 'OpenAILLM')
The prompt ID for looking up AIPromptModel credentials
The model ID for looking up AIPromptModel and AIModelVendor credentials
The vendor ID for looking up AIModelVendor and AIVendor credentials
The prompt execution parameters containing contextUser and optional credentialId
The API key/configuration string to pass to the LLM constructor
ProtectedResolveThe prompt layer of the native tool-calling cascade, with a hierarchical run's child prompt merged over its parent.
An agent runs its agent type's system prompt as prompt and its own prompt as the first
child, so the parent's bag alone describes the agent TYPE, not the agent. Native tool calling
is a property of the agent's own prompt, so the child's bag sits between the parent and the
selected AIPromptModel row: parent → child → row. The merge is per key, so an explicit value
in the child (including false) overrides the parent, and an absent key inherits.
The prompt being run (the parent on a hierarchical run)
The caller's params — supplies the child prompts, if any
The merged prompt configuration, or null when neither layer sets one
OptionalpromptModelConfiguration: AIConfigurationSectionsResolves the native tool-calling gate WITHOUT touching a request.
Split out because the answer is needed twice and must be the same both times: once before the
template renders — the loop template drops its action catalog and the 'Actions' step type
when native mode is on (plan §8.4), and it can only do that truthfully if it knows the real
decision rather than the caller's intent — and once when the request is assembled.
Never throws: the gate is an opt-in enhancement and must not be able to fail a run that would otherwise succeed, so any configuration problem resolves to the path that has always worked.
OptionalpromptModelConfiguration: AIConfigurationSectionsProtectedResolveThe usage to record for one call: the driver's when it reports a quantity (tokens, or units in a measure), otherwise the runner's own count, which keeps any cost the driver gave. With neither, the driver's usage (possibly undefined) is returned, and nothing is invented.
A runner counts what it can see without the driver: the images returned, or the characters it sent. The driver's figure wins because it is the vendor's own measure.
The usage the driver returned, if any.
The runner's own count as ModelUsage.ForMedia, or undefined when it has none.
Selects the best result from multiple parallel execution results.
Uses various strategies to determine which result should be considered the "best" output from the parallel executions.
Array of successful execution results to choose from
Configuration for result selection method
OptionalparentPromptRunId: stringOptional parent prompt run ID for hierarchical logging
OptionalcancellationToken: AbortSignalOptionalcontextUser: UserInfoOptionalexecutionScope: AIPromptExecutionScopeThe parallel prompt's AIPromptExecutionScope; a PromptSelector
judge runs under it, so it spends the same credentials the candidates did
Promise<ExecutionTaskResult | null> - The selected best result, or null if none suitable
ProtectedselectSelects candidate models for failover based on the strategy and current failure.
The model that just failed
The vendor ID that just failed
The failover strategy to use
The model selection preference
All available model-vendor candidates
History of previous failover attempts
Array of candidates sorted by priority (highest first)
This method implements different strategies for selecting failover candidates:
Override this method to implement custom candidate selection logic.
ProtectedselectSelects the appropriate AI model based on prompt configuration and parameters. Uses the unified BuildModelVendorCandidates method to create an ordered list of candidates, then selects the first one with an available API key.
OptionalexplicitModelId: stringOptionalcontextUser: UserInfoOptionalconfigurationId: stringOptionalvendorId: stringOptionalparams: AIPromptParamsProtectedshouldDetermines whether a failover attempt should be made based on the error and configuration.
The error that occurred during execution
The failover configuration
The current attempt number (1-based)
True if failover should be attempted, false otherwise
ProtectedupdateUpdates prompt run with successful failover tracking data
Awaits all in-flight prompt-run saves queued by this runner instance. The normal execution path does NOT call this — prompt-run persistence is intentionally fire-and-forget. Exposed for tests and for callers that need the AIPromptRun rows durably written before proceeding.
StaticRenderThe render options a prompt asks for: a raw render when templateAutoescape is false, else the engine's default.
Coordinates parallel multi-model prompt execution.
SUBCLASSES AIPromptRunner purely to REUSE its execution primitives — most importantly the
protected executeModel, which owns credential resolution, driver selection, ChatParams construction, prefill, media handling, and streaming. Each parallel task delegates to that ONE method (see executeSingleTask) instead of re-implementing it, so the parallel and single-model paths can never drift. The coordinator only adds task fan-out, grouping, timeouts, result selection, and child-run tracking on top.Registered under the base class with a dedicated key so AIPromptRunner.ParallelCoordinator can instantiate it via the ClassFactory without a static (circular) import.