Class RealtimeAgentType

Implementation of the Realtime Agent Type pattern.

A first-class peer of Loop and Flow that drives a BaseRealtimeModel session rather than an iterative reasoning loop. Because it is a real MJ agent type, an agent of this type inherits the entire framework for free: server tools (actions), client tools, artifacts, prompts, memory, permissions, and observability. The first agent shipped of this type is the Realtime Co-Agent, which voices on behalf of a target agent.

Execution branch. The realtime path is session-driven: BaseAgent (in a later task) will detect IsSessionDriven (or instanceof RealtimeAgentType) and hand control to a RealtimeSessionRunner rather than entering the loop. As a consequence, the loop-oriented abstract methods below are implemented defensively — they should never be reached in normal operation.

RealtimeAgentType

Hierarchy (View Summary)

BaseAgentType
- RealtimeAgentType

Constructors

constructor

new RealtimeAgentType(): RealtimeAgentType
Returns RealtimeAgentType
Inherited from BaseAgentType.constructor

Properties

`Protected`_jsonValidator

_jsonValidator: JSONValidator = ...

JSON validator instance for cleaning and validating responses

`Static` `Readonly`CURRENT_PAYLOAD_PLACEHOLDER

CURRENT_PAYLOAD_PLACEHOLDER: "_CURRENT_PAYLOAD" = '_CURRENT_PAYLOAD'

Common placeholder for current payload injection

Static

Accessors

InjectLoopResultsAsMessage

get InjectLoopResultsAsMessage(): boolean
Determines if loop results should be injected as a temporary user message before the next prompt execution (for LLM reasoning).

Default: true (most agent types benefit from seeing loop results) Flow agents override to false (deterministic path navigation, no LLM)

Returns boolean
true to inject results as message, false to skip

Since
2.112.0
Inherited from BaseAgentType.InjectLoopResultsAsMessage
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:535

IsSessionDriven

get IsSessionDriven(): boolean
Marks this agent type as session-driven rather than loop-driven.

BaseAgent will use this getter (added on this subclass; base-agent-type.ts is left untouched in this task) to branch into a RealtimeSessionRunner instead of the iterative reasoning loop. Other agent types do not expose this member; BaseAgent should treat its absence as false, or equivalently detect this type via instanceof RealtimeAgentType.

Returns boolean
Always true — realtime agents are driven by a long-lived duplex session.
- Defined in AI/Agents/src/agent-types/realtime-agent-type.ts:64

RequiresAgentLevelPrompts

get RequiresAgentLevelPrompts(): boolean
Realtime agents drive their conversation through a live model session, not through agent-level loop prompts. The companion/system prompt is supplied to the realtime model at session start, so the agent-level prompt relationship is not required.

Returns boolean
Always false — agent-level loop prompts are not required for realtime agents.
Overrides BaseAgentType.RequiresAgentLevelPrompts
- Defined in AI/Agents/src/agent-types/realtime-agent-type.ts:75

Methods

`Optional`AfterLoopIteration

AfterLoopIteration?(
 iterationResult: {
 actionResults?: ActionResult[];
 currentPayload: P;
 index: number;
 item: any;
 itemVariable: string;
 loopContext: any;
 subAgentResult?: any;
 },
): P
Called after each loop iteration completes to process results.

Flow agents use this to apply ActionOutputMapping and update payload. Loop agents typically don't need this (just collect results).
Type Parameters
- P
Parameters
- iterationResult: {
 actionResults?: ActionResult[];
 currentPayload: P;
 index: number;
 item: any;
 itemVariable: string;
 loopContext: any;
 subAgentResult?: any;
 }
 Results from this iteration
Returns P
Modified payload or null for default behavior (just collect result)

Since
2.112.0
Inherited from BaseAgentType.AfterLoopIteration
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:579

`Optional`BeforeLoopIteration

BeforeLoopIteration?(
 context: {
 actionParams: Record<string, unknown>;
 index: number;
 item: any;
 itemVariable: string;
 loopType: "ForEach" | "While";
 payload: P;
 subAgentRequest?: {
 message: string;
 name: string;
 templateParameters?: Record<string, string>;
 };
 },
): {
 actionParams?: Record<string, unknown>;
 payload?: P;
 subAgentRequest?: {
 message: string;
 name: string;
 templateParameters?: Record<string, string>;
 };
}
Called before each loop iteration to prepare parameters and payload.

Loop agents use this to resolve template variables ("item.email"). Flow agents typically don't need this (params already resolved).
Type Parameters
- P
Parameters
- context: {
 actionParams: Record<string, unknown>;
 index: number;
 item: any;
 itemVariable: string;
 loopType: "ForEach" | "While";
 payload: P;
 subAgentRequest?: {
 message: string;
 name: string;
 templateParameters?: Record<string, string>;
 };
 }
 Current iteration context
Returns {
 actionParams?: Record<string, unknown>;
 payload?: P;
 subAgentRequest?: {
 message: string;
 name: string;
 templateParameters?: Record<string, string>;
 };
}
Modified context or null for default behavior

Since
2.112.0
Inherited from BaseAgentType.BeforeLoopIteration
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:550

`Protected`createNextStep

createNextStep(
 step:
 | "Actions"
 | "Chat"
 | "Failed"
 | "ForEach"
 | "Retry"
 | "Sub-Agent"
 | "Success"
 | "While",
 options?: Partial<BaseAgentNextStep>,
): BaseAgentNextStep
Protected
Creates a standardized next step object with common defaults
Type Parameters
- P
 The payload type
Parameters
- step:
 | "Actions"
 | "Chat"
 | "Failed"
 | "ForEach"
 | "Retry"
 | "Sub-Agent"
 | "Success"
 | "While"
 The step type
- options: Partial<BaseAgentNextStep> = {}
 Additional options to merge
Returns BaseAgentNextStep
The next step object
Inherited from BaseAgentType.createNextStep
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:329

`Protected`createRetryStep

createRetryStep(
errorMessage: string,
options?: Partial<BaseAgentNextStep>,
): BaseAgentNextStep
Protected
Creates a retry step with a standardized error message
Type Parameters
- P
 The payload type
Parameters
- errorMessage: string
 The error message
- options: Partial<BaseAgentNextStep> = {}
 Additional options
Returns BaseAgentNextStep
Retry step
Inherited from BaseAgentType.createRetryStep
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:351

`Protected`createSuccessStep

createSuccessStep(
options?: Partial<BaseAgentNextStep>,
): BaseAgentNextStep
Protected
Creates a success step with optional payload changes
Type Parameters
- P
 The payload type
Parameters
- options: Partial<BaseAgentNextStep> = {}
 Success options
Returns BaseAgentNextStep
Success step
Inherited from BaseAgentType.createSuccessStep
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:372

DetermineInitialStep

DetermineInitialStep(
 params: ExecuteAgentParams,
 payload: P,
 agentTypeState: ATS,
): Promise<BaseAgentNextStep>
Determines the initial step for a realtime agent.

The realtime conversation is owned by the session runner, so there is no loop step to begin. Returning null defers to the caller; in practice BaseAgent branches on IsSessionDriven before this would matter.
Type Parameters
- P = any
- ATS = any
Parameters
- params: ExecuteAgentParams
 The full execution parameters (unused).
- payload: P
 The current payload (unused).
- agentTypeState: ATS
 The agent-type state (unused).
Returns Promise<BaseAgentNextStep>
Always null — there is no loop step to seed.
Overrides BaseAgentType.DetermineInitialStep
- Defined in AI/Agents/src/agent-types/realtime-agent-type.ts:135

DetermineNextStep

DetermineNextStep(
 promptResult: AIPromptRunResult<unknown>,
 params: ExecuteAgentParams<any, P>,
 payload: P,
 agentTypeState: ATS,
): Promise<BaseAgentNextStep>
Determines the next step from a prompt result.

Realtime agents are session-driven and never enter the iterative reasoning loop, so this method should never be reached in normal operation. It is implemented defensively: it logs and returns a terminal Failed step rather than throwing, so a mis-wired caller degrades gracefully instead of crashing a live session.
Type Parameters
- P = any
- ATS = any
Parameters
- promptResult: AIPromptRunResult<unknown>
 Result from prompt execution (unused).
- params: ExecuteAgentParams<any, P>
 The full execution parameters (unused).
- payload: P
 The current payload (unused).
- agentTypeState: ATS
 The agent-type state (unused).
Returns Promise<BaseAgentNextStep>
A terminal Failed step describing the misuse.
Overrides BaseAgentType.DetermineNextStep
- Defined in AI/Agents/src/agent-types/realtime-agent-type.ts:109

GetPromptConfigurationGuidance

GetPromptConfigurationGuidance(): string
Provides agent-type-specific guidance for configuration errors related to missing prompts. This allows each agent type to give contextual help based on its architecture.

Default implementation provides generic guidance. Agent types should override to provide specific instructions relevant to their configuration requirements.

Returns string
Configuration guidance specific to this agent type

Since
2.113.0
Inherited from BaseAgentType.GetPromptConfigurationGuidance
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:488

GetPromptForStep

GetPromptForStep(
 params: ExecuteAgentParams,
 config: AgentConfiguration,
 payload: P,
 agentTypeState: ATS,
 previousDecision?: BaseAgentNextStep,
): Promise<MJAIPromptEntityExtended>
Gets the prompt to use for a step.

Realtime agents do not select per-step loop prompts — the companion/system prompt is handed to the realtime model at session start. The default child prompt (if any) is returned for completeness, but it is not used to drive the conversation.
Type Parameters
- P = any
- ATS = any
Parameters
- params: ExecuteAgentParams
 The execution parameters (unused).
- config: AgentConfiguration
 The loaded agent configuration.
- payload: P
 The current payload (unused).
- agentTypeState: ATS
 The agent-type state (unused).
- OptionalpreviousDecision: BaseAgentNextStep
 The previous step decision (unused).
Returns Promise<MJAIPromptEntityExtended>
The configured child prompt, or null if none is configured.
Overrides BaseAgentType.GetPromptForStep
- Defined in AI/Agents/src/agent-types/realtime-agent-type.ts:178

HandleStepFallback

HandleStepFallback(
 step: BaseAgentNextStep,
 config: AgentConfiguration,
 params: ExecuteAgentParams,
 payload: P,
 agentTypeState: ATS,
): Promise<BaseAgentNextStep>
Determines how to handle Success or Failed steps when no explicit termination is requested.

This allows agent types to control their own fallback behavior:
- Loop agents use default behavior (return null) to process results with their main prompt
- Flow agents should terminate instead of falling back to prompts (return terminate step)
- Pipeline agents might want to move to the next stage
Default implementation returns null, which causes base-agent to fall back to prompt execution if prompts are configured. Agent types can override this to provide custom behavior.
Type Parameters
- P = any
- ATS = any
Parameters
- step: BaseAgentNextStep
 The Success or Failed step that needs fallback handling
- config: AgentConfiguration
 The loaded agent configuration
- params: ExecuteAgentParams
 The execution parameters
- payload: P
 The current payload
- agentTypeState: ATS
 Agent type's state
Returns Promise<BaseAgentNextStep>
Custom step to execute, or null for default behavior

Since
2.113.0
Inherited from BaseAgentType.HandleStepFallback
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:513

InitializeAgentTypeState

InitializeAgentTypeState<ATS = any, P = any>(
params: ExecuteAgentParams<any, P>,
): Promise<ATS>
Initializes agent-type-specific state for a realtime run.

The Realtime agent type keeps no iterative loop state — all live state lives inside the RealtimeSessionRunner and the underlying provider session — so an empty state object is returned, mirroring LoopAgentType.InitializeAgentTypeState.
Type Parameters
- ATS = any
- P = any
Parameters
- params: ExecuteAgentParams<any, P>
 The agent execution params (unused for realtime state).
Returns Promise<ATS>
An empty agent-type state object.
Overrides BaseAgentType.InitializeAgentTypeState
- Defined in AI/Agents/src/agent-types/realtime-agent-type.ts:89

InjectPayload

InjectPayload(
 payload: P,
 agentTypeState: ATS,
 prompt: AIPromptParams,
 agentInfo: { agentId: string; agentRunId?: string },
): Promise<void>
Injects a payload into a prompt.

Realtime conversations are not driven by loop prompts, so there is no per-turn payload to inject. This is a no-op (defensive against a mis-wired caller), in contrast to LoopAgentType.InjectPayload which injects the current payload into the loop prompt.
Type Parameters
- P = any
- ATS = any
Parameters
- payload: P
 The payload to inject (unused).
- agentTypeState: ATS
 The agent-type state (unused).
- prompt: AIPromptParams
 The prompt parameters (unused).
- agentInfo: { agentId: string; agentRunId?: string }
 Agent identification info (unused).
Returns Promise<void>
Overrides BaseAgentType.InjectPayload
- Defined in AI/Agents/src/agent-types/realtime-agent-type.ts:200

`Protected`parseJSONResponse

parseJSONResponse<T>(promptResult: AIPromptRunResult): T
Protected
Parses JSON response from prompt execution with automatic validation syntax cleaning
Type Parameters
- T
 The expected response type
Parameters
- promptResult: AIPromptRunResult
 The prompt execution result
Returns T
Parsed response or null if parsing fails
Inherited from BaseAgentType.parseJSONResponse
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:291

PostProcessActionStep

PostProcessActionStep(
 actionResults: ActionResult[],
 actions: AgentAction[],
 currentPayload: P,
 agentTypeState: ATS,
 currentStep: BaseAgentNextStep,
): Promise<AgentPayloadChangeRequest>
Post-processes the result of action execution.

This method is called by BaseAgent after action(s) have been executed. Agent types can override this method to perform custom processing of action results, such as mapping output parameters to the payload or storing results in agent-specific context.
Type Parameters
- P = any
- ATS = any
Parameters
- actionResults: ActionResult[]
 The results from action execution
- actions: AgentAction[]
 The actions that were executed
- currentPayload: P
 The current payload
- agentTypeState: ATS
- currentStep: BaseAgentNextStep
 The current step being executed
Returns Promise<AgentPayloadChangeRequest>
Optional payload change request

Since
2.76.0
Inherited from BaseAgentType.PostProcessActionStep
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:423

PostProcessSubAgentStep

PostProcessSubAgentStep(
 subAgentResult: any,
 subAgentRequest: AgentSubAgentRequest,
 currentPayload: P,
 agentTypeState: ATS,
 currentStep: BaseAgentNextStep,
): Promise<AgentPayloadChangeRequest>
Post-processes the result of sub-agent execution.

This method is called by BaseAgent after a sub-agent has been executed. Agent types can override this method to perform custom processing of sub-agent results, such as extracting specific data from the sub-agent's payload or updating context.
Type Parameters
- P = any
- ATS = any
Parameters
- subAgentResult: any
 The result from sub-agent execution
- subAgentRequest: AgentSubAgentRequest
 The sub-agent request that was executed
- currentPayload: P
 The current payload
- agentTypeState: ATS
- currentStep: BaseAgentNextStep
 The current step being executed
Returns Promise<AgentPayloadChangeRequest>
Optional payload change request

Since
2.76.0
Inherited from BaseAgentType.PostProcessSubAgentStep
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:451

PreProcessActionStep

PreProcessActionStep(
 actions: AgentAction[],
 currentPayload: P,
 agentTypeState: ATS,
 currentStep: BaseAgentNextStep,
 params?: ExecuteAgentParams,
): Promise<void>
Pre-processes an action step before execution.

This method is called by BaseAgent before action(s) are executed. Agent types can override this method to perform custom pre-processing, such as mapping payload values to action input parameters or modifying action configurations.
Type Parameters
- P = any
- ATS = any
Parameters
- actions: AgentAction[]
 The actions that will be executed (can be modified)
- currentPayload: P
 The current payload
- agentTypeState: ATS
- currentStep: BaseAgentNextStep
 The current step being executed
- Optionalparams: ExecuteAgentParams
Returns Promise<void>
Actions are modified in place

Since
2.76.0
Inherited from BaseAgentType.PreProcessActionStep
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:396

PreProcessNextStep

PreProcessNextStep(
 params: ExecuteAgentParams,
 step: BaseAgentNextStep,
 payload: P,
 agentTypeState: ATS,
): Promise<BaseAgentNextStep>
Pre-processes a retry step.

Realtime agents have no loop retry semantics; null is returned to indicate "no custom behavior", consistent with LoopAgentType.PreProcessNextStep.
Type Parameters
- P = any
- ATS = any
Parameters
- params: ExecuteAgentParams
 The full execution parameters (unused).
- step: BaseAgentNextStep
 The retry step (unused).
- payload: P
 The current payload (unused).
- agentTypeState: ATS
 The agent-type state (unused).
Returns Promise<BaseAgentNextStep>
Always null.
Overrides BaseAgentType.PreProcessNextStep
- Defined in AI/Agents/src/agent-types/realtime-agent-type.ts:155

`Protected` `Static`getAgentInstanceWithDriverClass

getAgentInstanceWithDriverClass(driverClass: string): Promise<BaseAgentType>
Protected
Instantiates an agent type class using a specific driver class name.

This method is used when an individual agent has its own DriverClass override, allowing for specialized implementations per agent instance.
Parameters
- driverClass: string
 The driver class name to instantiate
Returns Promise<BaseAgentType>
Instance of the agent type class
Inherited from BaseAgentType.getAgentInstanceWithDriverClass
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:266

`Protected` `Static`getAgentTypeInstance

getAgentTypeInstance(agentType: MJAIAgentTypeEntity): Promise<BaseAgentType>
Protected
Instantiates the appropriate agent type class based on the agent type entity.

This method uses the MemberJunction class factory to dynamically instantiate agent type classes. It uses the DriverClass field. If DriverClass is not specified it throws an error.
Parameters
- agentType: MJAIAgentTypeEntity
 The agent type entity to instantiate
Returns Promise<BaseAgentType>
Instance of the agent type class
Example
```
// For an agent type with DriverClass "LoopAgentType"
const agentTypeInstance = await this.getAgentTypeInstance(loopAgentType);

@protected
```
Inherited from BaseAgentType.getAgentTypeInstance
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:244

`Static`GetAgentTypeInstance

GetAgentTypeInstance(agentType: MJAIAgentTypeEntity): Promise<BaseAgentType>
Helper method that retrieves an instance of the agent type based on the provided agent type entity.

This method uses the ClassFactory to create an instance of the agent type class specified in the DriverClass field of the agent type entity. If the DriverClass is not specified, it throws an error.
Parameters
- agentType: MJAIAgentTypeEntity
 The agent type entity to instantiate
Returns Promise<BaseAgentType>
An instance of the agent type class

Async
Static
Method
GetAgentTypeInstance

Throws
If the agent type does not have a DriverClass specified or if instantiation fails
Inherited from BaseAgentType.GetAgentTypeInstance
- Defined in AI/Agents/src/agent-types/base-agent-type.ts:216

Class RealtimeAgentType

Hierarchy (View Summary)

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Returns RealtimeAgentType

Properties

Protected_jsonValidator

Static ReadonlyCURRENT_PAYLOAD_PLACEHOLDER

Static

Accessors

InjectLoopResultsAsMessage

Returns boolean

Since

IsSessionDriven

Returns boolean

RequiresAgentLevelPrompts

Returns boolean

Methods

OptionalAfterLoopIteration

Type Parameters

Parameters

Returns P

Since

OptionalBeforeLoopIteration

Type Parameters

Parameters

Returns { actionParams?: Record<string, unknown>; payload?: P; subAgentRequest?: { message: string; name: string; templateParameters?: Record<string, string>; };}

Since

ProtectedcreateNextStep

Type Parameters

Parameters

Returns BaseAgentNextStep<P>

ProtectedcreateRetryStep

Type Parameters

Parameters

Returns BaseAgentNextStep<P>

ProtectedcreateSuccessStep

Type Parameters

Parameters

Returns BaseAgentNextStep<P>

DetermineInitialStep

Type Parameters

Parameters

Returns Promise<BaseAgentNextStep<P>>

DetermineNextStep

Type Parameters

Parameters

Returns Promise<BaseAgentNextStep<P>>

GetPromptConfigurationGuidance

Returns string

Since

GetPromptForStep

Type Parameters

Parameters

Returns Promise<MJAIPromptEntityExtended>

HandleStepFallback

Type Parameters

Parameters

Returns Promise<BaseAgentNextStep<P>>

Since

InitializeAgentTypeState

Type Parameters

Parameters

Returns Promise<ATS>

InjectPayload

Type Parameters

Parameters

Returns Promise<void>

ProtectedparseJSONResponse

Type Parameters

Parameters

Returns T

PostProcessActionStep

Type Parameters

Parameters

`Protected`_jsonValidator

`Static` `Readonly`CURRENT_PAYLOAD_PLACEHOLDER

`Optional`AfterLoopIteration

`Optional`BeforeLoopIteration

Returns {
actionParams?: Record<string, unknown>;
payload?: P;
subAgentRequest?: {
message: string;
name: string;
templateParameters?: Record<string, string>;
};
}

`Protected`createNextStep

`Protected`createRetryStep

`Protected`createSuccessStep

`Protected`parseJSONResponse

`Protected` `Static`getAgentInstanceWithDriverClass

`Protected` `Static`getAgentTypeInstance

`Static`GetAgentTypeInstance