Interface IRealtimeVideoConformanceHarness

ReadonlyName

ReadonlyTraits

OptionalAnswerStarted

• AnswerStarted?(): Promise<void>

  The model starts speaking its answer: its first words, as the provider transcribes them (Gemini: the output transcription). The video that follows is the answer's until the turn's generation completes; video outside an answer (an avatar's between answers, which Vertex AI streams) plays, but is not the agent speaking. Without it the kit can't tell the two apart, and VC13 leaves that part out.

  Returns Promise<void>

OptionalAskedProviderForVideo

• AskedProviderForVideo?(): boolean

  Whether the last connect asked the provider for video: an avatar nobody sees is still generated, and billed.

  Returns boolean

Connect

• Connect(
      client: BaseRealtimeClient,
      config: ClientRealtimeSessionConfig,
      microphone: MediaStream,
  ): Promise<void>

  Connects the client, and opens whatever the fake transport needs, until the session listens.

  Parameters

  • client: BaseRealtimeClient
  • config: ClientRealtimeSessionConfig
  • microphone: MediaStream

  Returns Promise<void>

CreateClient

• CreateClient(media: RealtimeVideoConformanceMedia): BaseRealtimeClient

  A client driver with a fake transport, its creation seams wired to media. Register no handlers on it: the kit registers its own. A check may run two sessions one after the other, so a second call starts over with a new client.

  Parameters

  • media: RealtimeVideoConformanceMedia

  Returns BaseRealtimeClient

OptionalDispose

• Dispose?(): void | Promise<void>

  Releases what the harness set up. Called once after each check, failed ones too.

  Returns void | Promise<void>

OptionalGenerationComplete

• GenerationComplete?(): Promise<void>

  The model has generated the whole turn; its media may still be arriving. Without it the kit uses TurnComplete.

  Returns Promise<void>

Interrupted

• Interrupted(): Promise<void>

  The user barged in: the provider stopped the model's turn.

  Returns Promise<void>

Mint

• Mint(grant: RealtimeVideoConformanceGrant): ClientRealtimeSessionConfig

  The session config the provider's server mints for the grant. The kit adds the host's track request to it.

  Parameters

  • grant: RealtimeVideoConformanceGrant

  Returns ClientRealtimeSessionConfig

OptionalRefusePlayback

• RefusePlayback?(): () => void

  Makes this browser unable to play the provider's video, before the next connect. Returns the undo.

  Returns () => void

OptionalResume

• Resume?(): Promise<void>

  A planned move to a new connection, mid-turn (the provider's session resumption). Resolves once the session listens again.

  Returns Promise<void>

OptionalSendPart

• SendPart?(mimeType: string, data: ArrayBuffer): Promise<void>

  The model sends a part of any MIME type, as the provider's wire names it.

  Parameters

  • mimeType: string
  • data: ArrayBuffer

  Returns Promise<void>

SendVideo

• SendVideo(frame: RealtimeVideoFrame): Promise<void>

  The model sends a frame of the agent's video, of the provider's kind (RealtimeVideoConformanceTraits.VideoFrameKind), as Core's RealtimeVideoFrame: the harness puts it on its wire as the provider would. Resolves once the client has handled it.

  Parameters

  • frame: RealtimeVideoFrame

  Returns Promise<void>

SendVoice

• SendVoice(pcm16: ArrayBuffer, mediaTimeMs?: number): Promise<void>

  The model sends PCM16 of the agent's voice. Resolves once the client has handled it.

  Parameters

  • pcm16: ArrayBuffer

    The chunk.

  • OptionalmediaTimeMs: number

    Where the chunk starts on the stream's media timeline, for a provider with RealtimeVideoConformanceTraits.TimedVoice; a provider whose wire has no time ignores it.

  Returns Promise<void>

TurnComplete

• TurnComplete(): Promise<void>

  The model's turn is over.

  Returns Promise<void>