OptionalAudioAudio-modality tokens.
OptionalCachedTokens served from the provider's prompt cache (billed at the cached rate).
OptionalImageImage-modality tokens (input only on current providers). Authoritative financial billing basis reported by inference providers (e.g. Gemini Live reports inbound video frames under promptTokensDetails.IMAGE).
OptionalTextText-modality tokens.
OptionalVideoInbound only: the video frames sent so far on the session's video tracks (usage basis 'frames'), a running total rather than an amount for this update. Client-side telemetry; comparing it against ImageTokens surfaces dropped frames.
OptionalVideoSeconds of video (usage basis 'seconds'). The two directions mean different things:
InputTokenDetails): the wall-clock span between the first and last frame sent so
far (span-not-sum), a running total for telemetry. Provider-reported ImageTokens remain the
billing basis.OutputTokenDetails): the seconds of video the model generated since the last
update (an avatar's video, from its fragment durations), an amount like the token fields.
Consumers add the updates up.OptionalVideoVideo-modality tokens, as the provider reports them (Gemini Live: responseTokensDetails VIDEO for
a generated avatar's video). Part of the update's output total when the provider counts them there.
Per-modality token counts inside a RealtimeUsage update. All fields optional — providers report different subsets (OpenAI GA: text/audio/cached on input, text/audio on output; Gemini Live: text/audio/image on input, text/audio/video on output).
Every token field is an amount for this update, like the totals. The video fields differ by direction: inbound they are the session's running totals (snapshots), outbound
VideoSecondsis an amount for this update.