The bytes. A consumer may transfer the buffer to another thread: read what you need before handing it on.
OptionalHeightThe picture's height in pixels, when known.
OptionalIdletrue when the frame is video the model streams between its answers (Vertex AI sends an avatar's face then too):
it plays, but the agent is not speaking, whatever audio the frame carries, so a room's floor gate never counts it
as speech. Set by a driver that can tell; absent when it can't, or when the frame is part of an answer.
OptionalKeyWhether a decoder can start here: a key frame, or an fMP4 fragment whose first video sample is one.
The format: 'video/mp4' for an fMP4 piece; for a chunk, its codec ('video/vp8',
'video/h264; codecs="avc1.42e01f"'); for an image, its type ('image/jpeg').
'init': an init segment (ftyp + moov). 'fragment': media data (moof + mdat, or a part of one).
OptionalPresentationWhen the frame shows, in milliseconds on the stream's own media timeline, not wall-clock time. The voice that goes with it is on the same timeline: a player shows the frame when the voice reaches this time. A new stream (an fMP4 init segment, or a key frame whose time goes backwards) may restart it.
OptionalWidthThe picture's width in pixels, when known.
A piece of a fragmented MP4 stream as the model sent it. Its tracks may include the voice (a Gemini avatar's MP4 has an AAC track): then the voice does not also come as PCM.