Member Junction
Preparing search index...
@memberjunction/ai-realtime-client
Module
Classes
AssemblyAIRealtimeClient
AudioLevelSmoother
BaseRealtimeClient
ChannelInboundVideoBridge
ElevenLabsRealtimeClient
FrameSampler
GeminiEnterpriseRealtimeClient
GeminiPcmPlayback
GeminiRealtimeClient
HuggingFaceRealtimeClient
LocalMediaController
MediaLayoutPrefs
MediaPreview
OpenAILiveClient
OpenAIProtocolRealtimeClient
OpenAIProtocolWebSocketRealtimeClient
OpenAIRealtimeClient
RealtimeAudioMeter
RealtimePcmPlayback
VideoFrameDecoderRegistry
VideoPlayout
VideoSourceArbiter
xAIRealtimeClient
Interfaces
AssemblyAIServerEvent
AudioLevelFrame
AudioLevelSmootherOptions
ChannelInboundVideoBridgeOptions
DisplayCapture
DisplayCaptureOptions
DisplayCaptureSupport
ElevenLabsServerEvent
FrameCaptureOptions
FrameSamplerOptions
FrameSize
GeminiClientConnectArgs
GeminiLiveClientSession
IAssemblyAIClientSocket
IAvatarVideoPlayout
IChannelFrameProvider
IElevenLabsClientSocket
IFrameCapture
ILocalMediaController
IOpenAIProtocolClientSocket
IPcmMicCapture
IPlaybackClock
IRealtimeAudioMeter
IRealtimeAudioSink
IRealtimeDataChannel
IRealtimeLivePeerConnection
IRealtimePcmPlayback
IRealtimePeerConnection
IRealtimeRtpSender
IVideoFrameDecoder
IVideoFrameSink
LocalMediaControllerOptions
LocalMediaState
LocalTrackState
MediaDevice
MediaDeviceSelection
MediaLayoutKeys
MediaLayoutSettings
MediaParticipant
MediaPipRect
MediaPlacementMove
MediaPreviewChoices
MediaPreviewOptions
MediaPreviewState
MediaStageInput
MediaStageLayout
MediaSurface
OAIProtocolAudioDelta
OAIProtocolConversationItemCreateEvent
OAIProtocolErrorEvent
OAIProtocolFunctionCallArgumentsDone
OAIProtocolFunctionCallOutputItem
OAIProtocolInputAudioBufferAppendEvent
OAIProtocolInputTranscriptionCompleted
OAIProtocolMessageItem
OAIProtocolOutputAudioBufferClearEvent
OAIProtocolOutputAudioBufferStarted
OAIProtocolOutputAudioBufferStopped
OAIProtocolResponseCancelEvent
OAIProtocolResponseCreated
OAIProtocolResponseCreateEvent
OAIProtocolResponseDone
OAIProtocolSessionCreated
OAIProtocolSessionUpdateEvent
OAIProtocolSpeechStarted
OAIProtocolTranscriptDelta
OAIProtocolTranscriptDone
OAIProtocolUnknownEvent
OAIProtocolUsage
RealtimeAudioActivity
RealtimeClientError
RealtimeClientToolCall
RealtimeClientTranscript
RealtimeClientUsage
SampledFrame
VideoFrameDecoderContext
VideoFrameDecoderRegistration
VideoPlayoutOptions
VideoSourceArbiterOptions
VideoSourceDescriptor
VideoSourceState
Type Aliases
CapturedDisplaySurface
DisplayCaptureFailure
DisplayCaptureResult
DisplayCaptureSurface
IGeminiAudioPlayback
IGeminiMicCapture
LocalMediaFailure
LocalMediaKind
LocalMediaResult
MediaAgentState
MediaConnectionQuality
MediaConnectionStatus
MediaDeviceKind
MediaDisconnectReason
MediaPlacement
MediaSourceKind
MediaTile
MediaVideoSource
OpenAIProtocolClientEvent
OpenAIProtocolServerEvent
PanelCaptureMethod
RealtimeClientState
SampledFrameMimeType
VideoFrameSinkSource
VideoPlayoutProblem
VideoSourceKind
VideoSourcePolicyRule
Variables
ASSEMBLYAI_AGENT_WS_URL
ASSEMBLYAI_PCM_SAMPLE_RATE
DEFAULT_AUDIO_ATTACK
DEFAULT_AUDIO_DECAY
DEFAULT_INBOUND_VIDEO_RATE
DEFAULT_VIDEO_SOURCE_POLICY
GEMINI_AVATAR_MP4_TYPE
HUGGINGFACE_DEFAULT_PCM_SAMPLE_RATE
MEDIA_LAYOUT_SAVED_LIMIT
MEDIA_PLACEMENTS
REALTIME_AUDIO_BIN_COUNT
REALTIME_CLIENT_TIME_LIMIT_ERROR_CODE
REQUESTED_TRACKS_SESSION_KEY
VIDEO_FRAME_JITTER_HEADROOM
XAI_CLIENT_SECRET_SUBPROTOCOL_PREFIX
XAI_PCM_SAMPLE_RATE
XAI_REALTIME_WS_URL
Functions
AttachVideoSource
base64ToArrayBuffer
Base64ToArrayBuffer
BucketizeFrequencyData
bytesToBase64
BytesToBase64
CapturedSurfaceOf
ComputeRmsLevel
createCameraCapture
CreateCameraCapture
createPcmMicCapture
CreatePcmMicCapture
createScreenCapture
CreateScreenCapture
createStreamFrameCapture
CreateStreamFrameCapture
encodeFloat32ToPcm16Base64
EncodeFloat32ToPcm16Base64
GateAudioLevel
GetDisplayCaptureSupport
IsMp4InitSegment
IsVideoCaptureKind
LayoutMediaStage
LoadAssemblyAIRealtimeClient
LoadElevenLabsRealtimeClient
LoadGeminiEnterpriseRealtimeClient
LoadGeminiRealtimeClient
LoadHuggingFaceRealtimeClient
LoadOpenAILiveClient
LoadOpenAIRealtimeClient
LoadxAIRealtimeClient
MinVideoFrameSpacingMs
NominalVideoFrameIntervalMs
ParsePipRects
ParsePlacementMoves
ParseSelfViewHidden
ParticipantTileKey
ParticipantTileSurface
pcm16ToFloat32
Pcm16ToFloat32
PlacementOffStage
RecordPipRect
RecordPlacementMove
RequestDisplayCapture
ResolveSurfacePlacements
ScaleToMaxDimension
SelectDisplayParticipants
SelectScreenSharer
SelectSplitSpeaker
SelectSpotlight
SerializePipRects
SerializePlacementMoves
SerializeSelfViewHidden
SmoothAudioBars
SmoothAudioLevel
StageParticipantIdentity
SynthesizeAudioBars
ToProviderSessionConfig
WithoutStageMoves
Settings
Member Visibility
Protected
Inherited
External
Theme
OS
Light
Dark
On This Page
Classes
Assembly
AI
Realtime
Client
Audio
Level
Smoother
Base
Realtime
Client
Channel
Inbound
Video
Bridge
Eleven
Labs
Realtime
Client
Frame
Sampler
Gemini
Enterprise
Realtime
Client
Gemini
Pcm
Playback
Gemini
Realtime
Client
Hugging
Face
Realtime
Client
Local
Media
Controller
Media
Layout
Prefs
Media
Preview
Open
AI
Live
Client
Open
AI
Protocol
Realtime
Client
Open
AI
Protocol
Web
Socket
Realtime
Client
Open
AI
Realtime
Client
Realtime
Audio
Meter
Realtime
Pcm
Playback
Video
Frame
Decoder
Registry
Video
Playout
Video
Source
Arbiter
x
AI
Realtime
Client
Interfaces
Assembly
AI
Server
Event
Audio
Level
Frame
Audio
Level
Smoother
Options
Channel
Inbound
Video
Bridge
Options
Display
Capture
Display
Capture
Options
Display
Capture
Support
Eleven
Labs
Server
Event
Frame
Capture
Options
Frame
Sampler
Options
Frame
Size
Gemini
Client
Connect
Args
Gemini
Live
Client
Session
I
Assembly
AI
Client
Socket
I
Avatar
Video
Playout
I
Channel
Frame
Provider
I
Eleven
Labs
Client
Socket
I
Frame
Capture
I
Local
Media
Controller
I
Open
AI
Protocol
Client
Socket
I
Pcm
Mic
Capture
I
Playback
Clock
I
Realtime
Audio
Meter
I
Realtime
Audio
Sink
I
Realtime
Data
Channel
I
Realtime
Live
Peer
Connection
I
Realtime
Pcm
Playback
I
Realtime
Peer
Connection
I
Realtime
Rtp
Sender
I
Video
Frame
Decoder
I
Video
Frame
Sink
Local
Media
Controller
Options
Local
Media
State
Local
Track
State
Media
Device
Media
Device
Selection
Media
Layout
Keys
Media
Layout
Settings
Media
Participant
Media
Pip
Rect
Media
Placement
Move
Media
Preview
Choices
Media
Preview
Options
Media
Preview
State
Media
Stage
Input
Media
Stage
Layout
Media
Surface
OAI
Protocol
Audio
Delta
OAI
Protocol
Conversation
Item
Create
Event
OAI
Protocol
Error
Event
OAI
Protocol
Function
Call
Arguments
Done
OAI
Protocol
Function
Call
Output
Item
OAI
Protocol
Input
Audio
Buffer
Append
Event
OAI
Protocol
Input
Transcription
Completed
OAI
Protocol
Message
Item
OAI
Protocol
Output
Audio
Buffer
Clear
Event
OAI
Protocol
Output
Audio
Buffer
Started
OAI
Protocol
Output
Audio
Buffer
Stopped
OAI
Protocol
Response
Cancel
Event
OAI
Protocol
Response
Created
OAI
Protocol
Response
Create
Event
OAI
Protocol
Response
Done
OAI
Protocol
Session
Created
OAI
Protocol
Session
Update
Event
OAI
Protocol
Speech
Started
OAI
Protocol
Transcript
Delta
OAI
Protocol
Transcript
Done
OAI
Protocol
Unknown
Event
OAI
Protocol
Usage
Realtime
Audio
Activity
Realtime
Client
Error
Realtime
Client
Tool
Call
Realtime
Client
Transcript
Realtime
Client
Usage
Sampled
Frame
Video
Frame
Decoder
Context
Video
Frame
Decoder
Registration
Video
Playout
Options
Video
Source
Arbiter
Options
Video
Source
Descriptor
Video
Source
State
Type Aliases
Captured
Display
Surface
Display
Capture
Failure
Display
Capture
Result
Display
Capture
Surface
I
Gemini
Audio
Playback
I
Gemini
Mic
Capture
Local
Media
Failure
Local
Media
Kind
Local
Media
Result
Media
Agent
State
Media
Connection
Quality
Media
Connection
Status
Media
Device
Kind
Media
Disconnect
Reason
Media
Placement
Media
Source
Kind
Media
Tile
Media
Video
Source
Open
AI
Protocol
Client
Event
Open
AI
Protocol
Server
Event
Panel
Capture
Method
Realtime
Client
State
Sampled
Frame
Mime
Type
Video
Frame
Sink
Source
Video
Playout
Problem
Video
Source
Kind
Video
Source
Policy
Rule
Variables
ASSEMBLYAI_
AGENT_
WS_
URL
ASSEMBLYAI_
PCM_
SAMPLE_
RATE
DEFAULT_
AUDIO_
ATTACK
DEFAULT_
AUDIO_
DECAY
DEFAULT_
INBOUND_
VIDEO_
RATE
DEFAULT_
VIDEO_
SOURCE_
POLICY
GEMINI_
AVATAR_
MP4_
TYPE
HUGGINGFACE_
DEFAULT_
PCM_
SAMPLE_
RATE
MEDIA_
LAYOUT_
SAVED_
LIMIT
MEDIA_
PLACEMENTS
REALTIME_
AUDIO_
BIN_
COUNT
REALTIME_
CLIENT_
TIME_
LIMIT_
ERROR_
CODE
REQUESTED_
TRACKS_
SESSION_
KEY
VIDEO_
FRAME_
JITTER_
HEADROOM
XAI_
CLIENT_
SECRET_
SUBPROTOCOL_
PREFIX
XAI_
PCM_
SAMPLE_
RATE
XAI_
REALTIME_
WS_
URL
Functions
Attach
Video
Source
base64
To
Array
Buffer
Base64
To
Array
Buffer
Bucketize
Frequency
Data
bytes
To
Base64
Bytes
To
Base64
Captured
Surface
Of
Compute
Rms
Level
create
Camera
Capture
Create
Camera
Capture
create
Pcm
Mic
Capture
Create
Pcm
Mic
Capture
create
Screen
Capture
Create
Screen
Capture
create
Stream
Frame
Capture
Create
Stream
Frame
Capture
encode
Float32
To
Pcm16
Base64
Encode
Float32
To
Pcm16
Base64
Gate
Audio
Level
Get
Display
Capture
Support
Is
Mp4
Init
Segment
Is
Video
Capture
Kind
Layout
Media
Stage
Load
Assembly
AI
Realtime
Client
Load
Eleven
Labs
Realtime
Client
Load
Gemini
Enterprise
Realtime
Client
Load
Gemini
Realtime
Client
Load
Hugging
Face
Realtime
Client
Load
Open
AI
Live
Client
Load
Open
AI
Realtime
Client
Loadx
AI
Realtime
Client
Min
Video
Frame
Spacing
Ms
Nominal
Video
Frame
Interval
Ms
Parse
Pip
Rects
Parse
Placement
Moves
Parse
Self
View
Hidden
Participant
Tile
Key
Participant
Tile
Surface
pcm16
To
Float32
Pcm16
To
Float32
Placement
Off
Stage
Record
Pip
Rect
Record
Placement
Move
Request
Display
Capture
Resolve
Surface
Placements
Scale
To
Max
Dimension
Select
Display
Participants
Select
Screen
Sharer
Select
Split
Speaker
Select
Spotlight
Serialize
Pip
Rects
Serialize
Placement
Moves
Serialize
Self
View
Hidden
Smooth
Audio
Bars
Smooth
Audio
Level
Stage
Participant
Identity
Synthesize
Audio
Bars
To
Provider
Session
Config
Without
Stage
Moves
Member Junction
Loading...