Changelog
Back to All

v5.41.0

14 days ago by Caeleb Balanesi

New Features

  • Real-Time AI Agents — Sessions, Channels & the Realtime Model. New AIAgentSession / AIAgentChannel / AIAgentSessionChannel schema and the BaseRealtimeModel server primitive (OpenAI Realtime + Gemini Realtime drivers, both server-bridged and client-direct ephemeral-token topologies). Adds the new @memberjunction/ai-realtime-client browser package, the Realtime agent type and Voice Co-Agent, full session lifecycle (manager, janitor, start/close/heartbeat, review-mode resume), and the full-panel realtime voice call UX in Conversations (phone trigger, agent/model picker, delegation cards, paced first-person progress narration, in-call text composer).
  • Realtime Bridges (Phase 0+1) — meetings & telephony. A new media-transport layer connecting the realtime agent engine to external endpoints: meetings (Zoom/Teams/Slack/Meet/Webex/Discord) and telephony (Twilio/Vonage/RingCentral/VOIP). New @memberjunction/ai-bridge-base and @memberjunction/ai-bridge-server packages with the BaseRealtimeBridge media driver, capability gating, a platform-agnostic turn-taking policy, a LoopbackBridge, and 5 new bridge entities. Media is fully type-agnostic (directional audio/video/screen tracks).
  • Remote Browser channel. An in-house realtime channel where an agent drives a live, CDP-connected browser while it talks — sales demos, support walkthroughs, trainer agents. New AIRemoteBrowserProvider registry with capability gating, a universal base + shared CDP kit, a server engine with control modes (AgentOnly/ViewOnly/Collaborative), and five thin backends (Self-Hosted Chrome, Browserbase, Steel, Browserless, Hyperbrowser).
  • New realtime voice providers. xAI Grok Voice (OpenAI-Realtime-compatible, client-direct) and Inworld join OpenAI, Gemini, ElevenLabs Agents, and AssemblyAI Voice Agent — a multi-provider realtime matrix with zero contract changes.
  • Collaborative Whiteboard channel (@memberjunction/ng-whiteboard). Agent-drivable board with a tool API, OneNote-style multi-page support, sandboxed interactive widgets, markdown panels, multi-select/marquee editing, exports, and cancelable before/after events.
  • Realtime co-agent pairing & type configuration. New MJ: AI Agent Paired Agents junction for opt-in co-agent targeting, plus per-agent and per-type JSON configuration (realtime model preference, voice, tone, narration pacing) with server-authoritative deep-merge precedence and authorization-gated runtime overrides.
  • Agent in-flight memory writes. Agents can commit durable cross-run memories mid-run via the memoryWrites loop field (gated by AIAgent.AllowMemoryWrite, on by default). Writes land as immediately-injectable Provisional notes under framework guards (scope clamp, dedupe, per-run cap, TTL) and are hardened/pruned by a new Memory Manager pass each cycle.
  • Multimodal embeddings. New EmbedContent method + GetFileCapabilities on BaseEmbeddings, with GeminiEmbedding2 and Cohere providers and AIEngine.EmbedContent (includes the @google/genai 2.x bump).
  • Skip entity-form support. #entity mentions in conversations, interactive-form host wiring, and reusable form-field components.
  • Interactive components persist savedUserSettings per-user, cross-device via UserInfoEngine, automatically scoped per component, with merge-not-replace semantics and an opt-out input. Fully additive — existing components gain durable persistence for free.
  • Heartbeat-based lease renewal for scheduled jobs. Running jobs can opt in via context.heartbeat() to keep their concurrency slot alive, plus a new ScheduledJob.MaxRuntimeMinutes override for single-long-call jobs.
  • Entity permission checking resolver + GraphQL client method for verifying a user's read access on entities.
  • Cascade delete on MJ: Entity Documents.

Improvements

  • Realtime UX wave 2 — progressive-disclosure voice console (pure-audio-first overlay, breathing hero orb, per-user disclosure levels, unified app bar, fused composer dock) and audio-reactive call visuals driven by a per-direction RMS + spectrum metering capability across all four drivers.
  • Realtime AI Agents wave 3 — consolidated v5.41 migration with the AIAgentCoAgent affinity registry replacing AIAgentPairedAgent (typed relationship vocabulary, type-level defaults as junction rows removing the only FK cycle in core MJ), plus CodeGen single-winner IsNameField enforcement and clean-database bootstrap robustness.
  • Realtime co-agent direct channel control — the voice co-agent now drives interactive channels (browser_, Whiteboard_) directly instead of delegating every request, with Auto/Default model resolution that walks candidates by power, full co-agent observability (transcript turns + channel tool calls captured on the AIPromptRun), and a generalized channel-onboarding panel.
  • Realtime ledger completion — the server-side channel plugin half (BaseRealtimeChannelServer + RealtimeChannelServerHost with a WhiteboardChannelServer reference impl), end-to-end transcript corrections, and an AssemblyAI resume window on socket drops.
  • Provider threading through the duplicate-detection stack — VectorBase, EntityVectorSyncer, and the dupe detector now use the request-scoped IMetadataProvider instead of the process-global one.
  • Voice* → Realtime* rename across conversation session/adapter/component symbols (no functional change).
  • Default app agent changed to Sage (metadata) and CLI Skyway dependencies bumped to ^0.6.2.

Bug Fixes

  • RunView cache-miss now projects to the caller's requested Fields instead of returning all entity columns, and includes Fields/ResultType in the dedup and client-cache fingerprint keys so concurrent callers with different field subsets no longer receive each other's result shapes.
  • CodeGen SP generation fixed for PK-only entities, mj sync push now returns a correct non-zero exit code on failure, and API Key scope checks use GetSystemUser.
  • Stale collection contents after removing an artifact fixed by reading collection membership from the join entity directly so the per-entity RunView cache invalidates correctly.
  • Removed committed merge-conflict markers in scripts/run-dupe-detection.ts.
  • Removed the dead RealtimeCallOverlayComponent that a rename-vs-delete merge had silently resurrected.