Video Studio Guide
1. What it is
Section titled “1. What it is”Video Studio turns a creative brief, an organization’s Brand Kit (MJ: Brand Kits and MJ: Brand Kit Assets), and governed MemberJunction data (MJ: Queries, RunView, or MJ: Artifact Versions) into deterministic, broadcast-grade multi-aspect MP4 films (16:9, 9:16, 1:1), poster stills, WebVTT captions, contact sheets, and a versioned Video Production (MJ: Artifacts) deliverable. The entire production lifecycle — from creative brief to final delivery — is governed by a single typed VideoGenerationSpec JSON contract (VIDEO_STUDIO_SPEC_VERSION = '1.0').
Video Studio separates code-rendered motion from generated footage. Every scene beat is authored as deterministic HTML/CSS/SVG/GSAP motion for HyperFrames (0.8.134) and rendered frame-accurately in headless Chromium + ffmpeg, ensuring that brand typography, color palettes, logos, product screenshots, and data charts (MJChartKit) are pixel-exact and never hallucinated. Generative AI (FLUX 3 Image via AIImageGenerationRunner and FLUX 3 Video via AIVideoRunner) is used only when a beat explicitly requests atmospheric B-roll, conceptual illustrations, or transition clips under Assets.Generate; every request is screened before spend against EffectiveBrandGenerationPolicy and NeverInvent (ScreenGenerationRequest) and disclosed with full provenance (ModelID, PromptRunID, C2PA, Cost, TotalVideoSeconds, TotalCost; the stored MJ: Files row’s Description also records job=<VendorJobID>).
2. Architecture
Section titled “2. Architecture”Workspace and UI packages
Section titled “Workspace and UI packages”| Package | Path | Runtime boundary & key exports |
|---|---|---|
@memberjunction/video-studio-core | packages/AI/VideoStudio/Core | Browser + Server (zero Node/DB imports): VideoGenerationSpec, pure validator ValidateVideoGenerationSpec, MJChartKit (CreateMJChartKit, WithChartKit), data literal helpers (BuildDataCheckInputs, FormatAsOfCardText), DescribeGeneratedAssets, and the render worker wire contract (RENDER_WORKER_ROUTES in render-contract.ts). |
@memberjunction/video-studio | packages/AI/VideoStudio/Engine | Server / MJAPI: VideoStudioAgent and stage drivers (VideoStudioArtDirectorAgent, VideoStudioPreProductionAgent, VideoStudioMotionEngineerAgent, VideoStudioRenderAgent, VideoStudioCriticAgent, VideoStudioAgentBase), ValidateVideoGenerationSpecAsync, EvaluateGates, CaptureDataSnapshot, RunNarrationPipeline, GenerateRequestedAssets, VideoRenderEngine (LocalWorkerRenderDriver, RemoteRenderDriver, DockerRenderDriver), StoreFile / ReadStoredFile, and ProduceVideoAction. |
@memberjunction/video-studio-render-worker | packages/AI/VideoStudio/RenderWorker | Isolated Node >=22 worker / container: CreateRenderWorkerServer (server.ts), one-shot CLI (one-shot.ts), StageRenderBundle (bundle.ts), InjectPreambleIntoHtml (preamble.ts), RunLintGate (lint-gate.ts), and HyperFramesRenderBackend (hyperframes-backend.ts). |
@memberjunction/core-entities | packages/MJCoreEntities | Metadata caches BrandKitEngine (MJ: Brand Kits, MJ: Brand Kit Assets) and VideoStudioEngine (MJ: Video Archetypes). |
@memberjunction/ng-brand-kits & @memberjunction/ng-media-player | packages/Angular/Generic/brand-kits, packages/Angular/Generic/media-player | Reusable Brand Kit editors (<mj-brand-kit-assets-editor>, <mj-brand-kit-generation-policy-editor>) and streaming video/audio player (<mj-media-player>, <mj-storage-media-player>). |
@memberjunction/ng-artifacts & @memberjunction/ng-dashboards | packages/Angular/Generic/artifacts, packages/Angular/Explorer/dashboards/src/VideoStudio | VideoProductionViewerComponent (VideoProductionViewerPlugin), VideoProductionArtifactPreviewComponent, <mj-video-format-glyph>, <mj-video-beat-strip>, FormatVideoTime / GetVideoStatusBadgeVariant, and the Explorer Video Studio application (Productions, Brand Kits, Archetypes, sharing a single right-side ng-conversations slide-in host VSStudioSlideInComponent + VSStudioSlideInStateService pinned to Video Studio across all three pages and hosting VSProductionBriefComponent in its empty state). |
Root orchestrator and seven stage sub-agents
Section titled “Root orchestrator and seven stage sub-agents”VideoStudioAgent (packages/AI/VideoStudio/Engine/src/agents/video-studio-agent.ts) drives a seven-stage state machine defined in VIDEO_STUDIO_STAGES and seeded in metadata/agents/.video-studio-agent.json:
| Order | Stage Sub-Agent (Name) | DriverClass / execution mode | Spec Status transition & responsibility |
|---|---|---|---|
1 | Video Studio - Director | LoopAgentType (Video Studio - Director prompt) | Briefing → Briefing: Selects the archetype, target duration/formats (Film), DataBindings, and initial Assets.Approved / Assets.Required mapping (G1 Facts gate is evaluated by VideoStudioAgent; a 'Blocked' G1 stops the run). |
2 | Video Studio - Art Director | VideoStudioArtDirectorAgent (art-director-agent.ts) | Briefing → Styled: Inspects up to MAX_REFERENCES = 6 reference media items via Video Studio - Reference Analysis (PickReferences / BuildReferenceBlocks, inlining video up to MAX_INLINE_REFERENCE_VIDEO_BYTES = 20 * 1024 * 1024 when the primary model for Video Studio - Reference Analysis supports video input via ResolvePromptPrimaryModelId, or extracting up to MAX_KEYFRAME_IMAGES = 4 keyframes via ExtractKeyframes), then runs Video Studio - Art Director to populate Style (Palette, Typography, Composition, Pacing, MotionRules, References). |
3 | Video Studio - Storyboard | LoopAgentType (Video Studio - Storyboard prompt) | Styled → Storyboarded (then Approved once G2 Storyboard approval passes): Designs contiguous scene Beats, Audio.Narration.Script, Audio.Captions, Audio.Music, Audio.Cues, and any Assets.Generate requests. |
4 | Video Studio - Pre-Production | VideoStudioPreProductionAgent (pre-production-agent.ts) | Approved → PreProduced: Deterministic 4-step 'Tool' pipeline ("Data Snapshot", "Generate Assets", "Narration", "Stage Assets"; Pre-Production records no gate). |
5 | Video Studio - Motion Engineer | VideoStudioMotionEngineerAgent (motion-engineer-agent.ts) | PreProduced → Composed: Runs Video Studio - Motion Engineer per deliverable format to author Output.Source[fmt], runs static CheckProductionRules + driver.Check(..., { StaticOnly: true }), and repairs lint errors up to MAX_LINT_ATTEMPTS = 3 via Video Studio - Motion Engineer Lint Fix. |
6 | Video Studio - Render | VideoStudioRenderAgent (render-agent.ts) | Composed → RoughCut: Executes 4 steps per format ("Check <fmt>", "Stills <fmt>", "Contact sheets <fmt>", "Render <fmt>"), uploads MP4/stills/sheets via StoreFile, and records gate G3 once every deliverable format has contact sheets. |
7 | Video Studio - Critic | VideoStudioCriticAgent (critic-agent.ts) | RoughCut → Reviewed: Reviews contact sheets and transition strips (PickSheetsForCall) via Video Studio - Critic, scores the Video Production Quality rubric (RubricScoring.Compute, MAX_DEFECTS_PER_FORMAT = 3), and writes Review.Rounds and Review.Scores (the Critic records no gate; the root VideoStudioAgent evaluates all six gates G1–G6 via EvaluateGates and sets Status = 'Delivered' only after all six pass). |
flowchart LR
D["1. Director (Briefing / G1 Facts)"] --> AD["2. Art Director (Styled)"]
AD --> SB["3. Storyboard (Storyboarded)"]
SB --> G2{"G2 Storyboard Approval"}
G2 -- "Approved" --> PP["4. Pre-Production (PreProduced)"]
PP --> ME["5. Motion Engineer (Composed)"]
ME --> RN["6. Render (G3 / RoughCut)"]
RN --> CR["7. Critic (Reviewed)"]
CR --> EV{"EvaluateGates (G1-G6)"}
EV -- "All G1-G6 pass" --> DEL["Delivered"]
EV -- "Code defect (rounds < MaxRepairRounds)" --> ME
EV -- "Asset regen (RegenerationCount < MaxRegenerations)" --> PP
EV -- "Unrecoverable gate/integrity failure" --> FLD["Step: Failed"]
3. The VideoGenerationSpec
Section titled “3. The VideoGenerationSpec”VideoGenerationSpec (packages/AI/VideoStudio/Core/src/video-generation-spec.ts; see also packages/AI/VideoStudio/Core/README.md and packages/AI/VideoStudio/Core/generated-for-prompt/video-generation-spec.ts.generated-for-prompt.md) is the single payload mutated across every stage of a production (SpecVersion: '1.0').
Facts vs. creative choices
Section titled “Facts vs. creative choices”The spec separates caller/system facts from agent-authored creative choices:
- Authoritative facts (
CompanyID,BrandKitID,Archetype,Interaction,DataBindings,DataSnapshot,Assets.Approved,Assets.Required,Assets.NeverInvent): Seeded by the caller (ProduceVideoAction, Studio Start form, orSubAgentInputMapping) and resolved by deterministic Pre-Production tools (CaptureDataSnapshot,BrandKitEngine). Stage prompts cannot invent inline data or bypass brand policies. - Creative and runtime sections (
Film,Style,Beats,StateList,Audio(Audio.Narration.Script,Audio.Captions,Audio.Music,Audio.Cues),Assets.Generate,Render,Deliverables,Review,Output): Populated progressively by the seven stage sub-agents asStatusadvances through the 10VIDEO_PRODUCTION_STATUSES:'Briefing'→'Styled'→'Storyboarded'→'Approved'→'PreProduced'→'Composed'→'RoughCut'→'Reviewed'→'Delivered'(or'Blocked').
Pure and async validation
Section titled “Pure and async validation”- Stage-aware pure validator (
ValidateVideoGenerationSpecinpackages/AI/VideoStudio/Core/src/validate-spec.ts):- Enforces
STAGE_REQUIRED_SECTIONS(VideoSpecSectionlabels invalidate-spec.ts) for the currentStatus: requires['Film']at'Briefing'(and'Blocked'); adds'Style'at'Styled'; adds'Beats'and'NarrationScript'at'Storyboarded'(and'Approved'); adds'PreProduction'at'PreProduced'(through'Reviewed'); and adds'Output'only at'Delivered'(there is no'Review'entry inSTAGE_REQUIRED_SECTIONS). - Validates contiguous beat timelines (
BEAT_TIMELINE_TOLERANCE_SECONDS = 0.05viaBeatsMustStartAtZero,NonContiguousBeats,BeatsDurationMismatch), asset and reference resolution (UnmetMustUseRequirement,GeneratedAssetNotAllowedForRequirement,UnresolvedReferenceKey), data binding integrity (UnknownOnScreenPlaceholder,UnboundNumericOnScreenText,MissingDataSnapshot), and deliverable storage (MissingOutputManifest,MissingOutputVideoPayload,OutputVideoFormatKeyMismatch) across all 33VIDEO_SPEC_ERROR_CODES.
- Enforces
- Server-side async validator (
ValidateVideoGenerationSpecAsyncinpackages/AI/VideoStudio/Engine/src/validate-spec-async.ts):- Runs
ValidateVideoGenerationSpecfirst, then checksBrandKitEngine(BrandKitNotFound,BrandKitNotInScope),VideoStudioEngine(ArchetypeNotFound), and batchedMJ: Filesexistence andStatus === 'Uploaded'(FILE_VALIDATION_BATCH_SIZE = 500,InvalidFileID,FileNotReadable,AsyncValidationFailed).
- Runs
4. Power levels
Section titled “4. Power levels”Video Studio ships three MJ: AI Configurations presets in metadata/ai-configurations/.video-studio-configurations.json, parsed at runtime by ResolveVideoStudioSettings / ParseVideoStudioSettings (packages/AI/VideoStudio/Engine/src/settings.ts, where DEFAULT_VIDEO_STUDIO_SETTINGS matches Standard):
| Setting / model | Draft (Video Studio Draft) | Standard (Video Studio Standard, default) | High (Video Studio High) |
|---|---|---|---|
Parent MJ: AI Configurations | Fast | Standard | High Power |
| Primary creative/critic LLM | Gemini 3.8 Flash (EffortLevel: 25) | Gemini 3.8 Flash | Claude Opus 5.5 |
VideoStudio.RenderQuality & FrameSizeFor | 'draft' (1280×720, 720×1280, 720×720) | 'standard' (1920×1080, 1080×1920, 1080×1080) | 'high' (2560×1440, 1440×2560, 1440×1440) |
VideoStudio.Fps | 24 | 30 | 30 |
VideoStudio.Formats | 'First' (renders Film.Formats[0] only) | 'All' | 'All' |
VideoStudio.MaxRepairRounds | 0 | 2 | 3 |
VideoStudio.GenerationQuality | 'draft' | 'standard' | 'high' |
VideoStudio.MaxGeneratedVideoSeconds | 20 | 40 | 60 |
VideoStudio.MaxRegenerations | 1 | 1 | 1 |
Callers select a power level via:
- The
@Video Studiomention preset pin or conversation preset picker (saved inMJ: User SettingsunderAgentModeSettingKey), - The
PowerLevelinput parameter ('Draft' | 'Standard' | 'High') onProduce Video, ConfigurationIDonAgentScheduledJobDriverorExecuteAgentParams.configurationId, orMJ: AI Agent Relationships.ConfigurationPresetID(seeded toStandardforQuery BuilderandResearch Report Writer).
5. Gates, critic, and the Video Production Quality rubric
Section titled “5. Gates, critic, and the Video Production Quality rubric”Gates G1–G6 (EvaluateGates) and Interaction.Mode
Section titled “Gates G1–G6 (EvaluateGates) and Interaction.Mode”EvaluateGates (packages/AI/VideoStudio/Engine/src/gates.ts) evaluates six production gates on Review.Gates (Status = 'Delivered' is set by VideoStudioAgent only after all six gates G1–G6 pass):
| Gate | Name (gates.ts) | Pass requirements |
|---|---|---|
G1 | Facts (evaluateGateG1) | Brief fields (Film.OneSentence, Film.Audience, Film.DurationSeconds > 0, and a non-empty format list in Film.Formats / Deliverables.Formats) are present (a missing one makes G1 'Failed', not 'Blocked'; Archetype, BrandKitID, and CompanyID are not checked for presence — with no BrandKitID, VideoStudioAgent seeds the default kit or none and counts it in scope). 'Blocked' (which stops the run) is returned when BrandKitInScope === false (out-of-scope BrandKitID), ValidateVideoGenerationSpec at 'Briefing' reports UnmetMustUseRequirement or GeneratedAssetNotAllowedForRequirement, or a DataBindings entry is malformed/unresolved (missing Key, invalid Source.Kind, 'Query' without QueryID, 'View' without EntityName and non-empty Fields, 'Artifact' without ArtifactVersionID, or 'Inline' Key missing from DataSnapshot.Inline). |
G2 | Storyboard / Plan approval (evaluateGateG2) | ValidateVideoGenerationSpec passes at 'Storyboarded' (Beats, Style, NarrationScript), then passes immediately when effective Interaction.Mode === 'Autonomous' or stays 'Pending' (Notes: 'AwaitingApproval') under 'ApproveStoryboard' / 'ApproveStoryboardAndRoughCut' until the G2 GateRecord has Notes === 'Approved'. |
G3 | Stills / Contact sheets (evaluateGateG3) | Preserves existingG3 (recorded as 'Passed' by VideoStudioRenderAgent after the last format) only when existingG3 is present and every deliverable format has non-empty Output.ContactSheetFileIDs[fmt]; otherwise G3 is 'Pending'. |
G4 | Rough cut (evaluateGateG4) | Every deliverable format has Output.Videos[fmt] (FileID or InlineBase64) and at least one Review.Rounds entry was completed at or after Output.RenderMetadata.RenderedAt; stays 'Pending' (Notes: 'AwaitingApproval') under 'ApproveStoryboardAndRoughCut' until the G4 GateRecord has Notes === 'Approved'. |
G5 | Repair (evaluateGateG5) | Requires a Review.Rounds entry with Timestamp >= Output.RenderMetadata.RenderedAt (hasCriticRoundOnOrAfterRender) and no terminal failure from FindIntegrityOrGateCriterionFailure, with either no open 'Blocker', 'Major', or generated-assets-are-honest-and-disclosed defect (an open generated-assets-are-honest-and-disclosed defect blocks G5 even below 'Major' severity), or the repair round cap (MaxRepairRounds) reached with zero open generated-assets-are-honest-and-disclosed defects (shown-values-match-the-data-snapshot acts only as a rubric gate criterion through FindIntegrityOrGateCriterionFailure, not as an open-defect check in evaluateGateG5). |
G6 | Deliverables (evaluateGateG6) | Every deliverable format has Output.Videos[fmt] (FileID or InlineBase64), Output.PosterFileID is set when Deliverables.Poster is true, Output.CaptionsFileID is set when Audio.Captions === 'Sidecar' and Audio.Narration.Script has lines, every line in Audio.Narration.Script has AudioFileID, and Output.RenderMetadata.RenderedAt is set. |
Approval modes (Interaction.Mode)
Section titled “Approval modes (Interaction.Mode)”Interaction.Mode (VideoInteractionMode in video-generation-spec.ts) supports three modes: 'Autonomous' | 'ApproveStoryboard' | 'ApproveStoryboardAndRoughCut'. VideoStudioAgent resolves the effective interaction mode in priority order:
- Caller-supplied
spec.Interaction?.Mode, if set; 'ApproveStoryboard'whenExecuteAgentParams.planMode === true;'Autonomous'when running below the root (params.parentRunorAgentHierarchy.length > 1/ sub-agent invocation, soQuery BuilderandResearch Report Writerconsult runs execute in'Autonomous'mode and never pause viaConsult);- The resolved archetype’s
DefaultInteractionMode; 'Autonomous'.
VideoStudioAgent never writes a derived Interaction.Mode into the spec — it retains Interaction.Mode only when supplied by the caller and otherwise deletes Interaction. When an approval pause is required, VideoStudioAgent emits a 'Plan' step (MJ: AI Agent Requests; see Agent Skills & Plan Mode Guide) at G2 before Pre-Production (DescribeStoryboardForApproval / ReadApprovalDecision) and, under 'ApproveStoryboardAndRoughCut', at G4 before delivery.
The Video Production Quality rubric
Section titled “The Video Production Quality rubric”VideoStudioCriticAgent (packages/AI/VideoStudio/Engine/src/agents/critic-agent.ts) always sends contact sheets (PickSheetsForCall in review/critic-input.ts: up to 4 grids plus the transition strip per deliverable format), runs one Video Studio - Critic prompt call per deliverable format (MAX_DEFECTS_PER_FORMAT = 3), keeps the worst level per criterion across formats, scores the nine criteria on the seeded Video Production Quality rubric (ID: 9860024F-9B41-4A44-B0DA-39066BF0FEE2 in metadata/rubrics/.video-production-quality.json, PassThreshold: 0.7, scale "Meets / Partial / Miss") via RubricScoring.Compute (see the Rubrics Guide), and writes Review.Rounds (Review.Rounds[].Defects) and Review.Scores (Review.Scores.OverallScore).
The rubric evaluates nine criteria:
hook-lands-in-the-first-two-seconds(Weight: 1)readable-at-phone-size(Weight: 1)type-and-color-stay-consistent(Weight: 1)motion-has-weight-and-a-clear-focus(Weight: 1)sound-is-in-sync-with-the-picture(Weight: 1)stays-on-brand(Weight: 1)shown-values-match-the-data-snapshot(Weight: 1,IsGate: true,GateMinimumScore: 0.6)final-frame-works-as-a-poster(Weight: 1)generated-assets-are-honest-and-disclosed(Weight: 1,IsGate: true,GateMinimumScore: 0.6)
Bounded repair loop and evidence clearing
Section titled “Bounded repair loop and evidence clearing”When VideoStudioCriticAgent reports defects (Review.Rounds[].Defects), VideoStudioAgent coordinates bounded repairs (packages/AI/VideoStudio/Engine/src/agents/video-studio-agent.ts, packages/AI/VideoStudio/Engine/src/gates.ts, and packages/AI/VideoStudio/Engine/src/review/human-review-notes.ts):
- Code/layout repair (
BuildRepairBrief→ Motion Engineer → Render → Critic) and asset regeneration (RequestRegeneration→ Pre-Production → Motion Engineer → Render → Critic): Repair rounds are capped by the active preset’sMaxRepairRounds(0inDraft,2inStandard,3inHigh), with a revision run counting only its own new rounds via_initialRoundCount(and per-asset regeneration capped atMaxRegenerations = 1). - Stale evidence invalidation (
ClearStaleEvidence(stageName, spec)): After a stage sub-agent succeeds (a failed stage clears nothing),ClearStaleEvidence(stageName, spec)clears downstream artifacts and gate evidence so a run cannot reach'Delivered'on stale outputs: when'Video Studio - Director','Video Studio - Art Director', or'Video Studio - Storyboard'succeeds, it clearsG2–G4,Output.Source, and render evidence; when'Video Studio - Pre-Production'or'Video Studio - Motion Engineer'succeeds, it clearsOutput.Videos,Output.ContactSheetFileIDs,Output.PosterFileID,Output.RenderMetadata, andG3/G4. - Unrecoverable gate/integrity failure vs. deferred review notes (
FindIntegrityOrGateCriterionFailure): Before theG4pause,FindIntegrityOrGateCriterionFailureends the run with step'Failed'(notStatus = 'Blocked') when a gate criterion fails (shown-values-match-the-data-snapshotorgenerated-assets-are-honest-and-disclosedbelowGateMinimumScore) and either no repairable defect remains or the round cap (MaxRepairRounds) is reached, or when an open integrity defect remains onceMaxRepairRoundsis reached. If only non-integrity defects remain whenMaxRepairRoundsis reached, remaining defects are marked'Deferred'and summarized inReview.HumanReviewNotesviaComposeHumanReviewNotes.
6. Render drivers and the isolated worker
Section titled “6. Render drivers and the isolated worker”VideoRenderEngine (packages/AI/VideoStudio/Engine/src/render/video-render-engine.ts) resolves a VideoRenderDriverBase implementation via MJ ClassFactory using the videoStudio.render.driver key in mj.config.cjs. Settings are parsed and validated by ReadVideoRenderSettings (packages/AI/VideoStudio/Engine/src/render/render-settings.ts).
mj.config.cjs settings (videoStudio.render)
Section titled “mj.config.cjs settings (videoStudio.render)”module.exports = { videoStudio: { render: { // 'LocalWorker' (default) | 'Remote' | 'Docker' driver: 'LocalWorker',
// Maximum bytes accepted for a rendered MP4 (shared by LocalWorker, Remote, and Docker; default: 536870912 = 512 MiB) maxOutputBytes: 536870912,
// Operation timeouts in milliseconds timeouts: { startupMs: 60000, // LocalWorker startup + /health readiness (default: 60,000 ms) renderMs: 2100000, // Full /render operation timeout (default: 2,100,000 ms = 35 min) requestMs: 600000, // /stills, /contact-sheet, /check, /keyframes timeout (default: 600,000 ms = 10 min) },
// Used when driver === 'LocalWorker' local: { // Optional absolute path to @memberjunction/video-studio-render-worker/dist/main.js workerEntry: undefined, // Optional V8 --max-old-space-size=<N> limit in MiB for the child process maxOldSpaceMb: undefined, },
// Used when driver === 'Remote' remote: { // Single worker base URL (falls back to process.env.VIDEO_STUDIO_RENDER_URL) url: undefined, // Optional round-robin pool of worker base URLs; when both urls and url are set, urls wins urls: undefined, // Shared bearer token sent as Authorization: Bearer <token> // (falls back to process.env.VIDEO_STUDIO_RENDER_TOKEN) token: undefined, },
// Used when driver === 'Docker' docker: { image: 'mj-video-render-worker:local', // Default image tag workspaceRoot: undefined, // Optional host directory for per-operation mounts cpus: 2, // --cpus (default: 2) memoryMb: 4096, // --memory in MiB (default: 4096) pidsLimit: 512, // --pids-limit (default: 512) shmSizeMb: 1024, // --shm-size in MiB for Chromium /dev/shm (default: 1024) }, }, },};| Setting key | Type | Default | Source constant / reader (render-settings.ts) |
|---|---|---|---|
videoStudio.render.driver | 'LocalWorker' | 'Remote' | 'Docker' | 'LocalWorker' | DEFAULT_VIDEO_RENDER_DRIVER |
videoStudio.render.maxOutputBytes | number | 536870912 (512 MiB) | DEFAULT_RENDER_MAX_OUTPUT_BYTES |
videoStudio.render.timeouts.startupMs | number | 60000 (60 s) | DEFAULT_RENDER_STARTUP_TIMEOUT_MS |
videoStudio.render.timeouts.renderMs | number | 2100000 (35 min) | DEFAULT_RENDER_TIMEOUT_MS |
videoStudio.render.timeouts.requestMs | number | 600000 (10 min) | DEFAULT_RENDER_REQUEST_TIMEOUT_MS |
videoStudio.render.local.workerEntry | string | undefined | undefined (auto-resolved) | LocalWorkerRenderDriver.resolveWorkerEntryPath (local-worker-render-driver.ts) |
videoStudio.render.local.maxOldSpaceMb | number | undefined | undefined | LocalWorkerRenderDriver.spawnAndWaitForReady (local-worker-render-driver.ts) |
videoStudio.render.remote.url | string | undefined | process.env.VIDEO_STUDIO_RENDER_URL | ReadVideoRenderSettings |
videoStudio.render.remote.urls | string[] | undefined | undefined (urls wins over url) | ReadVideoRenderSettings / RemoteRenderDriver (remote-render-driver.ts) |
videoStudio.render.remote.token | string | undefined | process.env.VIDEO_STUDIO_RENDER_TOKEN | ReadVideoRenderSettings |
videoStudio.render.docker.image | string | 'mj-video-render-worker:local' | DEFAULT_DOCKER_RENDER_IMAGE |
videoStudio.render.docker.workspaceRoot | string | undefined | undefined (process-wide 0o700 temp dir) | getOrCreateProcessWorkspaceRoot (docker-render-driver.ts) |
videoStudio.render.docker.cpus | number | 2 | DEFAULT_DOCKER_CPUS |
videoStudio.render.docker.memoryMb | number | 4096 | DEFAULT_DOCKER_MEMORY_MB |
videoStudio.render.docker.pidsLimit | number | 512 | DEFAULT_DOCKER_PIDS_LIMIT |
videoStudio.render.docker.shmSizeMb | number | 1024 | DEFAULT_DOCKER_SHM_SIZE_MB |
Driver comparison
Section titled “Driver comparison”| Driver | ClassFactory key & class | Isolation & lifecycle | When to use |
|---|---|---|---|
| LocalWorker | 'LocalWorker' (LocalWorkerRenderDriver in local-worker-render-driver.ts) | Spawns a local Node child process bound to 127.0.0.1:0 with a random 32-byte hex bearer token (VIDEO_STUDIO_RENDER_TOKEN) and a 12-key environment allowlist (ALLOWED_WORKER_PARENT_ENV_KEYS). Shares host network and filesystem permissions. | Local development and trusted compositions only. Logs LOCAL_WORKER_NETWORK_WARNING once per process when VideoRenderEngine first selects LocalWorker. |
| Remote | 'Remote' (RemoteRenderDriver in remote-render-driver.ts) | Sends HTTP requests (RenderWorkerHttpClient) to one url or a round-robin urls pool of pre-running mj-video-render-worker servers. | Dedicated render worker fleet, Kubernetes deployments, or Windows hosts (where DockerRenderDriver is unsupported). |
| Docker | 'Docker' (DockerRenderDriver in docker-render-driver.ts) | Provisions a fresh container per operation via DockerSandboxProvider (--network none, --read-only, --cap-drop ALL, --security-opt no-new-privileges), runs node /app/dist/main.js run <op>, and stops/kills the container in finally. | Recommended production boundary for untrusted or LLM-authored compositions on Linux and macOS hosts. |
LocalWorker host prerequisites (ffmpeg, Chromium headless shell, and standalone worker package)
Section titled “LocalWorker host prerequisites (ffmpeg, Chromium headless shell, and standalone worker package)”When videoStudio.render.driver is 'LocalWorker' (the default for dev and single-node installs), the host machine running MJAPI needs:
ffmpegandffprobeonPATH: Verified byGET /health(FfmpegVersion).- Chromium headless shell: HyperFrames renders frames in a Chromium headless shell binary that is downloaded separately from npm packages (or supplied via
CHROME_PATH):Alternatively, setTerminal window # Download the Chromium headless shell into HyperFrames' local cachepnpm --filter @memberjunction/video-studio-render-worker run browser:installCHROME_PATH(orPUPPETEER_EXECUTABLE_PATH/HYPERFRAMES_BROWSER_PATH) to an existing Chromium or Chrome executable. If eitherffmpegor Chromium is missing,LocalWorkerRenderDriver.Health()returns immediately withHealthy: falseand the specific remediation command on the Studio Setup Health card. - Standalone
@memberjunction/video-studio-render-workerpackage boundary (optionalDependencies): Per Plan §7.1–§7.2,@hyperframes/*is isolated exclusively inside@memberjunction/video-studio-render-worker(packages/AI/VideoStudio/RenderWorker) and is declared inoptionalDependenciesof@memberjunction/video-studio.- Monorepo checkouts and default
npm install:npm install @memberjunction/video-studioinstalls@memberjunction/video-studio-render-workerautomatically by default soLocalWorkerRenderDriverresolvesnode_modules/@memberjunction/video-studio-render-worker/dist/main.js(orpackages/AI/VideoStudio/RenderWorker/dist/main.js) out of the box. - Container builds using
DockerorRemote: Pass--omit=optionalduringnpm installto skip pulling@memberjunction/video-studio-render-workerand@hyperframes/*into the MJAPI image (packages/AI/VideoStudio/RenderWorker/Dockerfilebundles the worker,ffmpeg, and Chromium in a dedicated render-worker image).
- Monorepo checkouts and default
DockerRenderDriver execution and the three host rules
Section titled “DockerRenderDriver execution and the three host rules”Build the local worker image from the repository root (DOCKER_RENDER_IMAGE_BUILD_COMMAND in docker-render-driver.ts):
docker build -f packages/AI/VideoStudio/RenderWorker/Dockerfile -t mj-video-render-worker:local .For each operation ('render' | 'stills' | 'contact-sheet' | 'check' | 'keyframes' | 'health' in ONE_SHOT_OPERATIONS), DockerRenderDriver calls DockerSandboxProvider.Provision (packages/AI/AgentHarness/src/sandbox/DockerSandboxProvider.ts), which starts a detached container:
docker run --detach --rm --name mj-harness-<RunId> \ --volume <hostPath>:/workspace --workdir /workspace \ --network none --cpus 2 --memory 4096m --pids-limit 512 --shm-size 1024m \ --read-only --tmpfs /tmp:rw,size=256m \ [--user <uid>:<gid> on Linux] \ --cap-drop ALL --security-opt no-new-privileges \ mj-video-render-worker:local sleep infinityIt then writes <hostPath>/request.json, creates <hostPath>/out, <hostPath>/tmp, and <hostPath>/home, and runs the one-shot CLI inside the container via DockerExecExecutor.Run:
docker exec --interactive \ --env TMPDIR=/workspace/tmp --env HOME=/workspace/home \ mj-harness-<RunId> \ node /app/dist/main.js run <op> --request /workspace/request.json --out /workspace/outOperators must account for three host rules enforced by DockerRenderDriver:
- Host OS rules (Windows refused; Linux
--user <uid>:<gid>; macOS image user):- Windows (
process.platform === 'win32') is refused:DockerRenderDriverthrowsVideoRenderError('WorkerUnavailable')directing operators to use'Remote', because Windows lacksO_NOFOLLOWsymlink-safe host file reads. - Linux (
process.platform === 'linux'):resolveLinuxDockerUser()passes--user ${process.getuid()}:${process.getgid()}andchmod 0o700onhostRoot, so files written inside/workspace/outare owned by the host MJAPI process user rather thannode(UID1000from thenode:24-bookworm-slimbase image, selected byUSER nodeinRenderWorker/Dockerfile). - macOS (
process.platform === 'darwin'):--useris omitted so the container runs asUSER nodewhile Docker Desktop’s file sharing translates ownership back to the host user.
- Windows (
- Workspace root and sibling-container volume mounts (
videoStudio.render.docker.workspaceRoot):- When
workspaceRootis omitted,getOrCreateProcessWorkspaceRootcreates and reuses one process-wide0o700directory viafs.mkdtemp(path.join(os.tmpdir(), 'mj-video-render-')), under whichDockerSandboxProvidercreates per-operation<workspaceRoot>/run/<RunId>folders. - When MJAPI itself runs inside a Docker container and talks to the host’s Docker daemon socket (sibling containers),
--volume <hostPath>:/workspaceis resolved by the host daemon, not inside the MJAPI container’s overlay filesystem. Mount a shared host directory or named volume at the exact same absolute path on both the host and the MJAPI container and setvideoStudio.render.docker.workspaceRootto that path.
- When
- Safe output move,
O_NOFOLLOWreads, andFinalizeteardown:- Before reading any container output,
moveAndVerifySafeOutDircreates a0o700<hostRoot>.outdirectory (safeOutRoot), renames/workspace/outto<hostRoot>.out/out(safeOutDir), and verifies viafs.lstatthat it is a real directory (!outStat.isSymbolicLink() && outStat.isDirectory()). - Every output file (
<out>/result.json,<out>/video.mp4,<out>/error.json) is read viareadRegularFileNoFollow, which opens the descriptor withNOFOLLOW_OPEN_FLAGS(O_RDONLY | O_NOFOLLOW | O_NONBLOCK) and checkshandle.stat()(isFile(),size <= maxBytes, andexpectedBytesfor<out>/video.mp4). - On timeout or caller
AbortSignal, the driver kills thedocker execchild process (proc.Kill()). Infinally,provider.Finalize(handle, outcome)runsdocker stop --time 5 mj-harness-<RunId>(falling back todocker kill mj-harness-<RunId>) and removeshostRoot. The driver removes<hostRoot>.outonly whenprovider.IsContainerRunning(handle)(docker container inspect --format '{{.State.Running}}' mj-harness-<RunId>) returnsfalse, preventing cleanup races against a still-running container.
- Before reading any container output,
RemoteRenderDriver transport security, routes, and size caps
Section titled “RemoteRenderDriver transport security, routes, and size caps”RemoteRenderDriver (packages/AI/VideoStudio/Engine/src/render/remote-render-driver.ts) and RenderWorkerHttpClient (packages/AI/VideoStudio/Engine/src/render/worker-http-client.ts) communicate over the routes defined in RENDER_WORKER_ROUTES (packages/AI/VideoStudio/Core/src/render-contract.ts):
GET /health(RENDER_WORKER_ROUTES.Health) — Readiness probe returningRenderHealth({ Healthy, NodeVersion, HyperFramesVersion, ChromiumVersion?, FfmpegVersion?, Errors }).POST /render(RENDER_WORKER_ROUTES.Render) — Streamsapplication/x-ndjson(RenderStreamEventlines:ProgressandHeartbeat, followed by a terminalResultorError).GET /videos/<videoId>(RENDER_WORKER_ROUTES.Video='/videos') — Streams the renderedvideo/mp4binary once, then deletes it on the worker (or expires it after 10 minutes viaDEFAULT_VIDEO_TTL_MS).POST /stills,POST /contact-sheet,POST /check,POST /keyframes— JSON request/response endpoints.
Transport and authentication rules:
- Client side (
RemoteRenderDriver): Acceptshttp://andhttps://URLs. When a worker URL uses plainhttp://to a non-loopback host, theRemoteRenderDriverconstructor logs a one-time warning per distinct worker URL viaLogWarning; TLS is not enforced by the client.tokenis optional on the client and, when set, is sent asAuthorization: Bearer <token>. - Server side (
CreateRenderWorkerServerinpackages/AI/VideoStudio/RenderWorker/src/server.ts): Refuses at startup to bind a non-loopbackVIDEO_STUDIO_RENDER_HOST(such as0.0.0.0in Docker) unlessVIDEO_STUDIO_RENDER_TOKENis set to a non-empty secret, and requiresAuthorization: Bearer <token>(compared withcrypto.timingSafeEqual) on every route wheneverVIDEO_STUDIO_RENDER_TOKENis set.
Output size caps (enforced by both RenderWorkerHttpClient and DockerRenderDriver; response Content-Length is never trusted):
| Stream / file | Cap | Source |
|---|---|---|
/health, /check, and error JSON bodies (<out>/error.json in Docker), plus each HTTP /render NDJSON line (Docker reads Progress lines from docker exec stdout) | 1 MiB (1,048,576 bytes) | MAX_RENDER_JSON_RESPONSE_BYTES and HTTP-only MAX_RENDER_NDJSON_LINE_BYTES (worker-http-client.ts) |
Rendered MP4 (GET /videos/<videoId> and <out>/video.mp4) | videoStudio.render.maxOutputBytes (default 536,870,912 bytes / 512 MiB) + exact match against VideoSizeBytes on the Result event’s RenderResultMetadata (<out>/result.json in Docker) | DEFAULT_RENDER_MAX_OUTPUT_BYTES (render-settings.ts) |
Base64 image-carrying JSON responses (/stills, /contact-sheet, /keyframes, and <out>/result.json for image ops in Docker) | Math.min(maxOutputBytes, buffer.constants.MAX_STRING_LENGTH) | RenderWorkerHttpClient.httpPostJson (worker-http-client.ts) and DockerRenderDriver.executeContainerOperation (docker-render-driver.ts) |
Security boundary and isolated worker defenses
Section titled “Security boundary and isolated worker defenses”[!IMPORTANT] Static checks are not the data-leak boundary —
DockerRenderDriver(--network none) is. Compositions require inline scripts (script-src 'self' 'unsafe-inline'), soRunLintGate(packages/AI/VideoStudio/RenderWorker/src/lint-gate.ts) cannot prevent obfuscated inline JS (self["loc"+"ation"]) or dynamically constructed<script>elements at runtime. In addition,@hyperframes/producerandhyperframes checkmake Node-side outboundfetch()requests (for example tohttps://fonts.googleapis.comandhttps://cdnjs.cloudflare.comfor GSAP) outside Chromium, and@hyperframes/enginelauncheschrome-headless-shellwith--no-sandbox. Always useDockerRenderDriver(--network none) or an egress-firewalledRemoteworker for untrusted or LLM-authored compositions; useLocalWorkerRenderDriveronly for trusted local development.
For the full defense-in-depth walkthrough, see packages/AI/VideoStudio/RenderWorker/README.md. Key worker facts:
- Parent environment scrubbing (
BuildWorkerEnvinlocal-worker-render-driver.ts): Copies only the 12-keyALLOWED_WORKER_PARENT_ENV_KEYSallowlist (PATH,HOME,TMPDIR,TMP,TEMP,LANG,LC_ALL,SystemRoot,CHROME_PATH,PUPPETEER_EXECUTABLE_PATH,PUPPETEER_CACHE_DIR,HYPERFRAMES_BROWSER_PATH) plusVIDEO_STUDIO_RENDER_HOST=127.0.0.1,VIDEO_STUDIO_RENDER_PORT=0, andVIDEO_STUDIO_RENDER_TOKEN=<token>. BecauseVIDEO_STUDIO_RENDER_ALLOW_ROOTis not in the allowlist,LocalWorkerRenderDrivercannot spawn a worker when the parent process runs asuid === 0(AssertNonRootUserinRenderWorker/src/main.tsrefusesuid === 0unlessVIDEO_STUDIO_RENDER_ALLOW_ROOT=1is present in the worker environment). - Bundle staging (
StageRenderBundleinRenderWorker/src/bundle.ts): Stages files into a directory created withfs.mkdtemp(path.join(parentDir, 'mj-video-bundle-')).ValidateBundleRelativePathrejects empty paths, null bytes, backslashes, leading/, Windows drive prefixes, and./..segments before writing files inside the staged root directory withfs.writeFile. - Static URL scheme gate (
MJ_NON_BUNDLE_URLinRenderWorker/src/lint-gate.ts): Rejects every URL scheme in HTML attributes and CSSurl(...)exceptdata:andblob:. - Injected preamble and CSP (
InjectPreambleIntoHtmlinRenderWorker/src/preamble.ts): Injects<meta http-equiv="Content-Security-Policy" content="...">immediately after<!doctype ...>(or at byte 0) withCOMPOSITION_CSP:default-src 'self' data: blob:; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; connect-src 'none'; frame-src 'none'; worker-src 'none'; object-src 'none'; base-uri 'none'; form-action 'none' - HyperFrames CLI flags (
RenderWorker/src/hyperframes-backend.ts): Invokeshyperframes check --json --no-proxy --no-browser-gpu <bundleDir>. - Decision D15 (
onPageCreatedhook /pnpm.patchedDependenciesstatus): Plan §7.3 / Decision D15 notes an optional upstream@hyperframes/engineonPageCreatedhook (with apnpm.patchedDependenciesfallback if needed) to attach a CDPFetch.enablerequest interceptor insideLocalWorker, while explicitly requiring that security not depend on it. Because@hyperframes/producerandhyperframes checkalso make Node-side outboundfetch()calls outside Chromium (for Google Fonts and GSAP CDN checks),DockerRenderDriver(--network none) is the authoritative network-isolation boundary (Driver.IsNetworkIsolated = true), and nopnpmpatch is applied to@hyperframes/engine(patches/contains onlytype-graphql). - Runtime and container image (
RenderWorker/package.jsonandRenderWorker/Dockerfile): Requires Node>=22(engines.nodeinRenderWorker/package.json); the Dockerfile setsDO_NOT_TRACK=1,HYPERFRAMES_NO_TELEMETRY=1,VIDEO_STUDIO_RENDER_HOST=0.0.0.0,VIDEO_STUDIO_RENDER_PORT=8080, runs asUSER node(UID1000from thenode:24-bookworm-slimbase image),EXPOSE 8080, and usesCMD ["node", "/app/dist/main.js"].
7. File storage setup
Section titled “7. File storage setup”Video Studio and all seven stage sub-agents inherit their target MJ: File Storage Accounts row from the Content Creation row in MJ: AI Agent Categories via DefaultStorageAccountID (BaseAgent.ResolvedStorageAccountId). During Pre-Production and Render, StoreFile, ReadStoredFile, and ReadStoredFileWithType (packages/AI/VideoStudio/Engine/src/storage.ts) upload and download narration audio, WebVTT captions, generated stills/clips, poster images, contact sheets, and rendered MP4s through FileStorageEngine.Instance, recording MJ: Files rows (Status = 'Uploaded').
Inline fallback (videoStudio.outputs.inlineMaxBytes)
Section titled “Inline fallback (videoStudio.outputs.inlineMaxBytes)”ReadVideoStudioOutputSettings (packages/AI/VideoStudio/Engine/src/render/render-settings.ts) reads videoStudio.outputs.inlineMaxBytes (or videoStudio.render.inlineMaxBytes) from mj.config.cjs, defaulting to VIDEO_STUDIO_INLINE_MAX_BYTES = 2 * 1024 * 1024 (2 MiB, Plan §11.2 / §14):
- End-to-end no-storage fallback (
storage.ts,pre-production-agent.ts,render-agent.ts): When noMJ: File Storage Accountsrow resolves (StoreFilereturnsundefined), intra-run intermediate files — synthesized narration clips (AudioFileID), WebVTT captions (CaptionsFileID), the poster frame (PosterFileID), and contact sheets (ContactSheetFileIDs) — are stored in a bounded in-process ephemeral store underinline:<uuid>identifiers (StoreEphemeralFile/IsEphemeralFileID) soPre-Production,Render, andCriticcomplete without a cloud storage bucket. Paid image/video generation (GenerateRequestedAssets) still refuses withNoStoragewhen no storage account is configured. - Rendered MP4 inline fallback (
Output.Videos[fmt].InlineBase64): WhenStoreFilereturnsundefinedfor a rendered MP4 whose byte length is<= inlineMaxBytes(2 MiBby default),VideoStudioRenderAgentstores the MP4 inline onOutput.Videos[fmt].InlineBase64. During the rootVideoStudioAgentloop,Output.Videos[fmt].InlineBase64is replaced in_CURRENT_PAYLOADwithINLINE_VIDEO_BASE64_SENTINEL('[inline-mp4]') so base64 video bytes are never sent to LLM prompts, and the full base64 string is restored when the run exits (Delivered,G4Plan, orFailed). - Visible delivery and setup warnings: When no storage account resolves (
resolution.Source === 'None'),VideoStudio.SetupHealth(VideoStudioSetupHealthServerOperationinsetup-health.operation.ts) reportsStatus: 'Warning'(leavingHealthy: true) with message"No file storage connected. Videos up to 2 MB are kept in the database; larger videos and sharing need file storage."and admin action"An admin can add a file storage account; any active account works."Whenever any deliverable video usesInlineBase64,ComposeHumanReviewNotesadds a### Storagewarning ("16:9: 0.8 MB — Stored inline because no file storage account was configured...") toReview.HumanReviewNotesand<mj-video-production-viewer>renders a warning banner ("Stored inline: no file storage account was set up."). - Exceeding
inlineMaxBytes(or settinginlineMaxBytes: 0): IfStoreFilereturnsundefinedand the rendered MP4 exceedsinlineMaxBytes,VideoStudioRenderAgentfails the run with:The rendered <fmt> video is <sizeMb> MB and could not be stored in MJStorage. Inline storage is limited to <capMb> MB. Set Default Storage Account on the 'Content Creation' agent category or on the Video Studio agent.
8. Data stories
Section titled “8. Data stories”DataBindings and permission-honoring CaptureDataSnapshot
Section titled “DataBindings and permission-honoring CaptureDataSnapshot”A VideoGenerationSpec can bind named datasets in DataBindings across four VideoDataSourceKind values and four VideoDataShape values ('Scalar' | 'Series' | 'Table' | 'Delta'). VideoDataSource (video-generation-spec.ts) defines Kind, QueryID, Parameters, EntityName, ExtraFilter, Fields, and ArtifactVersionID:
Source.Kind | VideoDataSource fields & gate G1 check | How CaptureDataSnapshot (data-snapshot.ts) resolves it |
|---|---|---|
'Query' | QueryID (required by G1), optional Parameters (QueryName is accepted only on ProduceVideoAction’s DataRefs input, which resolves QueryName to QueryID before seeding the spec) | Runs in a single batched RunQuery.RunQueries call as ContextUser (enforcing query permissions and Row-Level Security). |
'View' | EntityName and non-empty Fields (both required by G1), optional ExtraFilter (there is no OrderBy or MaxRows on VideoDataSource) | Runs in a single batched RunView.RunViews call as ContextUser (ResultType: 'simple', always capped at MAX_ROWS_PER_BINDING = 500). |
'Artifact' | ArtifactVersionID (required by G1; there is no Path field) | Loaded in the same batched RunView.RunViews call as 'View' bindings (MJ: Artifact Versions filtered by ID, Fields: ['ID', 'Content', 'ContentMode', 'FileID', 'MimeType']) as ContextUser. |
'Inline' | Binding Key must exist in caller-seeded DataSnapshot.Inline | Read-only from caller-seeded DataSnapshot.Inline (such as Query Builder’s SubAgentInputMapping). A missing 'Inline' key blocks gate G1 (gates.ts), while MissingDataSnapshot is the 'PreProduced' check in ValidateVideoGenerationSpec requiring DataSnapshot.AsOf and inline/file data. |
When the serialized snapshot fits within SNAPSHOT_INLINE_MAX_BYTES = 65536 (64 KiB), CaptureDataSnapshot stores the values directly on DataSnapshot.Inline; larger snapshots are uploaded to MJ: Files and referenced via DataSnapshot.FileID (loaded on demand by LoadDataSnapshotValues). Permission and RLS enforcement on CaptureDataSnapshot and LoadDataSnapshotValues is verified by IT154 (video-data-snapshot checks VDS1–VDS3 in video-data-snapshot.checks.ts).
No invented numbers, MJChartKit, and the Data as of card
Section titled “No invented numbers, MJChartKit, and the Data as of card”- Bundle injection (
BuildRenderDataSnapshot&WithChartKit):- Before staging a composition bundle,
BuildRenderDataSnapshotinjectsAS_OF_DATA_KEY = '__asOf'(window.MJ_DATA.__asOf) alongside every resolved binding key onwindow.MJ_DATA. WithChartKit(packages/AI/VideoStudio/Core/src/chart-kit.ts) injects<bundle>/mj/chart-kit.js(MJ_CHART_KIT_PATH), exposingCreateMJChartKit(window.MJChartKit) with nine deterministic SVG/HTML chart primitives (KpiTile,DeltaBadge,Bar,Line,Area,Donut,RankedList,Sparkline,TableReveal) and number formatters (FormatValue,FormatTableCell).
- Before staging a composition bundle,
- Deterministic DOM data check (
BuildDataCheckInputs&dom-rules.ts):BuildDataCheckInputs(packages/AI/VideoStudio/Core/src/data-literals.ts) computesAllowedDataLiterals,BoundDataLiterals,SampleTimestampsSeconds,SettleTimestampsSeconds, andRequiredFinalFrameText(FormatAsOfCardText(asOf)→'Data as of YYYY-MM-DD').- During
driver.Check, the render worker inspects the live Chromium DOM at beat midpoints, settle timestamps, and the final frame (dom-rules.ts) and emits'Error'codes (MJ_UNBOUND_LITERAL_RENDEREDfor any on-screen numeric literal not traceable toAllowedDataLiterals,MJ_ASOF_CARD_MISSINGwhen a data-bound film’s final frame is missing'Data as of YYYY-MM-DD', etc.), which failVideoStudioRenderAgent’s"Check <fmt>"step before any video is rendered (UnboundNumericOnScreenTextis the static spec check onBeats[].OnScreenTextinValidateVideoGenerationSpec).
- Artifact sharing boundary:
- Once a
Video Productionis rendered, the captured numbers are visible in the MP4 andDataSnapshot. Sharing aVideo Productionartifact viaArtifactShareModalis always an explicit, user-confirmed action, and the sharer is responsible for the audience they grant access to.
- Once a
9. Brand kits and generation policy
Section titled “9. Brand kits and generation policy”Organization brand identity and generative AI guardrails are stored in MJ: Brand Kits (MJBrandKitEntity) and MJ: Brand Kit Assets (MJBrandKitAssetEntity), created by migrations/v6/V202610301713__v6.2.x__Brand_Kits_And_Video_Archetypes.sql.
BrandKitEngine and VideoStudioEngine (@memberjunction/core-entities)
Section titled “BrandKitEngine and VideoStudioEngine (@memberjunction/core-entities)”Both metadata cache engines live in @memberjunction/core-entities:
BrandKitEngine(packages/MJCoreEntities/src/engines/BrandKitEngine.ts) — CachesMJ: Brand KitsandMJ: Brand Kit Assets.GetKitsForCompany(companyId: string | null): ReturnsActivekits matchingcompanyIdplus org-wide (CompanyID == null)Activekits, ordered with company-specific kits before org-wide kits, then alphabetically byName.GetDefaultKit(companyId: string | null): Returns theActivedefault kit (IsDefault === true) forcompanyId, falling back to the org-wide (CompanyID == null)Activedefault kit, ornullwhen none exists.GetAssets(brandKitId: string): Returns cachedMJBrandKitAssetEntityrows forbrandKitId, ordered bySequenceascending, thenName.GetGenerationReferenceAssets(brandKitId: string): Returns cached assets forbrandKitIdwhereIsGenerationReference === true, ordered bySequenceascending, thenName.GetEffectiveGenerationPolicy(kit: MJBrandKitEntity | null): Parseskit.GenerationPolicyObjectinto anEffectiveBrandGenerationPolicyobject ({ AllowGeneratedImages, AllowGeneratedVideo, AllowSyntheticPeople, AvoidSubjects, StylePrompt?, MaxGenerationCostPerRun? }).
VideoStudioEngine(packages/MJCoreEntities/src/engines/VideoStudioEngine.ts) — CachesMJ: Video Archetypes(GetArchetypesForCompany(companyId),GetArchetypeByName(name, companyId)) and ensuresBrandKitEngine.Instanceis configured alongside it.
Default-uniqueness and asset reference invariants
Section titled “Default-uniqueness and asset reference invariants”- One default kit per
CompanyIDscope (MJBrandKitEntityServer.ValidateAsyncinpackages/MJCoreEntitiesServer/src/custom/MJBrandKitEntityServer.server.ts):- When
IsDefaultistrue,ValidateAsyncruns aRunViewoverMJ: Brand Kitsin the sameCompanyIDscope (CompanyID = '<id>'when set, orCompanyID IS NULLfor org-wide kits, regardless ofStatus, excluding the currentIDon update) and adds a validation error onIsDefaultif another default kit already exists in that scope.
- When
- Visual-only generation references (
CK_BrandKitAsset_GenerationReferenceinV202610301713__v6.2.x__Brand_Kits_And_Video_Archetypes.sql):BrandKitAssetenforcesIsGenerationReference = 0 OR AssetType IN (N'Image', N'Screenshot', N'Logo', N'LogoMark', N'Video')so non-visual assets (Font,Music,SoundEffect,Other) cannot be flagged as visual generation references.
GenerationPolicy defaults and fail-closed parsing
Section titled “GenerationPolicy defaults and fail-closed parsing”BrandKitEngine.GetEffectiveGenerationPolicy(kit) applies two safety rules:
- Null kit or blank
GenerationPolicy: Returns a copy ofDEFAULT_BRAND_GENERATION_POLICY:{"AllowGeneratedImages": true,"AllowGeneratedVideo": true,"AllowSyntheticPeople": false,"AvoidSubjects": []} - Malformed JSON or non-object
GenerationPolicy: Logs an error viaLogErrorand fails closed with all three booleans set tofalse:{"AllowGeneratedImages": false,"AllowGeneratedVideo": false,"AllowSyntheticPeople": false,"AvoidSubjects": []}
Shipped sample brand kit (metadata/brand-kits/.brand-kits.json)
Section titled “Shipped sample brand kit (metadata/brand-kits/.brand-kits.json)”MemberJunction ships one sample brand kit in metadata/brand-kits/.brand-kits.json:
Name:"MemberJunction"(ID: "DBBE8BC2-8FEF-4DA9-B24A-C6038129B696",Status: "Active",IsDefault: false,CompanyID: null).GenerationPolicy:{"AllowGeneratedImages": true,"AllowGeneratedVideo": true,"AllowSyntheticPeople": false,"StylePrompt": "flat illustration in the brand palette, no text, no logos","AvoidSubjects": ["people", "faces", "logos", "on-screen text", "product screens"]}NeverInvent:["product screens", "the MemberJunction logo", "metrics or statistics", "customer or member names", "quotes or testimonials"].
Angular Brand Kit form panels
Section titled “Angular Brand Kit form panels”The MJ: Brand Kits entity form is extended with two after-fields form panels registered via @RegisterClassEx(BaseFormPanel, { key, metadata: { entity: 'MJ: Brand Kits', slot: 'after-fields', sortKey } }) in packages/Angular/Explorer/core-entity-forms/src/lib/panels/brand-kits/ that wrap the reusable editors from @memberjunction/ng-brand-kits (packages/Angular/Generic/brand-kits):
| Panel component | RegisterClassEx key | sortKey | Wrapped editor selector |
|---|---|---|---|
BrandKitAssetsPanelComponent (brand-kit-assets.panel.ts) | 'MJBrandKitAssetsPanel' | 20 | <mj-brand-kit-assets-editor> |
BrandKitGenerationPolicyPanelComponent (brand-kit-generation-policy.panel.ts) | 'MJBrandKitGenerationPolicyPanel' | 30 | <mj-brand-kit-generation-policy-editor> |
10. Generated footage
Section titled “10. Generated footage”Generate Video (GenerateVideoAction) vs. Produce Video
Section titled “Generate Video (GenerateVideoAction) vs. Produce Video”Generate Video(@RegisterClass(BaseAction, "Generate Video"),GenerateVideoActioninpackages/Actions/CoreActions/src/custom/ai/generate-video.action.ts, seeded inmetadata/actions/.generate-video.json) is the single-clip primitive. It invokesAIVideoRunner.RunGenerateVideo(packages/AI/Prompts/src/video/AIVideoRunner.ts) across'TextToVideo' | 'ImageToVideo' | 'Continuation' | 'VideoToVideo'and saves the resulting MP4 toMJ: Files(falling back toVIDEO_GENERATED_INLINEwhen file storage fails after vendor spend so a paid clip is never lost).Produce Video(ProduceVideoAction) orchestrates a full multi-beatVideo Studioproduction where generated clips (Assets.Generate) are composited as scene assets inside code-rendered HyperFrames compositions.
Pre-spend policy screening (ScreenGenerationRequest) and spend backstop
Section titled “Pre-spend policy screening (ScreenGenerationRequest) and spend backstop”During Pre-Production (GenerateRequestedAssets in packages/AI/VideoStudio/Engine/src/generation/generate-assets.ts), every Assets.Generate item is screened before any model call by ScreenGenerationRequest (packages/AI/VideoStudio/Engine/src/generation/generation-screen.ts) against EffectiveBrandGenerationPolicy, the union of brand kit NeverInvent and spec Assets.NeverInvent, power-level caps, and EstimateGenerationCallCost. Any policy violation blocks the item with one of the 8 GENERATION_BLOCK_CODES:
GenerationDisabled(AllowGeneratedImages === false/AllowGeneratedVideo === falseonEffectiveBrandGenerationPolicy)FillsFactRequirement(attempts to satisfy aMustUse/ factual requirement onAssets.Required[]whereMustUse === trueorAllowGenerated !== true)NeverInventMatch(prompt matches a term in the union of brand kitNeverInventand specAssets.NeverInvent)PolicySubjectMatch(prompt matches a term inAvoidSubjects)SyntheticPersonDisallowed(AllowSyntheticPeople === falseand prompt matches person/face terms)ExceedsGeneratedVideoSecondsCap(cumulative generated video seconds exceedMaxGeneratedVideoSeconds:20s /40s /60s; default clip durationDEFAULT_GENERATED_VIDEO_DURATION_SECONDS = 5)SpendCapReached(estimated cumulative spend exceedsMaxGenerationCostPerRunwhen set onEffectiveBrandGenerationPolicy;MaxGenerationCostPerRunis optional and unset on the sample"MemberJunction"brand kit, so by default the only spend cap is the rootVideo Studioagent’sMaxCostPerRun: 50inmetadata/agents/.video-studio-agent.json)NoStorage(no resolved storage account ID)
Before calling a model, EstimateGenerationCallCost prices the request (for a ReferenceKeys video, the <key>-still anchor image and the video clip are priced together before the still is made). If a model has no active price row in MJ: AI Model Costs, GenerateRequestedAssets blocks the item fail-closed with BlockedReason = 'GenerationFailed: ...' (outside the 8 GENERATION_BLOCK_CODES).
Execution, C2PA inspection, provenance, and data boundary
Section titled “Execution, C2PA inspection, provenance, and data boundary”- Prompt composition and model routing:
ComposeGenerationPromptappends" Style: " + StylePromptwhenStylePromptis non-empty onEffectiveBrandGenerationPolicy.- Still images (
Kind === 'Image') run throughAIImageGenerationRunner(FLUX 3 Image). - Video clips (
Kind === 'Video') run throughAIVideoRunner.RunGenerateVideousing the carrier promptDefault Generative Video(AIVideoRunner.GENERATIVE_PROMPT_NAME, seeded inmetadata/prompts/.default-generative-video-prompt.jsonand bound toFLUX 3 Video, polling up toMAX_POLL_ATTEMPTS = 360atDEFAULT_POLL_INTERVAL_MS = 5000). Video generation supports three paths: explicitKeyframes(ImageToVideo),ReferenceKeys(first generates a<key>-stillanchor image viaAIImageGenerationRunner, then animates it viaImageToVideo), orTextToVideo. Per-second vendor cost accounting is verified byIT155(video-generation-costchecksVGC1–VGC2invideo-generation-cost.checks.ts).
- C2PA manifest detection and disclosure:
HasC2PAManifest(packages/AI/Providers/BlackForestLabs/src/c2pa.ts) scans top-level ISO-BMFFuuidboxes for the C2PA JUMBF UUID and stampsC2PA: booleanon the generated asset.BuildGenerationProvenanceSummaryandDescribeGeneratedAssets(packages/AI/VideoStudio/Core/src/generated-asset-disclosure.ts) populateOutput.GenerationProvenance({ Assets: [{ AssetKey, Kind, ModelID, PromptRunID, DurationSeconds?, Cost?, C2PA }], TotalVideoSeconds, TotalCost }; the storedMJ: Filesrow’sDescriptionalso recordsjob=<VendorJobID>) and format the human-readable disclosure rendered in the artifact viewer (honesty and disclosure are enforced by thegenerated-assets-are-honest-and-disclosedgate criterion viaG5andFindIntegrityOrGateCriterionFailure).
- Strict vendor reference and data boundary (
generate-assets.ts):- Only the composed visual prompt string and approved non-generated
Imageassets named inReferenceKeys(Assets.Approvedentries withProvenance: 'BrandKit'whereIsGenerationReference === true,'Supplied', or'Captured', plus generated<key>-stillanchor frames) are transmitted to the media vendor. UnflaggedBrandKitassets (IsGenerationReference === false),Generatedassets,DataSnapshotrows, and SQL queries are refused and never sent to external image/video providers.
- Only the composed visual prompt string and approved non-generated
11. Invoking Video Studio
Section titled “11. Invoking Video Studio”The root Video Studio agent (ID: 00151C2D-3284-4F43-973E-947A7222DC24, TypeID: Loop, DriverClass: VideoStudioAgent, InvocationMode: Any, ExposeAsAction: true, ChatHandlingOption: Consult in metadata/agents/.video-studio-agent.json) orchestrates seven direct child Loop sub-agents (ParentID = 00151C2D-3284-4F43-973E-947A7222DC24, all with InvocationMode: 'Sub-Agent'):
Video Studio - Director(ExecutionOrder: 1)Video Studio - Art Director(ExecutionOrder: 2)Video Studio - Storyboard(ExecutionOrder: 3)Video Studio - Pre-Production(ExecutionOrder: 4)Video Studio - Motion Engineer(ExecutionOrder: 5)Video Studio - Render(ExecutionOrder: 6)Video Studio - Critic(ExecutionOrder: 7)
1. Sage peer-to-peer hand-off (nextStep.type = 'Handoff')
Section titled “1. Sage peer-to-peer hand-off (nextStep.type = 'Handoff')”When a user asks Sage in chat to make a video, product film, social clip, or animated KPI briefing, Sage discovers and transfers the conversation to Video Studio (for the full client/server hand-off contract, see Handing a conversation to another agent):
- Semantic discovery (
AI Agents Search+IsDirectlyDiscoverable):- The
AI Agents SearchEntityDocument (ID: DA9AD383-A4DC-4C0C-93A8-EAC99C75054Binmetadata/entity-documents/.entity-documents.json) isStatus: 'Active', indexingMJ: AI AgentsforFindCandidateAgentsAction(packages/Actions/CoreActions/src/custom/ai/find-candidate-agents.action.ts) andFindBestAgentAction(packages/Actions/CoreActions/src/custom/ai/find-best-agent.action.ts). - When
ExcludeSubAgentsistrue(its default),FindCandidateAgentsActionoverridesapplyInvocationFilterwithAIAgentPermissionHelper.IsDirectlyDiscoverable(agent)(packages/AI/BaseAIEngine/src/AIAgentPermissionHelper.ts, which requiresagent.InvocationMode !== 'Sub-Agent' && !agent.ParentID), whileFindBestAgentActioninherits the baseBaseFindAgentsActionfilter (agent.InvocationMode !== 'Sub-Agent') andResolveHandoffTarget(packages/AI/Agents/src/handoff-target.ts) enforcesAIAgentPermissionHelper.IsDirectlyDiscoverable(agent). Only the rootVideo Studioagent passes; all seven stage children are hidden from direct discovery and refused withNotDiscoverableif targeted.
- The
- Top-level transfer (
AgentRunner.FollowHandoffs):- Sage (
metadata/prompts/templates/sage/sage.template.md) emitsnextStep.type = 'Handoff'withhandoff: { agentName: "Video Studio", brief, reason }andtaskComplete: false. BaseAgentcreates theHandoffstep on the source run withTargetIDandHandoffToAgentID, andAgentRunner.FollowHandoffs(packages/AI/Agents/src/AgentRunner.ts) creates a newMJ: Conversation Detailsrow forVideo Studio(ParentID= source detail row,Role = 'AI'), recordsTargetConversationDetailIDon theHandoffstep, and executesVideo Studioas a root run (ParentRunID IS NULL) on the sameConversationID. Subsequent user turns in that conversation route directly toVideo Studio.
- Sage (
- Target preset resolution (
ResolveConversationAgentPreset):- Handoff drops the source agent’s
ConfigurationID.ResolveConversationAgentPreset(packages/AI/BaseAIEngine/src/agent-preset-resolution.ts) resolvesVideo Studio’s preset by checking@mentionpins on the active branch path, then the user’s savedMJ: User Settingspreference atAgentModeSettingKey('00151C2D-3284-4F43-973E-947A7222DC24')(mj.agentMode.00151c2d-3284-4f43-973e-947a7222dc24), and falling back to its default presetStandard(AIConfigurationID: 2144CA24-5FD1-4C07-A4F5-FF0685DE32A1). - This end-to-end flow is verified without live LLM calls by
IT150(video-studio-handoffcheck bundleVSH1–VSH3inpackages/TestingFramework/integration-test-suite/src/checks/video-studio-handoff.checks.ts:VSH1direct discoverability and real top-level transfer,VSH2target preset resolution, andVSH3the continuityConversationDetailrow).
- Handoff drops the source agent’s
2. Direct @Video Studio mention and preset selection
Section titled “2. Direct @Video Studio mention and preset selection”Users can invoke @Video Studio directly from the conversation composer and pick one of the three MJ: AI Agent Configurations presets seeded in metadata/agents/.video-studio-agent.json (backed by metadata/ai-configurations/.video-studio-configurations.json):
| Preset Name | MJ: AI Agent Configurations.ID | AIConfigurationID (MJ: AI Configurations.ID) | IsDefault |
|---|---|---|---|
Draft | 79257776-F45E-44B1-9E50-4F1E399E7B48 | 6791A2B6-B58F-4E40-8EEB-5A2F624DBD45 (Video Studio Draft) | false |
Standard | CB675CDA-0C50-43B8-A709-2782C597EF36 | 2144CA24-5FD1-4C07-A4F5-FF0685DE32A1 (Video Studio Standard) | true |
High | 2EA8CBC6-17AA-4819-849B-97E8793D4AEB | 5F7A39BE-9726-429B-AD55-67B8709485CF (Video Studio High) | false |
3. Specialist agent consultation (Query Builder and Research Report Writer)
Section titled “3. Specialist agent consultation (Query Builder and Research Report Writer)”metadata/agents/.video-studio-agent.json defines two MJ: AI Agent Relationships rows where specialist agents invoke Video Studio as a related sub-agent with ChatHandlingOption: "Consult" and ConfigurationPresetID set to Standard:
| Parent Agent | Relationship ID | SubAgentInputMapping | SubAgentContextPaths | SubAgentOutputMapping |
|---|---|---|---|---|
Query Builder | 8833E6C6-17B4-468A-BC74-70C145FF4BB9 | { "rows": "DataSnapshot.Inline.caller.rows", "columns": "DataSnapshot.Inline.caller.columns" } | ["title", "interpretation", "savedQueryId", "savedQueryName", "metadata.sql"] | { "Status": "video.Status", "Film.OneSentence": "video.OneSentence", "Output.Videos": "video.Videos", "Output.PosterFileID": "video.PosterFileID", "Review.HumanReviewNotes": "video.HumanReviewNotes" } |
Research Report Writer | 39DD52B1-9707-473B-AC7D-8EEBA1F3F37A | (none) | ["synthesis", "report", "sources"] | { "Status": "video.Status", "Film.OneSentence": "video.OneSentence", "Output.Videos": "video.Videos", "Output.PosterFileID": "video.PosterFileID", "Review.HumanReviewNotes": "video.HumanReviewNotes" } |
When Query Builder calls Video Studio, SubAgentInputMapping populates DataSnapshot.Inline.caller.{rows,columns} on the seeded VideoGenerationSpec, and Video Studio - Director (metadata/prompts/templates/video-studio/director.template.md) either binds Source.Kind: 'Inline' with key 'caller' or prefers Source.Kind: 'Query' with savedQueryId when present in SubAgentContextPaths.
4. The Produce Video action (ProduceVideoAction) and MCP tools
Section titled “4. The Produce Video action (ProduceVideoAction) and MCP tools”Workflows (Flow agents), task graphs, MCP clients, and server code can invoke Video Studio through the Produce Video action (@RegisterClass(BaseAction, 'Produce Video'), PRODUCE_VIDEO_DRIVER_CLASS = 'Produce Video' in packages/AI/VideoStudio/Engine/src/actions/produce-video.action.ts and packages/AI/VideoStudio/Engine/src/actions/produce-video.mapping.ts, seeded in metadata/actions/video-studio/.video-studio-actions.json). When exposed through @memberjunction/ai-mcp-server (packages/AI/MCPServer/src/Server.ts), MCP tool names are built dynamically at runtime: "Execute_Produce_Video_Action" (Execute_<Action_Name>_Action in Server.ts:1769) for the action and "Execute_Video_Studio_Agent" (Execute_<Agent_Name>_Agent in Server.ts:1859) or "Run_Agent" (:2777) for direct agent execution.
Input parameters (ProduceVideoInput)
Section titled “Input parameters (ProduceVideoInput)”| Parameter | Required | Type | Behavior (BuildProduceVideoRequest in produce-video.mapping.ts) |
|---|---|---|---|
Topic | Yes | string | Non-empty topic/brief string; trimmed and passed as the initial user Message (produce-video.mapping.ts:177). assemblePartialSpec always seeds Interaction: { Mode: 'Autonomous' } on the initial payload. |
Archetype | No | string | Archetype Name or ID (UUID), resolved via VideoStudioEngine.Instance in CompanyID scope. When omitted, Archetype is left unset on the seeded spec; missing or inactive archetypes fail with NOT_FOUND, and an archetype scoped to a different CompanyID fails with INVALID_PARAMS. |
BrandKit | No | string | Brand kit Name or ID (UUID), resolved via BrandKitEngine.Instance.GetKitsForCompany / BrandKits in CompanyID scope. When omitted, defaults to BrandKitEngine.Instance.GetDefaultKit(companyId ?? null); missing or inactive kits fail with NOT_FOUND, and a kit scoped to a different CompanyID fails with INVALID_PARAMS. |
Formats | No | VideoFormat[] | string | Array, JSON array string, or comma-separated list of '16:9' | '9:16' | '1:1'. Defaults to the resolved archetype’s DefaultFormats when non-empty, or ['16:9'] when no archetype is specified or its DefaultFormats is empty. |
DurationSeconds | No | number | Positive duration in seconds. When an archetype is set, must fall within MinDurationSeconds..MaxDurationSeconds; when omitted, defaults to archetype.DefaultDurationSeconds if positive, else undefined. |
CompanyID | No | string | Optional company identifier string (trimmed via trimNonEmpty). Scopes archetype and brand kit resolution and is forwarded as companyId on AgentRunner.RunAgent and CompanyID on VideoGenerationSpec. |
PowerLevel | No | 'Draft' | 'Standard' | 'High' | Preset name resolved via AIEngine.Instance.FindActiveAgentConfigurationPreset (or GetDefaultAgentConfigurationPreset when omitted) to pass configurationId (AIConfigurationID) to RunAgent. |
DataRefs | No | ProduceVideoDataRefInput[] | string | Array or JSON array string of { Key, Kind: 'Query' | 'View' | 'Artifact', QueryID?, QueryName?, Parameters?, EntityName?, ExtraFilter?, Fields?, ArtifactVersionID?, Shape: 'Scalar' | 'Series' | 'Table' | 'Delta', Format?: { Kind, Currency?, Decimals?, Prefix?, Suffix? }, Label? } (Key and Shape required). Seeded into DataBindings; Kind: 'Inline' fails with INVALID_PARAMS, and an unknown QueryName fails with NOT_FOUND. |
Result codes and output parameters
Section titled “Result codes and output parameters”ResultCode | Success | When returned | Output parameters appended to Params |
|---|---|---|---|
SUCCESS | true | runResult.success === true. Calls runner.ProcessAgentArtifacts before returning. | AgentRunID, ArtifactID, ArtifactVersionID, Status (spec?.Status), VideoFileIDs (Record<string, string> from spec?.Output?.Videos), PosterFileID (spec?.Output?.PosterFileID), HumanReviewNotes (spec?.Review?.HumanReviewNotes) |
NEEDS_INPUT | false | runResult.agentRun?.FinalStep === 'Chat' | AgentRunID |
RUN_FAILED | false | !runResult.success (and FinalStep !== 'Chat') | AgentRunID |
INVALID_PARAMS | false | Parameter validation failure in BuildProduceVideoRequest (including out-of-scope Archetype or BrandKit) | (none) |
NOT_FOUND | false | 'Video Studio' agent missing, Archetype or BrandKit missing or inactive, or QueryName in DataRefs not found | (none) |
UNEXPECTED_ERROR | false | Unhandled exception during execution | (none) |
5. Scheduled recurring productions (AgentScheduledJobDriver)
Section titled “5. Scheduled recurring productions (AgentScheduledJobDriver)”AgentScheduledJobDriver (packages/Scheduling/engine/src/drivers/AgentScheduledJobDriver.ts) validates ConfigurationID, CompanyID, ConversationID, and OverrideModelID in AgentJobConfiguration (packages/Scheduling/base-types/src/types.ts) with IsValidUUID and forwards them to AgentRunner.RunAgent as configurationId, companyId, conversationId, and override: { modelId }.
Example ScheduledJob.Configuration JSON for a recurring Monthly KPI Briefing (using the Video Studio AgentID and Video Studio Standard ConfigurationID):
{ "AgentID": "00151C2D-3284-4F43-973E-947A7222DC24", "ConfigurationID": "2144CA24-5FD1-4C07-A4F5-FF0685DE32A1", "CompanyID": "33333333-3333-4333-8333-333333333333", "ConversationID": "11111111-1111-4111-8111-111111111111", "InitialMessage": "Produce the monthly KPI briefing video for the most recently closed month using our brand kit.", "StartingPayload": { "Interaction": { "Mode": "Autonomous" }, "Film": { "DurationSeconds": 60, "Formats": ["16:9"] } }}6. Production cycle guard
Section titled “6. Production cycle guard”A Video Studio production cannot recursively start another Video Studio production:
- In-process hierarchy check (
VideoStudioAgent.validateAgentinpackages/AI/VideoStudio/Engine/src/agents/video-studio-agent.ts): Inspectsthis.AgentHierarchy.slice(0, -1)case-insensitively before running. If'Video Studio'is already present in the ancestor chain, it fails the run immediately with:Video Studio is already running higher in this call chain (<chain>). A production cannot start another production. - Task-graph ancestor reconstruction (
TaskGraphAgentRunner.buildParentAgentHierarchyinpackages/MJServer/src/services/TaskGraphAgentRunner.ts): When a task-graph node invokes an agent,TaskGraphAgentRunnerwalksParentRunIDup toMAX_ANCESTOR_HOPS = 25hops viaRunViewto reconstructparentAgentHierarchyand passes it toAgentRunner.RunAgent, ensuringVideoStudioAgent.validateAgentalso catches indirectVideo Studio → task graph → Video Studiocycles.
12. The Video Studio app
Section titled “12. The Video Studio app”Explorer Video Studio application and BaseResourceComponent tabs
Section titled “Explorer Video Studio application and BaseResourceComponent tabs”The Video Studio application (ID: 499C1083-C3EC-42D4-ABC4-EAE6F764BC56 in metadata/applications/.video-studio-application.json) lives in packages/Angular/Explorer/dashboards/src/VideoStudio/ with AgentSettings.DefaultAgentID = "@lookup:MJ: AI Agents.Name=Video Studio" and three top-level DefaultNavItems (Productions, Brand Kits, Archetypes). All three pages mount the shared right-side <vs-studio-slide-in> (vs-studio-slide-in.component.ts) beside <mj-page-layout> inside .vs-resource-shell, backed by the root-provided VSStudioSlideInStateService so open/closed state, ConversationId, and brief selections survive switching tabs. Each page sets SuppressChatOverlay: true in its agent context so the floating <mj-chat-overlay> bubble stays hidden, and registers the four Studio brief client tools (SelectStudioArchetype, SelectStudioBrandKit, SetStudioFormats, SetStudioTopic via BuildStudioBriefClientTools in video-studio-agent-context.ts) alongside its own page tools while the brief is showing (ConversationId === null). VSCopilotPanelComponent projects the Video Studio title, the Setup OK icon button (mjButton Variant="flat" Size="sm", aria-label="Setup OK", title="Setup OK"), and the Close button into <ng-template mjChatSlot="headerActions"> so the slide-in uses a single header row (Setup OK takes no row in <vs-production-brief> unless clicked open or when a check is Warning/Error):
Tab (DriverClass) | Component file | Capabilities & registered client tools |
|---|---|---|
Productions (VideoStudioProductionsResource, default) | video-studio-productions-resource.component.ts, vs-production-brief.component.ts | Searchable gallery of Video Production artifacts with archetype, brand kit, company, status (PRODUCTION_STATUS_FILTERS), and date filters; inline detail drawer (<mj-video-production-viewer>); shared <vs-studio-slide-in> (New production button data-testid="productions-open-copilot-button" and collapsed right-edge strip data-testid="productions-copilot-strip"), which sets [SuppressNewConversationEmptyState]="true" so the standard MJ composer (attachments for reference media, voice, mentions, plan mode) is visible from the first open and projects the collapsible <vs-production-brief> above the composer via mjChatSlot="composerExtra" before the first message is sent (users can either type and send directly in the composer — with the brief’s choices passed via StudioAppContext — or click Start production in the brief to stage ComposeStudioFirstMessage into ComposerDraft, attach references if desired, and press Send; after the first message, the brief disappears); Iterate (opens the same slide-in bound to the production’s own ConversationID with the artifact queued as pending context and enters two-column focus mode IsIterateFocusMode hiding the grid until All productions is clicked in the viewer toolbar; disabled when the production has no ConversationID); and user-confirmed Share (ArtifactShareModal). Client tools: SearchProductions, FilterProductions, ClearProductionFilters, SelectProduction, RefreshProductions, SelectStudioArchetype, SelectStudioBrandKit, SetStudioFormats, SetStudioTopic (StartProduction is human-triggered only). |
Brand Kits (VideoStudioBrandKitsResource) | video-studio-brand-kits-resource.component.ts, brand-kit-creator.component.ts | Gallery of MJ: Brand Kits showing a 16:9 sample frame header (Background + Primary/Secondary/Accent bars + HeadingFont + logo mark when the kit has a Logo asset), color swatches, .mj-badge status and default indicators, asset counts, icon-only Variant="flat" delete action (ConfirmDelete), New brand kit (<mj-brand-kit-creator> in mj-dialog Size="lg" with live 16:9 frame preview, logo upload via CreateBrandKitAssetFromUpload, k-means logo color suggestions via SuggestColorsFromRgbaPixels, font pickers, and collapsible More options; also reachable from the brief’s Brand Kit section, which selects the new kit after save), Open Brand Kit (navigates to the MJ: Brand Kits entity form with BrandKitAssetsPanelComponent and BrandKitGenerationPolicyPanelComponent), and the shared <vs-studio-slide-in> (New production button data-testid="brand-kits-open-copilot-button" and collapsed right-edge strip data-testid="brand-kits-copilot-strip"). Publishes context (SuppressChatOverlay: true) and client tools (SearchBrandKits, FilterBrandKits, ClearBrandKitFilters, OpenBrandKit, RefreshBrandKits, SelectStudioArchetype, SelectStudioBrandKit, SetStudioFormats, SetStudioTopic). |
Archetypes (VideoStudioArchetypesResource) | video-studio-archetypes-resource.component.ts, vs-archetype-editor.component.ts | Single-grid gallery of the eight shipped archetypes (CompanyID === null in metadata/video-archetypes/.video-archetypes.json: Topic Explainer, Product Launch, KPI Briefing, Data Story, Social Short, How-To Tutorial, Announcement, Executive Briefing) and company archetypes with <mj-filter-chip> filters (All, each category with count, Shipped, Yours), <mj-video-beat-strip> timelines, m:ss duration formatting (FormatVideoTime / FormatVideoTimeRange), <mj-video-format-glyph> aspect-ratio indicators, user-driven New archetype, Make a copy, and Edit via <vs-archetype-editor> in <mj-dialog Size="lg"> (live beat strip preview, Fit to length beat scaling, and DeriveBeatKeyFromLabel key generation; also reachable from the brief’s Make your own link), and the shared <vs-studio-slide-in>. Publishes context (SuppressChatOverlay: true) and client tools (SearchArchetypes, FilterArchetypes, ClearArchetypeFilters, SelectArchetype, OpenArchetype, RefreshArchetypes, SelectStudioArchetype, SelectStudioBrandKit, SetStudioFormats, SetStudioTopic). |
VideoProductionViewerComponent and @memberjunction/ng-media-player
Section titled “VideoProductionViewerComponent and @memberjunction/ng-media-player”When a Video Production artifact (ID: 2E23F219-E790-4A77-9C86-CB19C711B71E in metadata/artifact-types/.video-production-artifact-type.json) is opened in Explorer, Conversation Artifacts, or the Productions drawer, VideoProductionViewerComponent (@RegisterClass(BaseArtifactViewerPluginComponent, 'VideoProductionViewerPlugin') in video-production-viewer.component.ts) and VideoProductionArtifactPreviewComponent render:
- Multi-format video player: Uses
<mj-storage-media-player>/<mj-media-player>from@memberjunction/ng-media-player(packages/Angular/Generic/media-player) to streamOutput.Videos[fmt].FileIDwith posterOutput.PosterFileID(or playOutput.Videos[fmt].InlineBase64viaInlineTrackswhen stored inline), plus a toggleablemjButtonaspect-ratio switcher with<mj-video-format-glyph>(16:9,9:16,1:1). - Provenance and disclosure badges: Displays
GetVideoStatusBadgeVariant(spec.Status),FormatVideoTimeduration (m:ss),FormatAsOfCardText(DataSnapshot.AsOf)(Data as of YYYY-MM-DD) when data-bound, andDescribeGeneratedAssets(spec)whenever generated assets were used. - Inspection tabs:
Brief,Storyboard(VideoProductionStoryboardTabComponentwith an interactive<mj-video-beat-strip>above the beat table and contact sheets),Style,Data,Review(VideoProductionReviewTabComponentdisplaying gatesG1–G6, rubric criteria scores, contact sheets, andHumanReviewNotes), and per-formatSource <fmt>tabs.
13. Testing
Section titled “13. Testing”Unit test suites (FakeRenderDriver)
Section titled “Unit test suites (FakeRenderDriver)”Fast deterministic unit tests run without Chromium, ffmpeg, Docker, or live LLM calls (@memberjunction/video-studio uses FakeRenderDriver in fake-render-driver.ts):
pnpm --filter @memberjunction/video-studio-core testpnpm --filter @memberjunction/video-studio testpnpm --filter @memberjunction/video-studio-render-worker testpnpm --filter @memberjunction/ng-brand-kits testpnpm --filter @memberjunction/ng-media-player testpnpm --filter @memberjunction/ng-artifacts testpnpm --filter @memberjunction/ng-dashboards testOpt-in real Chromium + ffmpeg, Docker, and remote render tests
Section titled “Opt-in real Chromium + ffmpeg, Docker, and remote render tests”To run the real HyperFrames + headless Chromium + ffmpeg end-to-end render tests in @memberjunction/video-studio-render-worker and @memberjunction/video-studio:
# Install the Chromium headless shell once per machinepnpm --filter @memberjunction/video-studio-render-worker run browser:install
# Run local Chromium + ffmpeg render tests (LocalWorkerRenderDriver / HyperFramesRenderBackend)DO_NOT_TRACK=1 HYPERFRAMES_NO_TELEMETRY=1 VIDEO_STUDIO_RENDER_TESTS=1 pnpm --filter @memberjunction/video-studio-render-worker testDO_NOT_TRACK=1 HYPERFRAMES_NO_TELEMETRY=1 VIDEO_STUDIO_RENDER_TESTS=1 pnpm --filter @memberjunction/video-studio test
# Run DockerRenderDriver real container tests (requires VIDEO_STUDIO_DOCKER_TESTS=1 and mj-video-render-worker:local; skips when only VIDEO_STUDIO_RENDER_TESTS=1 is set)DO_NOT_TRACK=1 HYPERFRAMES_NO_TELEMETRY=1 VIDEO_STUDIO_DOCKER_TESTS=1 pnpm --filter @memberjunction/video-studio test
# Run RemoteRenderDriver live worker tests against a running render worker URL (add VIDEO_STUDIO_RENDER_TOKEN=<token> when the worker requires one)DO_NOT_TRACK=1 HYPERFRAMES_NO_TELEMETRY=1 VIDEO_STUDIO_RENDER_TESTS=1 VIDEO_STUDIO_RENDER_URL=http://127.0.0.1:8080 pnpm --filter @memberjunction/video-studio testIntegration check bundles (@memberjunction/integration-test-suite)
Section titled “Integration check bundles (@memberjunction/integration-test-suite)”Deterministic integration check bundles live in packages/TestingFramework/integration-test-suite/src/checks/:
IT154(video-data-snapshot, checksVDS1–VDS3invideo-data-snapshot.checks.ts): VerifiesContextUserRow-Level Security and entity-permission enforcement onCaptureDataSnapshotandLoadDataSnapshotValues.IT155(video-generation-cost, checksVGC1–VGC2invideo-generation-cost.checks.ts): VerifiesAIVideoRunner.RunGenerateVideocarrier prompt run creation (Default Generative Video/FLUX 3 Video) and per-second cost calculation.IT150(video-studio-handoff, checksVSH1–VSH3invideo-studio-handoff.checks.ts): VerifiesAIAgentPermissionHelper.IsDirectlyDiscoverable, top-levelAgentRunner.FollowHandoffstransfer toVideo Studio,ResolveConversationAgentPresetpreset resolution, and the continuityConversationDetailrow.IT151(video-studio-production, checksVSP1–VSP5invideo-studio-production.checks.ts): Deterministic end-to-end production pipeline suite using orderedTestLLMreply scripts (fixtures/video-studio/*.json) and the shared fakes invideo-studio-fakes.ts(UseFakeRenderDriverwrappingFakeRenderDriver,RegisterFakeVideoGenerator,RegisterFakeImageGenerator,RegisterFakeTextToSpeech,RegisterFakeAPIKeys, andUseMemoryStorageprovisioning a runtimeMemoryFileStorageDriveraccount on an active testMJ: File Storage Providersrow and resolving it viaResolveAgentStorageAccount):VSP1: DirectSocial Short(Draft) run through Root → Director → Art Director → Storyboard → Pre-Production → Motion Engineer → Render → Critic → Delivered; verifiesMJ: Filesrows forOutput.Videos['16:9'].FileID,Output.PosterFileID, andOutput.ContactSheetFileIDs, zeroInlineBase64, zero MP4 bytes inMJ: AI Agent Run Medias.InlineData, andFakeRenderDriver.RenderCallCount === 1.VSP2: Related-agentIT: VS Data Caller→Video Studio(Social Short) using a runtime copy of the shippedQuery Builder → Video Studiorelationship (Video Studio Standardpreset,DataSnapshot.Inline.caller.rows, andfileOutputspropagation).VSP3:ActionEngineServer.Instance.RunActiononProduce Video(PowerLevel: 'Draft',CredentialScope: 'Any'), assertingArtifactID,ArtifactVersionID,VideoFileIDs['16:9'], andPosterFileID, and verifying every prompt run under the root run usedTestLLM.VSP4: Disclosed 4 s generated clip (Provenance: 'Generated', carrierPromptRunIDwith cost,Output.GenerationProvenance.TotalVideoSeconds === 4, and"AI-generated assets (1)"inReview.HumanReviewNotes).VSP5: Pre-spend policy and budget guards (NeverInventblocking"our CEO speaking"against['real people'], andDraft’s 20 s generated-video cap failing GateG2withGenerationDurationExceedsLimiton a 25 s clip beforePre-Production, any vendor submit, or any render call).
IT152(video-studio-live-smoke, checkVSLS1invideo-studio-live-smoke.checks.ts): Opt-inlive-modeltier smoke test (RequiresLiveModel: true) that runs a real 15 sSocial ShortunderDraftagainstVideoRenderEngine.Instance.Driverand real credentialed models (registering unarmed refusing video and image generator fakes unlessVS_LIVE_GENERATION=1), assertingStatus === 'Delivered', a real MP4FileID, and duration within 1 s of the spec (self-skips loudly when render driverHealth()is unhealthy orBaseModelRunner.PreviewModelSelectionfinds no candidate withHasCredentials: true).IT153(agent-consult, checkAC1inagent-consult.checks.ts): Verifies deterministic sub-agent consultation (ChatHandlingOption = 'Consult'), human escalation viaMJ: AI Agent Requests, and resumed child execution viaresumeRunId.
# Run deterministic IT151 end-to-end production checks (no live LLM, Chromium, or vendor calls)MJ_INTEGRATION_TEST=1 node packages/MJCLI/bin/run.js test run --name "IT151 - Video Studio Production"
# Run opt-in IT152 live smoke test (requires healthy render worker + real model credentials)MJ_INTEGRATION_TEST=1 node packages/MJCLI/bin/run.js test run --name "IT152 - Video Studio Live Smoke"14. Troubleshooting
Section titled “14. Troubleshooting”| Symptom / error code | Source / message | Resolution |
|---|---|---|
Rendered MP4 exceeds inline storage cap (inlineMaxBytes, default 2 MiB) or generation blocks with NoStorage | "The rendered <fmt> video is <sizeMb> MB and could not be stored in MJStorage. Inline storage is limited to <capMb> MB. Set Default Storage Account on the 'Content Creation' agent category or on the Video Studio agent." (render-agent.ts / generation-screen.ts) | Configure an MJ: File Storage Accounts row and set DefaultStorageAccountID on the Content Creation row in MJ: AI Agent Categories (or adjust videoStudio.outputs.inlineMaxBytes in mj.config.cjs for local no-storage runs). |
Review.HumanReviewNotes and viewer warn that videos were stored inline | ComposeHumanReviewNotes and <mj-video-production-viewer> when Output.Videos[fmt].InlineBase64 is used (human-review-notes.ts) | Configure DefaultStorageAccountID on Content Creation so rendered MP4s, poster stills, contact sheets, and narration clips upload to MJ: Files via StoreFile instead of inline / ephemeral fallback. |
| Worker health or render fails with missing Chromium headless shell | "Chromium headless shell binary was not found. Run \pnpm —filter @memberjunction/video-studio-render-worker run browser:install` or set CHROME_PATH.” (hyperframes-backend.ts`) | Run pnpm --filter @memberjunction/video-studio-render-worker run browser:install on the worker host or set CHROME_PATH / PUPPETEER_EXECUTABLE_PATH / HYPERFRAMES_BROWSER_PATH. |
Worker GET /health reports Healthy: false with missing ffmpeg | FfmpegVersion undefined in RenderHealth (hyperframes-backend.ts) | Install system ffmpeg and ffprobe on PATH on the worker host, or switch videoStudio.render.driver to 'Docker' (mj-video-render-worker:local bundles ffmpeg). |
VideoRenderError('WorkerUnavailable') when starting a render | LocalWorkerRenderDriver, RemoteRenderDriver, or DockerRenderDriver | Docker: Build mj-video-render-worker:local (DOCKER_RENDER_IMAGE_BUILD_COMMAND), verify the Docker daemon is running, and on Windows switch videoStudio.render.driver to 'Remote'. LocalWorker: Ensure @memberjunction/video-studio-render-worker is installed and MJAPI is not running as uid === 0 (AssertNonRootUser). Remote: Verify videoStudio.render.remote.url / urls and ensure VIDEO_STUDIO_RENDER_TOKEN is set when the worker binds a non-loopback VIDEO_STUDIO_RENDER_HOST. |
VideoRenderError('Timeout') during /render or /check | Operation exceeded timeouts.renderMs (2,100,000 ms) or timeouts.requestMs (600,000 ms) (render-settings.ts, render-contract.ts) | Increase videoStudio.render.timeouts.renderMs / requestMs in mj.config.cjs, increase Docker container cpus / memoryMb / shmSizeMb, or run with the Draft preset (24 fps, 'First' format). |
Generated asset is Blocked (Status: 'Blocked') or production fails on G5 / FindIntegrityOrGateCriterionFailure (step 'Failed') | ScreenGenerationRequest (GENERATION_BLOCK_CODES in generation-screen.ts), GenerateRequestedAssets (generate-assets.ts), or FindIntegrityOrGateCriterionFailure (gates.ts / video-studio-agent.ts) | Inspect Assets.Generate[i].BlockedReason and Review.Gates: adjust the prompt, Assets.Required[].AllowGenerated, or brand kit GenerationPolicy / NeverInvent (GenerationDisabled, FillsFactRequirement, NeverInventMatch, PolicySubjectMatch, SyntheticPersonDisallowed), raise MaxGenerationCostPerRun (SpendCapReached), ensure active MJ: AI Model Costs rows exist (GenerationFailed), reduce clip count/duration (ExceedsGeneratedVideoSecondsCap), or configure storage (NoStorage). |
15. NOTICE
Section titled “15. NOTICE”Video Studio includes or depends on the following third-party open-source components:
- HyperFrames (
@hyperframes/core,@hyperframes/engine,@hyperframes/lint,@hyperframes/parsers,@hyperframes/producer,@hyperframes/studio-server, andhyperframesCLI, pinned at0.8.134) — Copyright 2026 HeyGen, Inc., licensed under the Apache License, Version 2.0 (https://github.com/heygen-com/hyperframes). - Inter (bundled as
MJ Sansinpackages/AI/VideoStudio/RenderWorker/assets/fonts/) — Copyright 2020 The Inter Project Authors (https://github.com/rsms/inter), licensed under the SIL Open Font License, Version 1.1. - JetBrains Mono (bundled as
MJ Monoinpackages/AI/VideoStudio/RenderWorker/assets/fonts/) — Copyright 2020 The JetBrains Mono Project Authors (https://github.com/JetBrains/JetBrainsMono), licensed under the SIL Open Font License, Version 1.1. - All eight shipped archetype
Guidancemarkdown documents inmetadata/video-archetypes/guidance/*.mdare original MemberJunction works; no HyperFrames skill text was copied or adapted.