# Voice memo to explainer video — agent playbook > Paste this whole file (with the BRIEF filled in) into Claude cowork, chat, or Code. The agent walks through it with you and stops for your approval at every step that matters. ## What you'll get A polished explainer video built from the voice memo you already recorded — the "I had an idea while walking" workflow. The agent transcribes, generates B-roll timed to your phrases, burns captions, and either lipsyncs your face or rides B-roll only. - **Clean transcript** — your voice memo, transcribed with timestamps. - **4-8 B-roll images** — generated to illustrate specific phrases in your transcript. Brand-locked palette. - **Burned-in captions** — kinetic lower-third, palette-matched to your brand. - **Light music bed** — sub-mixed at -15dB under your voice. - **Talking-head version** — your face animated (`kling-v3-i2v`) then lipsynced (`lipsync`) to your voice (only if you supplied `face_url`). - **Final MP4** — chosen aspect ratio, ready for upload. - **Short tease clip** — the strongest 15-20s as a hook for social. **Time**: ~18 minutes wall clock. **Your attention**: about 4 minutes across 5 approvals. **Cost**: $1.20 – $2.80 end to end. ## Tell the agent about the voice memo ```yaml voice_memo_url: # the audio file — upload via /upload first, then paste the URL. m4a / mp3 / wav all fine. topic_summary: # 1 sentence — what is the memo about? e.g. "why founders should write their own job descriptions" · "the 5-minute morning routine that fixed my mornings" brand_palette: # OPTIONAL — 4 hex colors. If blank, the agent picks based on broll_style. aspect_ratio: # 9:16 (default for social) or 16:9 (for YouTube / website embed) broll_style: # "corporate clean" · "warm coaching" · "scrappy authentic" · "minimal editorial" · "high-contrast bold" include_face: # "yes" (and supply face_url below) or "no" (B-roll only) face_url: # OPTIONAL — your portrait photo, if include_face = yes. Used as the talking-head keyframe. your_name: # for the lower-third name tag video_slug: # kebab-case, e.g. founder-job-descriptions ``` ## Quality controls (new — auto-applied) This playbook leverages the closed quality loop. The agent applies these by default; you don't have to ask: - **Cost upfront.** Before any multi-step batch fires, the agent calls `submit_plan` with `confirm:false` and shows you the proposed steps + total cost. Nothing runs until you approve. - **Auto-retry on weak shots.** The server default is `quality_threshold: 0.7`. For brand-locked B-roll the playbook step tells the agent to pass `quality_threshold: 0.85` so a low-confidence first try gets one auto-retry before you see it. - **Critic verdict on the result.** When a multi-asset step finishes, the agent reads the `critique` field on the response (verdict: ship / iterate, plus issues / missing items) and surfaces it for your review. - **One-click recovery.** If a step fails, `get_plan` returns a `recovery` hint pointing at `replan({plan_id})` — the agent classifies the failure and proposes a recovery plan you can approve. - **End-of-project scorecard.** The final wrap-up calls `get_scorecard({project_id})` with succeeded count, mean quality score, critic verdict, total cost, and wall time. You can override any of these per step: pass `quality_gate: false` on cost-sensitive tries, `quality_threshold: 0.6` for stylized work, or `submit_plan({plan_id, confirm:true})` to skip the cost-preview pause when you trust the chain. ## How this works Five gates. The first one is the most consequential — confirming the transcript + B-roll plan. Get those right and the rest assembles cleanly. Get them wrong and you'll be re-rendering B-roll images you don't need. ## The steps ### Step 0 — Confirm the brief The agent reads back the BRIEF and previews the voice memo inline so you can re-listen. **STOP**: "Brief looks right? Voice memo plays? (approve / edit [field] [new value])" ### Step 1 — Transcribe + propose B-roll plan (~2 min, ~$0.20) The agent transcribes `voice_memo_url` with `wizper` (Whisper-v3 → timestamped SRT; use `nemotron-asr` for non-English memos). Then it segments the transcript into phrases and proposes a B-roll plan: 4-8 visuals, each anchored to a specific phrase + timestamp window. ``` 0:00–0:08 "Why founders should write their own JDs..." → b-roll: founder at desk, laptop open, late evening 0:08–0:18 "Because nobody knows what you actually need..." → b-roll: empty chair, posted job sign on it, slightly absurd ... ``` You review and edit the brief per visual before any image gen fires. > **For the agent**: B-roll count should match memo length (~1 visual per 10-15s of audio). Each B-roll brief should describe a SCENE not an ICON — generic stock-photo-feeling images are what kill explainer videos. Use the `broll_style` keyword in every prompt. **STOP**: "Transcript + B-roll plan look right? (approve / edit b-roll [#] [new description])" ### Step 2 — Lock the brand kit (~30s, ~$0.05) Build a brand kit from `broll_style` + `brand_palette` (or inferred palette if blank). Voice keywords inferred from `topic_summary`. Renders 3 reference images, sets as active brand kit so every B-roll inherits the palette. **STOP**: "Approve the brand kit, or refine? (approve / refine [direction])" ### Step 3 — Generate B-roll batch (~5 min, ~$1.00) The agent calls `generate_project` with the B-roll plan from Step 1 — 4-8 images, all brand-locked, rendered in parallel. Composition variety enforced (close-up / wide / detail / metaphorical) so the video doesn't visually repeat. > **For the agent**: pass `quality_threshold: 0.85` per image. Use `flux-dev` for `broll_style: scrappy authentic`, `recraft-v4` for `broll_style: minimal editorial`, `gpt-image` for any B-roll that needs legible text on signs / posters / screens. Surface the critic verdict on the batch. **STOP**: "B-roll grid approved? (approve / redo [#] [direction])" ### Step 4 — Light music bed + lower-thirds (~2 min, ~$0.30) The agent calls `create_media({ action: "music", prompt: , lyrics_prompt: "[Instrumental]", duration: <≥ memo length> })`. Then `hyperframes-lower-third` composes a "your_name + topic_summary" lower-third graphic for the first 5s of the video. > **For the agent**: music gen at -15dB equivalent style brief — "soft, non-distracting, instrumental, palette-mood matched." Don't generate punchy / energetic music for an explainer. **Critical:** pass `lyrics_prompt: "[Instrumental]"` explicitly — without it minimax-music will sing your style prompt verbatim ("soft non-distracting instrumental palette-mood matched" → out loud). See `public/playbooks/music-video-mtv.md` §"The lyrics-vs-style bug". **STOP**: "Music bed approved? (approve / regenerate / skip music)" ### Step 5 — Compose final video (no gate, ~5 min, ~$0.30) Branching based on `include_face`: **If `include_face: yes`** — agent animates `face_url` with `kling-v3-i2v` (subtle motion, `resolution:"auto"`), then `create_media(action:"lipsync", model_override:"lipsync", source_url:, audio_url:)` so you read your own memo. (Lipsync the MOTION clip, never a static still — a frozen face reads as a paper puppet. The `talking-head` BYOC cap has a known param-shape bug; `lipsync` is the reliable path.) Then intercut B-roll over the lipsync video per the Step-1 timestamps via `ffmpeg-overlay` (B-roll covers the face during specific seconds; face returns on pauses / emphasis). **If `include_face: no`** — agent stitches the B-roll images with `ffmpeg-concat` (cross-fade transitions) and Ken Burns pan/zoom via `ffmpeg-overlay` so stills feel cinematic. Both paths then lay audio with `create_media(action:"mux_audio", model_override:"ffmpeg-audio-mix", source_url: