---
reliability: 3.4 # 5 − wizper .2 − brand-kit refs .3 − B-roll gen .3 − music .2 − 8-stage chain .6; trim/burn/overlay/export deterministic; B-roll + zoom-focus + music OPTIONAL (skip all three for a captioned-trim-only run at ~4.3); agent can't watch the recording — trim plan quality rides on the transcript
---

# Screen recording to tutorial — agent playbook

> Paste this whole file (with the BRIEF filled in) into Claude cowork, chat, or Code. The agent walks through it with you and stops for your approval at every step that matters.

## What you'll get

A finished tutorial video built from the raw screen recording you already captured. This is what Loom / Synthesia / Tella charge $10-30/mo for — one playbook, no monthly cost, with B-roll inserts and chapter markers theirs don't include.

- **Trim plan** — the agent proposes where to cut filler / mistakes / dead air. You approve before any rendering.
- **Auto-captioned tutorial** — your narration transcribed and burned in as captions in the lower-third.
- **3-6 B-roll insert frames** — concept illustrations / charts that cut in over the phrases you supplied. Brand-palette locked.
- **Chapter markers + key-term callouts** — lower-third overlays that name the section the viewer's in.
- **Intro + outro frames** — a clean topic-title card at start, a "more at <your handle>" card at end.
- **Light music bed** — sub-mixed under silence beats, ducked under your voice.
- **Zoom-and-focus moments** — short ffmpeg-zoom emphases on the UI moments you mark.
- **Final tutorial MP4** — chapter-marked, captioned, professional.
- **60s tease clip** — a hooky cut for social with the key insight upfront.

**Time**: ~25 minutes wall clock. **Your attention**: about 6 minutes across 7 approvals. **Cost**: $1.20 – $2.80 end to end.

## Tell the agent about the recording

```yaml
recording_url:           # the raw screen capture — upload via /upload first. mp4 / mov / webm all fine.
tutorial_topic:          # 1-2 sentence summary. e.g. "How to set up Webhooks in Stripe for a Next.js app"
target_audience:         # "beginner" · "intermediate" · "advanced" · "internal team"
narration_audio_url:     # OPTIONAL — if you recorded narration separately (cleaner than capture audio), paste URL here
brand_palette:           # 4 hex colors space-separated, e.g. #635BFF #00D924 #1A1A2E #F5F5F5
broll_inserts:           # list of objects — see example below. Each one cuts a generated visual over a specific phrase.
zoom_focus_moments:      # OPTIONAL — list of timestamps where you want a 2x zoom + focus. e.g. ["1:23", "4:47", "9:12"]
your_handle:             # for the outro frame
tutorial_slug:           # kebab-case, e.g. stripe-webhooks-nextjs

# Example broll_inserts format:
# broll_inserts:
#   - { trigger_phrase: "what a webhook actually is", visual_brief: "diagram showing event source → HTTP POST → handler" }
#   - { trigger_phrase: "the security pitfall", visual_brief: "padlock with caution tape, isometric style" }
```

## Quality controls (new — auto-applied)

This playbook leverages the closed quality loop. The agent applies these by default; you don't have to ask:

- **Cost upfront.** Before any multi-step batch fires, the agent calls `submit_plan` with `confirm:false` and shows you the proposed steps + total cost. Nothing runs until you approve.
- **Auto-retry on weak shots.** The server default is `quality_threshold: 0.7`. For brand-locked B-roll the playbook step tells the agent to pass `quality_threshold: 0.85` so a low-confidence first try gets one auto-retry before you see it.
- **Critic verdict on the result.** When a multi-asset step finishes, the agent reads the `critique` field on the response (verdict: ship / iterate, plus issues / missing items) and surfaces it for your review.
- **One-click recovery.** If a step fails, `get_plan` returns a `recovery` hint pointing at `replan({plan_id})` — the agent classifies the failure and proposes a recovery plan you can approve.
- **End-of-project scorecard.** The final wrap-up calls `get_scorecard({project_id})` with succeeded count, mean quality score, critic verdict, total cost, and wall time.

You can override any of these per step: pass `quality_gate: false` on cost-sensitive tries, or `submit_plan({plan_id, confirm:true})` to skip the cost-preview pause when you trust the chain.

## How the agent should run this (interaction contract)

1. **CONFIRM (one message, ≤1 question):** restate in 1 line ("your <M>-min recording → trimmed, captioned, chaptered tutorial + 60s tease, ~$1.20–2.80, ~25 min") + real choices: full kit vs trim+captions only (skips B-roll/music/zooms — the cheap reliable core) · B-roll count (3 vs 6) · music yes/no. One question max.
2. **PREVIEW CHECKPOINT:** the transcript + TRIM PLAN table (~$0.25) — the agent can't watch your screen, so this table is the whole premise; every cut is user-approved before any render. Second cheap gate: B-roll stills shown before they're cut into the master.
3. **NARRATE:** per stage: "transcribing (~3 min)…", "B-roll 3/5 rendering (job …)", "burning captions (~2 min)"; poll `get_create_media` every 10–15s; never silent >2 min.
4. **FAIL GRACEFULLY:** `wizper` fails → `nemotron-asr`; B-roll text/labels garble → `gpt-image` retry → `recraft-v4` diagram-style → drop that insert (the screen capture underneath is always valid — partial beats nothing); `trigger_phrase` not in transcript → fuzzy match → ask, never guess a timestamp; music overpowers VO → direct-SDK `ffmpeg-audio-mix` with `sidechain:true` → drop the bed. ≤2 retries per asset.
5. **DELIVER:** tutorial MP4 + SRT + chapter list + tease + one honest line ("captions re-aligned post-trim — scrub 2 cut points to confirm sync; B-roll #2 dropped after 2 garbled tries") + ONE next step ("upload to YouTube with the SRT as caption track + chapters in the description").

## How this works

Seven gates. The transcript + trim plan in Step 1 is the entire premise — the agent can't watch your screen recording, but with your transcript and confirmation of trim points, it assembles everything else. Get the trim plan right and the rest renders cleanly.

## The steps

### Step 0 — Confirm the brief

The agent reads back the BRIEF and lists `recording_url` metadata (duration, resolution) + each broll_insert with its trigger_phrase and visual_brief.

**STOP**: "Brief looks right? Recording loads? (approve / edit [field] [new value])"

### Step 1 — Transcribe + propose trim plan (~3 min, ~$0.25)

The agent calls `wizper` (Whisper-v3 batch transcription) on `narration_audio_url` if set, otherwise on `recording_url` audio. Returns timestamped SRT. Then it analyzes the transcript for filler patterns ("um", "wait, let me try that again", long silences) and proposes trim points:

```
0:00–0:08    keep (intro)
0:08–0:14    DROP (re-took, said "let me try that")
0:14–2:30    keep
2:30–2:45    DROP (long silence, scrolling)
...
```

You confirm or edit each proposed cut. The agent won't auto-trim — you approve every removal.

> **For the agent**: present the trim plan as an editable table. Total post-trim duration should be ≤ 80% of original (typical filler removal). If trim plan reduces by <10%, ask the user if more aggressive cutting is welcome.

**STOP**: "Trim plan approved? (approve / edit cut [#] / restore [#])"

### Step 2 — Lock brand kit (~30s, ~$0.05)

Build a brand kit from `brand_palette`. Voice keywords inferred from `tutorial_topic` + `target_audience`. Renders 3 reference graphics. Sets as active brand kit so every B-roll + intro/outro frame inherits the palette.

**STOP**: "Approve the brand kit, or refine? (approve / refine [direction])"

### Step 3 — Apply trim + transcription captions (no gate, ~3 min, ~$0.10)

The agent calls `ffmpeg-trim` per the approved trim plan, producing a clean trimmed master. Then `ffmpeg-burn-subtitles` overlays the (re-aligned) transcript at the lower-third in a kinetic style. Palette-matched.

Pure ffmpeg, no AI inference, fast.

> **For the agent**: re-align SRT timestamps after trimming — the cuts shift everything. If the burn step gets out-of-sync, regenerate the SRT from the trimmed master and re-burn.

### Step 4 — Generate B-roll inserts (~5 min, ~$0.80)

The agent calls `generate_project` with the `broll_inserts` list — one image per entry. Brand-locked palette, premium quality, composition matched to `target_audience` (beginner audience gets clearer diagrams; advanced audience gets denser conceptual visuals).

Then `ffmpeg-overlay` cuts each B-roll in over its trigger_phrase's timestamp window in the trimmed master — typically 3-5 seconds of B-roll covering the screen capture, then back to the screen. Audio remains continuous underneath.

> **For the agent**: pass `quality_threshold: 0.85` on each B-roll. `recraft-v4` is best for diagrams, `gpt-image` for anything with embedded text / labels, `flux-dev` for general conceptual illustration. If a `trigger_phrase` can't be found in the transcript (typo), surface it for user confirmation rather than guessing.

**STOP**: "B-roll inserts ready. Approve all, or redo specific? (approve / redo [#] [direction])"

### Step 5 — Chapter markers + zoom-focus moments (no gate, ~3 min, ~$0.10)

The agent calls Gemini Flash to segment the trimmed transcript into 4-6 chapters with section titles. Then `hyperframes-lower-third` overlays a chapter title card (brand-palette, fades in / fades out 1.5s) at the start of each chapter.

For each timestamp in `zoom_focus_moments`, `ffmpeg-overlay` applies a 2x smooth zoom-and-hold on the screen capture for 2 seconds — emphasizes whatever UI element you wanted the viewer to focus on.

> **For the agent**: chapter markers should be 5-15 words each — descriptive section labels, not vague ("Setting the webhook endpoint", not "Setup"). If user provided `zoom_focus_moments`, ask what UI element each one focuses on so the zoom centers correctly (default: center of frame).

### Step 6 — Music bed + intro/outro frames (~2 min, ~$0.40)

The agent calls `create_media({ action: "music", prompt: "minimal corporate / non-distracting", lyrics_prompt: "[Instrumental]", duration: <≥ trimmed video length> })` for a soft instrumental bed. Then `ffmpeg-audio-mix` lays it under everything at -18dB, ducked further during voiceover sections so it never competes with your narration. **Critical:** pass `lyrics_prompt: "[Instrumental]"` explicitly — otherwise minimax-music sings the style description as the lyrics.

`hyperframes-lower-third` composes the intro frame (`tutorial_topic` as title, your `your_handle` as subtitle, palette-matched, fades over 2s) and the outro frame ("More at @your_handle" + "Built with Storyboard"). Concatenated to start + end of the master.

> **For the agent**: pass `quality_threshold: 0.75` on music (wide acceptable variance). If the user wanted no music, skip music gen — surface a confirmation that music was skipped on purpose.

**STOP**: "Music + intro + outro approved? (approve / regenerate music / refine intro)"

### Step 7 — Export + tease clip (no gate, ~2 min, ~$0.10)

`ffmpeg-export` produces the final tutorial MP4 at platform-ready bitrate. Then for the tease: the agent picks the strongest 60-second window (LLM-selected: the most "hooky" insight + the most visually clear B-roll), and calls `ffmpeg-export` again with `crop: "center-crop"` to convert that window to 9:16 portrait for social.

### Final step — scorecard wrap-up

Call `get_scorecard({project_id: ...})` — paste output to your channel.

```
Scorecard for cjob_xxx
N/N steps shipped — critic: SHIP
Title: {tutorial_slug}
Steps: N/N succeeded (100%)
Quality: mean X.XX · M/N passed · K retries
Cost: $X.XX actual (est $Y.YY) · Ts wall time
Viewer: <tutorial URL>
```

## When you're done

```
✅ Final tutorial: <URL>
✅ 60s tease clip (9:16): <URL>
✅ Transcript SRT: <URL>
✅ B-roll grid: <URL × N>
✅ Chapter list: <text>

Total spent: $X.XX
Total wall-clock: MM:SS
```

## What to do next

Upload the full tutorial to YouTube — use the SRT as the official caption track + the chapter list as YouTube chapter timestamps in the description. Post the 60s tease to LinkedIn / Threads / X with a link back to the full video.

For developer advocates and SaaS marketers: record one screen capture per feature ship, run this playbook, post. This is the unlock — you don't need a video team to ship pro-quality tutorials weekly.

For internal training / onboarding: record the SME once, run this playbook, the result is the new-hire training material. Save the `brand_palette` once and every future tutorial runs in 25 minutes from upload to publish-ready.

For solo course creators: stack tutorials from one playbook into a multi-lesson course. Brand kit stays locked across lessons so the course feels unified — not "a bunch of Loom recordings stitched together."

---

## Notes for the agent (only read if a step fails)

**Transcription fallback**: if `wizper` fails, fall back to extracting audio with `ffmpeg-trim` (no trim — full file) → re-route through `nemotron-asr` (streaming multilingual ASR with native punctuation).

**B-roll trigger_phrase miss**: if a `trigger_phrase` doesn't appear verbatim in the transcript (paraphrased by speaker), do fuzzy matching on substrings; if still no match within 90% confidence, surface for user clarification before continuing.

**Music ducking**: `ffmpeg-audio-mix` doesn't always honor sidechain-ducking params via the MCP envelope. Fall back to direct SDK with explicit `sidechain: true` + threshold params if music is overpowering narration.

**Zoom-focus center**: default is image center. If user wants a specific UI element (button, code line), ask them to mark the pixel coordinates or describe it precisely.