One network for AI media creation
AI generation, deterministic finishing tools, and external publishing — under a single envelope, a single bearer, and a single agent runtime. Built for teams shipping media-creation features without owning the GPU stack, the tool stack, or the orchestration glue.
Generation, finishing, and publishing are all reachable through the same
POST /inference {capability, input}shape, billed to the same Daydream key.The three deployment shapes
| Shape | Who runs the loop | Pick when |
|---|---|---|
| CLI / SDK | Your terminal or Node service | Backend workflows, batch jobs, CI integration |
| MCP server | Claude Code / Cursor / ChatGPT | Lowest-friction distribution. Your users live in their AI tool already. |
| Embedded runtime | Your app imports @livepeer/agent | Full control. Add your own tools alongside the network. |
Quick Start
Four supported integration paths. Pick by where your users live. Each one authenticates with the same Daydream API key (sk_…) — get yours at app.daydream.live.
① Set up the MCP server (Claude Code, Cursor, ChatGPT)
The Storyboard MCP exposes the entire network as tools your AI client can call:generate_project,create_media,list_capabilities,submit_agent_task.
{
"mcpServers": {
"storyboard": {
"url": "https://storyboard.daydream.monster/api/mcp/sse",
"transport": "sse",
"headers": {
"Authorization": "Bearer sk_<your-daydream-key>"
}
}
}
}Restart Claude Code. The tools appear as mcp__storyboard__*. Try:
Use the storyboard MCP to give me a 12-scene illustrated short story about a Scottish ferryman's last crossing — watercolor pencil, John Bauer x Ghibli, 3:2, character-locked across scenes.
② Install and use the CLI
One-line install. Runs locally as a terminal app or a one-shot verb.
curl -fsSL https://storyboard.daydream.monster/cli/install.sh | bash livepeer setup --non-interactive --api-key sk_<your-daydream-key> # One-shot livepeer agent run "make me a watercolor fox in Studio Ghibli style" # Interactive REPL livepeer livepeer> /story 8-scene story about a fox in autumn livepeer> /film 4-shot sci-fi opener about a derelict station livepeer> /talk "Welcome to Aurora" --face img-1
Switch providers anytime — your Daydream key is the default (Livepeer-hosted), but you can swap:
livepeer config llm gemini --model gemini-2.5-pro # Gemini direct livepeer config llm anthropic --model claude-sonnet-4-6 # Claude direct livepeer config llm ollama --model llama3.1 # local livepeer config llm livepeer # back to hosted
③ Use the Storyboard webapp
The visual surface — canvas, chat, projects, LoRA training, live streams. Best for visual approval and creative iteration.
1. Open https://storyboard.daydream.monster
2. Sign in with your Daydream account
3. Type in chat:
/story 6 scenes of a fox catching salmon in autumn
4. Watch the scenes render on the canvas in parallel
5. Right-click any card for context-menu actions:
• Animate (i2v) • Variations (×4)
• Restyle / Edit • Talking Video
• Convert to 3D • Virtual Try-On
• Analyze Media • Cinematic Video
6. /render to stitch the canvas into a final video④ Use the Agent SDK directly (@livepeer/agent)
Embed the runtime in your own app. Same agent loop, your tools alongside the network's.
npm install @livepeer/agent
import {
AgentRunner,
ToolRegistry,
WorkingMemoryStore,
LivepeerProvider,
} from "@livepeer/agent";
// 1. Provider (uses YOUR users' Daydream key for billing)
const provider = new LivepeerProvider({
apiKey: process.env.DAYDREAM_API_KEY!,
});
// 2. Tool registry — built-in tools work out of the box
const tools = new ToolRegistry();
// Or add your own:
tools.register({
name: "send_to_slack",
description: "Post an asset URL to a Slack channel",
parameters: {
type: "object",
properties: {
channel: { type: "string" },
url: { type: "string" },
},
required: ["channel", "url"],
},
execute: async (args) => {
await myAppSlackPost(args.channel as string, args.url as string);
return JSON.stringify({ ok: true });
},
});
// 3. Memory and runner
const memory = new WorkingMemoryStore();
const runner = new AgentRunner(provider, tools, memory);
// 4. Stream events
for await (const event of runner.runStream({
user: "Generate a hero image for the spring launch and post it to #marketing",
})) {
if (event.kind === "text") process.stdout.write(event.text);
if (event.kind === "tool_call") console.log("→", event.name, event.args);
if (event.kind === "tool_result") console.log("←", event.content);
}Direct HTTP — for any language. Same envelope every capability shares:
curl -X POST https://sdk.daydream.monster/inference \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk_<your-daydream-key>" \
-d '{
"capability": "flux-dev",
"input": { "prompt": "a watercolor fox", "image_size": "landscape_16_9" }
}'Architecture
Three consumer surfaces (CLI / MCP / Embedded) → SDK service → BYOC + Scope orchestrators → AI / Tool / MCP adapters. Everything routes the same envelope, authenticated by the same bearer.
Consumer surfaces (pick one or all three)
─────────────────────────────────────────
• Direct CLI / SDK (`@livepeer/agent`)
• As MCP server (Claude Code, Cursor, ChatGPT)
• Embedded runtime (import the SDK into your app)
│
│ Bearer: sk_<user-daydream-key>
│ Wire: POST /inference { capability, input } → { output }
▼
Livepeer agent layer
┌───────────────────────────────────────────────┐
│ • SDK service sdk.daydream.monster │
│ • Storyboard MCP + /api/mcp/sse + /api/llm │
│ • Hermes (long-task) agent.daydream.monster │
└───────────────────────────┬───────────────────┘
│
▼
Orchestrator network (go-livepeer, decentralized)
┌───────────────────────────────────────────────┐
│ • BYOC orchs AI / Tool / MCP capability │
│ • Scope orchs live video-to-video │
│ • Signer Arbitrum payment tickets │
└───────────────────────────┬───────────────────┘
│
▼
Adapters — all speak the same /inference envelope
┌───────────────────────────────────────────────┐
│ AI : Flux, Seedance, GPT-Image, Veo, │
│ Chatterbox, Music, Tripo, Gemini-text │
│ TOOL : ffmpeg-concat / -mux / -export, │
│ opencv-smart-crop, yolo-segment/-detect│
│ MCP : Slack, Drive, Linear, GitHub │
└───────────────────────────────────────────────┘Why this design wins
- One bearer pays for everything. LLM, AI inference, tools, MCP-bridged actions, live streams. No vendor-account juggling.
- Three kinds of capability, one envelope. AI / Tool / MCP all speak the same wire format. Pick by capability name; kind is metadata only.
- Config-only capability adds. New models land via JSON edit + adapter restart. No orchestrator code change, no client SDK release.
- Agent loop is the consumer's. CLI runs it locally. Claude Code runs it. Your app runs it. Same network underneath.
- Finishing + publishing included. ffmpeg-concat / opencv-smart-crop / slack-send-file are first-class — generation never has to leave the network.
Tools Reference
The agent uses 21 tools to create and manipulate media. Each tool has typed parameters and returns structured results. Tools are composed into multi-step workflows automatically by the LLM.
Media Creation
Canvas Operations
Live Streaming (Scope)
Skills
Skills are the agent's soft-coded knowledge layer — markdown files that teach the agent how to think, not whattools to call. They're fetched at runtime from /skills/*.md, ranked against the current intent, and injected into the system prompt before each turn. Edit a skill, save the file, the next turn already runs the updated behavior — no code change, no deploy.
How skills fit into the tool architecture
Tools execute. Skills orient — they bias the agent toward the right tool, the right parameters, and the right finishing pass. The agent loop pulls relevant skills at every turn:
User turn
│
▼
findSkills(userMessage, context) ─── ranks all skills by frontmatter
relevance (task, domain, persona,
scope) — top 3-5 win
│
▼
Skills concatenated into system prompt:
"You are a creative agent. CRITICAL CONSTRAINTS:
[base.md content]
[director-mcp.md content]
[finishing-quality.md content]
[storyteller.md content]
..."
│
▼
LLM call (with tools + system prompt)
│
▼
Agent fires tools according to skill guidance:
- "movie quality" → seedance-i2v (from director-mcp)
- "watercolor pencil" → gpt-image (from director-mcp model table)
- finishing pass mandatory (from finishing-quality)
- cut transitions for 3+ clips (from director-mcp)
│
▼
Skill outcomes scored after turn ends — feeds future
ranking (skills that produced good outcomes get bumped)Skill frontmatter (the routing metadata)
Every skill file starts with YAML frontmatter that the ranker reads. The body is markdown the LLM reads.
--- name: director-mcp description: "Director-language → multi-step MCP recipes; pick defaults, don't name models" agent_rule: "User describes a deliverable — pick model/params, don't make them name caps." task: [render, compose, animate, narrate] domain: [video, image, audio, film] persona: [agent-only] scope: epic --- # Director — natural language → MCP recipes When the user says "30-second cinematic clip," you pick flux-schnell for keyframes + seedance-i2v-fast for animation + ffmpeg-concat for stitching. They don't name models; you do. [... full markdown body teaches the agent ...]
Skills available today (40 in the registry)
Grouped by where they sit in a creative session.
base.mdCore agent rules — when to call create_media vs project_create, finishing-pass discipline, refusal handling.intent-classifier.mdIntent vocabulary — `compare_models`, `batch_generate`, `style_sweep`, `variations`, `story`, `single`, `unclear`.director.mdGeneral multi-step director: plan, generate, review, iterate for multi-scene work.director-mcp.mdDirector-language → MCP recipe table. Model defaults per intent (cinematic / cinemagraph / talking-head / social-reel).episode-director.mdTurn images-into-episode workflow. Cohesion prefix, motion prompts, transitions.claude-code-mcp.mdPlaybook for Claude Code orchestrating the Storyboard MCP — long-context, multi-tool, AAA quality.live-director.mdChat commands → LV2V stream_control mapping for live streams.scope-agent.mdScope Domain Agent: graphs, presets, LoRA, VACE, natural-language parameter mapping.storyboard.mdMulti-shot storyboard from a pasted scene description — split into shots, call create_media ONCE.storyteller.mdGenerates 6-scene visual stories: title, audience, arc, context, scenes.character-design.mdPer-character anchor + LoRA training guidance for cross-shot identity.prompt-craft.md7-layer prompt formula: camera, subject, surface, background, lighting, colors, atmosphere.text-to-image.mdModel selection guide, size rules, common pitfalls per model family.image-editing.mdkontext-edit usage, style transfer, reference-driven composition.cinematography.mdCamera framing + movement vocabulary. Lens choices, depth-of-field cues.style-presets.md15 built-in style presets with optimized prompt suffixes.grok-imagine.mdxAI Grok Imagine quality + edit usage notes.ideogram-bg-remove.mdHigh-fidelity subject isolation patterns.video.mdi2v/t2v pipeline selection, motion prompts, when to chain.hifi-video.mdTwo-step pipeline: GPT Image 2 keyframe → Seedance 2.0 animation.keyframe-video.mdBuilding cohesive multi-shot video from keyframes.happy-horse-cinematic.mdWorked example — cinematic horse video from keyframe → animation → finishing.scope-lv2v.mdScope live video-to-video parameters: noise, transitions, denoising step list.scope-graphs.mdScope pipeline graphs and selection (simple-lv2v, depth-guided, etc.).scope-pipelines.mdPipeline catalog: LongLive, LTX 2.3, Krea, MemFlow. 9 composable recipes.stream-source-analysis.mdSource-quality gates before starting a live stream.finishing-quality.mdThe finishing checklist — concat / mux / export / crop / overlay / burn captions. "Don't ship a URL list."refinement.mdIterative refinement: generate, evaluate, upscale.remix.mdCombine multiple canvas cards into composites.advanced-models.mdAdvanced model guide: video gen, audio, editing, 3D, export workflows.non-ai-tools.mdDeterministic Docker tool catalog: ffmpeg-*, opencv-*, yolo-*. When to reach for each.yolo-vision.mdSubject detection / segmentation / pose patterns for cinemagraphs + smart-crop.mcp-bridged-tools.mdWhen to publish via Slack / Drive / Linear / GitHub — what payload shape each expects.marketing.mdBrand-locked imagery, audience considerations, A/B variant patterns.commercial.mdCommercial-style pacing, hero-shot composition, brand frame conventions.ugc.mdInfluencer / handheld / phone-shot aesthetics — when and how to use them.daily-briefing.mdEmail-driven visual deck (Gmail MCP → slides).lora-training.mdLoRA training: image selection, trigger words, step counts, drift detection.nemotron-omni.mdNVIDIA Nemotron multimodal usage — text+image+video+audio reasoning.How skills change agent behavior — concrete before/after
A skill is text the LLM reads before deciding what to do. Same prompt + same tools, but with a skill loaded, the agent picks different models, different parameters, and a different finishing pipeline. Three short examples:
- 1× create_media(prompt='cinematic surfer sunset', capability='flux-dev')
- Returns ONE image URL — call ends.
- User has to ask for finishing, voice, music separately.
- Wave 1 parallel: 6 × flux-schnell keyframes + chatterbox-tts narration + music score (30s, instrumental).
- Wave 2 parallel: 6 × seedance-i2v-fast (5s, slow-dolly camera motion per shot).
- Wave 3 sequential: ffmpeg-concat (cut) → ffmpeg-audio-mix (voice 1.0, music 0.30) → ffmpeg-mux (replace).
- Returns ONE finished MP4 with audio. ~250s wall-clock, single deliverable.
- Agent uses base prompt as-is → flux-dev defaults.
- Scenes vary in style (#1 watercolor, #4 photoreal, #6 cartoon). Character drifts between shots.
- Outputs 8 inconsistent URLs.
- storyteller plans 8 scenes with title + arc + character anchor.
- studio-ghibli auto-prepends 'studio ghibli style, hand-painted watercolor...' to every prompt.
- Picks gpt-image (best stylization) over flux-dev for all 8 scenes.
- Outputs 8 cohesive scenes, returned as one viewer URL via generate_project.
- Agent guesses: tries kontext-edit with prompt 'remove background, keep cat'. ~6s.
- Result: ok but jagged edges, half-faded paw.
- Agent recognizes 'cut out' intent → yolo-segment with classes=['cat'], output_mode='alpha-cutout'. ~1s.
- Pixel-precise alpha PNG. Tells the user 'cinemagraph-ready'.
The skill doesn't add new capabilities — those already exist on the network. The skill teaches the agent which one to reach for, with what parameters, in what wave order, and what finishing pass to run. Same hands, better training.
Loading, adding, and unloading skills — across all three surfaces
Skills are managed the same way conceptually across MCP, CLI, and the Storyboard webapp — the syntax differs by surface. The active-set is per-user (persisted), so a skill loaded via the CLI is also active for your next REPL session and surfaces in the webapp's "active skills" pill bar.
| Action | MCP (Claude Code / Cursor) | CLI | Storyboard chat |
|---|---|---|---|
| list | Natural language: "Which skills are available?" Agent runs livepeer.list_skills | livepeer skills list | /skills/list |
| show | "Show me what director-mcp teaches." | livepeer skills show <id> | /skills/show <id> |
| load (pin for session) | "Apply the studio-ghibli skill for the rest of this chat." Agent runs livepeer.apply_skill_pack | livepeer skills load <id> | /skills/load <id> |
| load by category | "Activate all marketing skills." | livepeer skills load --category marketing | /skills/load-by-category marketing |
| unload | "Drop the ugc skill, I want polished output now." | livepeer skills unload <id> | /skills/unload <id> |
| show active set | "What skills are active right now?" | livepeer skills active | Top of chat — always-visible pill bar |
| add (author + install) | "Make a new skill for our brand voice." Agent runs livepeer.gen_skill | livepeer skills new livepeer skills validate livepeer skills install | /skills/new Or autogenesis (post-turn) |
| publish to public registry | Roadmap — MCP publish tool | livepeer skills publish <id> | /skills/publish <id> |
| private sync across devices | Bearer-scoped; happens automatically | livepeer skills push livepeer skills pull | Automatic on sign-in |
| auto-load on/off | Server-side; admin-controlled | livepeer skills auto-load on|off | Settings → Skills |
The active-set is shared across surfaces
Skills you load via the CLI persist to ~/.livepeer/skills/active.json AND to a per-user blob mirror keyed by your Daydream bearer. The next time you sign into the Storyboard webapp the same skills are active. Sign into Claude Code with the same MCP key — same skills again. One bearer, one active set, all three surfaces.
# Day 1 — author and pin on your laptop CLI $ livepeer skills new --name aurora-brand-voice $ livepeer skills load aurora-brand-voice $ livepeer agent run "make me a launch hero image" # Hero respects the brand voice immediately. # Day 2 — fire up the Storyboard webapp # The chat header pill bar shows: [aurora-brand-voice ●] # Every generation already brand-locked. # Day 3 — switch to Claude Code (MCP) for an iterative session > "Use the storyboard MCP to draft 4 social cuts." # Claude calls list_skills, sees aurora-brand-voice active # and uses it without you asking. # Roll back anywhere: $ livepeer skills unload aurora-brand-voice # OR in the webapp: /skills/unload aurora-brand-voice # OR ask Claude Code: > "Drop the aurora-brand-voice skill — I want generic output."
Under-the-hood — what each surface actually does
Every surface ends up doing the same thing: it modifies the user's active-set and the next agent turn reads the updated set when it builds the system prompt.
User action on any surface
│
▼
1. Active-set update:
local: ~/.livepeer/skills/active.json
remote: storyboard-mcp/user-mirror/<user-hash>.json (Vercel Blob,
mirrored across
devices)
│
▼
2. Next agent turn fires:
findSkills(userMsg, { tools, persona })
→ ranks ALL skills by relevance
→ top-N + every pinned skill from active-set
always wins (pinned override ranking)
│
▼
3. System prompt assembled:
[base.md content]
[pinned skill 1] ← from your active-set
[pinned skill 2]
[ranked-in skill 3] ← matched the current intent
[ranked-in skill 4]
│
▼
4. LLM call (with system prompt + tools)
│
▼
5. Outcome scored at turn end:
- did the agent reach for a tool the skill recommends?
- was the result rated thumbs-up?
- update skill's effectiveness score in the per-user ledger
│
▼
6. Future rankings boost or demote based on outcomes
(your skills get smarter with use)Extending and personalizing — write your own
Skills are file-based. Drop a new .md in public/skills/ (or via the chat autogenesis flow), register a one-line entry in public/skills/_registry.json, and your skill is loaded by the ranker on next session.
---
name: my-brand-voice
description: "Brand voice for Aurora Audio — premium, understated, hopeful"
agent_rule: "All copy targets Aurora Audio brand voice. Premium without superlatives."
task: [render, narrate]
domain: [image, video, audio]
persona: [agent-only]
scope: epic
---
# Aurora Audio brand voice
Tone: confident, understated, premium. Avoid superlatives ("uncompromising",
"the best", "world's first"). Prefer warmth ("the headphones that come home
with you", "made for the quiet hour").
Visual cues:
- Always cool blue / charcoal / brushed silver palette
- Studio lighting, never sunlight
- Product on minimalist surfaces — concrete, walnut, marble
Caption patterns:
- TikTok: short, sensory, second-person ("you'll hear...")
- Instagram: longer, story-led, soft pull-quote
- LinkedIn: hopeful, work-life balance angle
When generating campaign creative, always reference this skill alongside
director-mcp.md and finishing-quality.md.{
"id": "my-brand-voice",
"category": "vertical",
"description": "Aurora Audio brand voice — premium understated hopeful",
"tags": ["brand", "voice", "aurora"]
}Skill autogenesis (the meta-skill)
Skills can be born from successful sessions. After a turn finishes well, the agent (via the skills B3 autogenesis flow) drafts a candidate skill capturing what made it work — proposed via the promotion UI; the user approves or rejects. Approved skills go into the registry and start informing future turns. Your agent gets smarter without a human writing markdown.
Agent Marketplace
The Agent Marketplace is the registry of pre-built, runnable agents you can summon by name. Where a skill teaches, an agent does — each marketplace entry bundles a system prompt + a curated skill stack + default tool gates + (optionally) a trained LoRA + a brand kit. Summon one and the conversation already knows who it is and how to behave.
How agents work in the network
User chat: Or CLI: Or MCP:
/agent aurora-marketer livepeer agent run \ mcp__storyboard__
"launch the X1 next Tuesday" --agent aurora-marketer \ create_agent_task
"launch the X1 next Tuesday" agent_id: aurora-marketer
│ │ prompt: "launch ..."
│ │ │
└──────────────┬─────────────────┘ │
│ │
▼ ▼
┌──────────────────────────────────────────────────────────────┐
│ Agent registry resolves "aurora-marketer" to: │
│ • system_prompt: "You are Aurora's brand director ..." │
│ • skill_refs: [director-mcp, marketing, my-brand-voice] │
│ • lora_refs: [bkit_aurora_lora_v3] │
│ • tool_allowlist: [create_media, project_create, …] │
│ • brand_kit_id: bkit_aurora_a1b2c3 │
│ • author + signature + reputation score │
└──────────────┬───────────────────────────────────────────────┘
│
▼
Agent loop runs with this bundle pre-loaded.
First turn is already brand-aware, model-aware,
and finishing-pass-aware.Off-the-shelf agents (live in the marketplace today)
Available at storyboard.daydream.monster/marketplace. Each agent is callable from CLI, MCP, and the webapp.
story-architect12-scene illustrated story planner. Outputs scenes + style prefix + character anchor. Best for fiction, fables, brand stories.graphic-novelistMulti-page graphic-novel pacing — panels, dialogue, character consistency, page transitions.documentary-narratorB-roll planning + voice-over scripting from a brief. Pairs with chatterbox-tts.short-film-directorEnd-to-end 2-min short film: keyframes + i2v + voice + score + finishing + export.cinemagraph-artistMagic-moving-photo specialist. yolo-segment + i2v + ffmpeg-overlay pipeline.music-video-directorLyric/music-driven video. Beat-synced cuts, prompt traveling across scenes.happy-horse-cinematicWorked-example agent showing the gpt-image → seedance → finishing chain.campaign-launcherBrand-LoRA + multi-platform campaign — hero spot, social cuts, stills, carousel, scheduling.social-shorts-creator8-episode TikTok-format series with continuity. Same character, evolving narrative.product-stageSingle-product hero shots with brand frame and lifestyle context library.character-sheetGame character: front/3-4/side/back turnaround, expressions, LoRA training, 3D mesh export.environment-conceptStylized environment passes with controlnet preprocessor pack.scope-vjLive VJ — multi-scene Scope stream with prompt traveling, beat-synced parameter modulation.talking-presenterImage + script → talking-head video, lip-synced.image-analyzerGemini Vision-driven analysis: style, palette, characters, setting, mood.asset-finisherPure finishing-pass agent — give it N urls, get a stitched MP4 with audio, captions, platform export.brief-validatorPre-flight check on a campaign brief — compliance flags, missing fields, cost estimate.The agent manifest
Every marketplace agent is a signed JSON manifest in Vercel Blob storage. Tamper-proof; reputation tracked by signature.
{
"id": "aurora-marketer",
"version": "1.4.2",
"title": "Aurora Audio brand director",
"author": "@aurora-creative",
"signature": "0xabcdef...", // signed with author's wallet
"reputation": 4.8, // 0-5 from rated runs
"system_prompt": "You are Aurora's brand director. Premium, understated...",
"skill_refs": [
"director-mcp",
"marketing",
"finishing-quality",
"my-brand-voice"
],
"lora_refs": [
"bkit_aurora_lora_v3" // private LoRA, accessible to owners
],
"brand_kit_id": "bkit_aurora_a1b2c3",
"tool_allowlist": [ // what this agent is allowed to call
"livepeer.create_media",
"livepeer.project_create",
"livepeer.project_generate",
"livepeer.canvas_organize",
"ffmpeg-*",
"publora-schedule"
],
"pricing": {
"model": "free" | "per-run" | "subscription",
"fee_eth_per_run": "0.001", // marketplace fee on top of compute
"creator_split": 0.7 // 70% to author, 30% to network
}
}Building and publishing your own agent
Three steps. The webapp has a guided builder at /marketplace/register — the API is also exposed for scripted submission.
# 1. Build the manifest locally (CLI helper) livepeer agent build \ --id my-fashion-director \ --title "Fashion campaign director" \ --skills marketing,director-mcp,finishing-quality \ --brand-kit bkit_aurora \ --lora lora_aurora_v3 \ --tool-allow create_media,project_create,publora-schedule \ --pricing per-run --fee 0.0005 --output ./my-agent.json # 2. Sign it with your wallet livepeer agent sign ./my-agent.json --key ~/.daydream/wallet.key # 3. Register on the marketplace livepeer agent publish ./my-agent.json # → Published as agent_a1b2c3d4 — your share of fees lands in # → 0x1234... on Arbitrum within 24h of each invocation.
Sales & reputation
When users summon your agent, the marketplace records the run, optionally collects a rating, and updates your reputation. Earnings (your creator split of the per-run fee) settle on-chain to the wallet you signed the manifest with. Subscription pricing is supported for agents bundled with proprietary LoRAs or skills.
- Free agents — no per-run fee. You earn reputation, the network handles compute billing only.
- Per-run agents — flat fee per invocation. Best for utility agents (validators, finishers, analyzers).
- Subscription agents — monthly access. Best for agents that bundle a trained LoRA, a proprietary skill, or ongoing model updates.
Extending an existing agent (forking pattern)
Pull any public agent manifest, modify, sign, re-publish under your own id. Original author retains reputation; your version starts fresh and competes on its merits. Forking is encouraged — the network rewards differentiation via the rating system.
livepeer agent fork aurora-marketer --new-id studio-x-marketer # Opens the manifest in your editor. Tweak the system prompt, # swap skill_refs, change pricing. Sign + publish. livepeer agent publish ./studio-x-marketer.json
Workflows
Slash commands orchestrate multi-step creative workflows. Each creates a draft, lets the user edit inline, then applies.
/story <concept>/film <concept>/stream <concept>/talk <text>Conversation Continuity
User: /story a dragon adventure → 6-scene story card (editable) User: add more scenes about finding treasure → System detects continuation → generates 3 matching scenes → appends → Updated card with NEW badges → user deletes unwanted → Apply User: /story a space adventure → resetForNewWork() clears old context → fresh story, no bleed
Live Streaming
Real-time AI video generation using Scope. The stream processes input frames through an AI pipeline, transforming them based on text prompts.
Stream Recipes
classicLongLiveltx-responsiveLTX 2.3ltx-smoothLTX+RIFEdepth-lockLongLive+Depthkrea-hqKrea 14Bmemflow-consistentMemFlowStream Lifecycle
import { ScopePlayer } from "@livepeer/scope-player";
// In your React component:
<ScopePlayer
sdkUrl="https://sdk.daydream.monster"
apiKey="sk_your_key"
externalStreamId={streamId}
onStateChange={(state) => console.log(state.status, state.fps)}
onSourceReady={(setSource) => {
// Drag an image onto the player to transform it live
setSource({ type: "image", url: imageUrl });
}}
/>Prompt Traveling (Scene Transitions)
import { PerformanceEngine } from "./performance";
const engine = new PerformanceEngine();
engine.setScenes([
{ index: 0, title: "Dawn", prompt: "golden sunrise over mountains", preset: "cinematic", duration: 30 },
{ index: 1, title: "Storm", prompt: "dark storm clouds, lightning", preset: "abstract", duration: 15 },
{ index: 2, title: "Night", prompt: "starry sky, aurora borealis", preset: "dreamy", duration: 25 },
]);
// Transitions use noise_scale + kv_cache ramp for smooth morphing
engine.play(controlFn, onStateUpdate);
engine.pause(); // freeze at current point
engine.resume(); // continue from where pausedCreative Kit
Shared framework for building creative apps. Both Storyboard and Creative Stage use it.
Stores (Zustand)
import {
createArtifactStore, // Canvas cards (images, videos, audio)
createChatStore, // Chat messages
createProjectStore, // Multi-scene projects
createGroupManager, // Episode/collection grouping
createConversationContext, // Active work tracking
} from "@livepeer/creative-kit";
const artifacts = createArtifactStore({ maxArtifacts: 200 });
artifacts.getState().add({ type: "image", title: "Dragon", url: "...", refId: "img-1" });Model Router (Self-Learning)
import { routeModel, recordModelLatency, getModelStats } from "@livepeer/creative-kit";
// Scores: speed (60%) + style match (30%) + capacity (10%)
const result = routeModel({
action: "generate",
prompt: "a logo with text",
availableModels: new Set(["flux-dev", "gpt-image", "recraft-v4"]),
});
// After each inference, feed back actual speed → router learns
recordModelLatency(result.model, elapsedMs);
// Check learned stats
const stats = getModelStats();
// → Map { "flux-dev" → { avgMs: 3200, count: 47, currentSpeed: 8.9 } }Pipeline Registry
import { createPipelineRegistry } from "@livepeer/creative-kit";
const registry = createPipelineRegistry();
// Resolve user intent to the best recipe
const recipe = registry.resolve("smooth 24fps stream");
// → { id: "ltx-responsive", pipeline: "ltx2", graph: {...}, defaults: { kv_cache: 0.3 } }
// List all recipes by quality tier
const quality = registry.listRecipes("quality");
// → [depth-lock, ltx-smooth, krea-hq, memflow-consistent, ...]REST API Reference
Base URL: https://sdk.daydream.monster · Auth: Authorization: Bearer sk_...
/inferenceRun AI inference (image, video, audio, 3D)/capabilitiesList all 48 available models/healthService health check/stream/startStart live AI stream/stream/{id}/publishSend input frame (JPEG)/stream/{id}/frameGet output frame (JPEG)/stream/{id}/controlUpdate stream params/stream/{id}/stopStop stream/streamsList active streams/api/agent/geminiGemini proxy (app route)/api/agent/chatClaude proxy (app route)/api/agent/openaiOpenAI proxy (app route)Example: Generate Image → Animate → Get Video
# Step 1: Generate key frame
IMG=$(curl -s -X POST https://sdk.daydream.monster/inference \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk_your_key" \
-d '{"capability":"flux-dev","prompt":"a majestic eagle soaring"}' \
| jq -r '.image_url // .data.images[0].url')
echo "Image: $IMG"
# Step 2: Animate to video (10 seconds with audio)
curl -X POST https://sdk.daydream.monster/inference \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk_your_key" \
-d "{
\"capability\": \"seedance-i2v\",
\"prompt\": \"eagle gliding through mountain valley, cinematic camera follow\",
\"params\": { \"image_url\": \"$IMG\", \"duration\": \"10\", \"generate_audio\": true }
}"
# → { "data": { "video": { "url": "https://..." } } }AI Models
60+ AI capabilities live on the network. Each is reachable ascapability: <name>on the /inference envelope. Adding a new model is a config-only change to CAPABILITIES_JSONon the BYOC orchestrator — no client release needed. Live list: /capabilities.
flux-dev~3sDefault. Cinematic photoreal.flux-schnell~1sFastest draft / preview.flux-pro / flux-flex~8sHigh-fidelity product photography.gpt-image~8sBest for text, logos, stylization.recraft-v4~6sProfessional illustration, editorial.seedream-5-lite~5sStrong photorealism.gemini-image~6sPainterly, watercolor, artistic.nano-banana~4sMultimodal image model.grok-imagine-quality~6sxAI Grok Imagine text-to-image.kontext-edit~6sReference-driven edit. Preserves identity.gpt-image-edit~10sInstruction-based edit (strong stylization).grok-imagine-edit~6sGrok edit model.flux-fill~5sInpaint / outpaint.seedance-i2v~30sDefault. 15s cinematic + per-clip audio.seedance-i2v-fast~12s5s preview clips. Storyboarding.veo-i2v~20sGoogle Veo i2v. 8s max.kling-v3-turbo-i2v~40sKling 3.0 Turbo. Fast, stable motion, superior lip-sync (720p; -pro = 1080p).ltx-i2v~15sOpen-source. Fast.pixverse-i2v~20sPixverse i2v.veo-t2v~20sGoogle Veo 3.1 fast text-to-video.ltx-t2v~15sFast open-source t2v.pixverse-t2v~20sPixverse t2v.veo-transition~25sInterpolate between two keyframes.pixverse-transition~25sSame, Pixverse.pixverse-ref2v~25sReference-driven video.void-inpaint~30sVideo inpainting.talking-head~40sLip-sync image + audio → video.lipsync~25sAdd lip-sync to existing video.face-swap~20sIdentity transfer.fashn-tryon~30sVirtual try-on (garment + person).chatterbox-tts~3sTTS + voice cloning (pass audio_url).gemini-tts~3sTTS with persona/style control.inworld-tts~3sFast natural TTS — staging-reliable.grok-tts~3sxAI Grok TTS.music~20sScore / instrumental music.sfx~10sSound-effects generation.tripo-i3d~30sImage → 3D mesh (.glb).tripo-t3d~30sText → 3D mesh.tripo-mv3d~40sMulti-view → 3D mesh.tripo-p1-t3d~40sTripo Pro tier text-to-3D.tripo-p1-i3d~40sTripo Pro tier image-to-3D.bg-remove~3sBirefnet — fast bg removal.ideogram-bg-remove~6sHigh-fidelity subject isolation.sam3~5sSAM 3 prompt-conditioned segmentation.topaz-upscale~15sAura-SR upscale (4x photo / video).nemotron-omni~10sNVIDIA Nemotron — text+image+video+audio.gemini-text~2sGemini 2.5 Flash — LLM via /llm/chat.longlive8-12fpsDefault Scope pipeline. LoRA + VACE.ltx224fpsLTX 2.3. Fast prompt response.krea_realtime_video6-8fps14B model. Highest quality.memflow8fpsMemory bank. Best consistency.Non-AI Tools (Deterministic Docker)
Finishing-pass capabilities. Same /inferenceenvelope, same Daydream key. These run as Docker containers on the orchestrator — sub-second, near-zero cost (compute only, no model fee). The agent uses these to turn AI output into deliverables: stitched videos, platform-cropped clips, watermarked assets, captioned exports.
ffmpeg-concat~5sStitch N clips with cut/crossfade.ffmpeg-mux~5sCombine video + audio track (replace mode).ffmpeg-audio-mix~5sMulti-track mix (voice + music + sfx, per-track volume + delay).ffmpeg-overlay~5sComposite logo / watermark on video.ffmpeg-burn-subtitles~10sBurn captions from cue list.ffmpeg-export~10sPlatform preset export (tiktok / instagram / youtube / x / linkedin).ffmpeg-loop~5sLoop a clip to target duration or count.ffmpeg-color-grade~10sLUT application, log-to-Rec.709 (planned).opencv-smart-crop~1sFace-aware crop. Pair with yolo-detect for focus_x.yolo-detect<1sObject detection. Returns bounding boxes + labels (annotated jpg optional).yolo-segment~1sPixel-precise segmentation. output_mode='alpha-cutout' for cinemagraph.yolo-pose~1sBody / face keypoints. Quality-gate before talking-head.brand-style-extract<1sDominant palette, detected font family, logo location (planned).whisperx-transcribe~5sAuto-caption SRT from audio (planned).caption-cuesheet-build<1sPer-platform teaser captions from one master (planned).hyperframes~2sHyperframe stylization passes.obscura~2sObscura-style image effects pack.rife-interpolate~8sRIFE frame interpolation 2x/4x (planned).MCP Bridges (External services)
Third-party services proxied through the Livepeer MCP bridge. Same envelope, same auth path — first call surfaces an OAuth link, subsequent calls succeed silently. The agent uses these to finish the loop: published, scheduled, delivered. No custom client SDKs to maintain.
slack-send-fileOAuthPost asset to a Slack channel with caption.drive-uploadOAuthPush to Google Drive folder (creates folder if missing).linear-create-issueOAuthFile a Linear issue with the campaign asset attached.github-create-prOAuthOpen a PR with generated assets staged.github-issue-commentOAuthComment on an existing issue with a generated mockup.notion-create-pageOAuthDrop a launch brief into a Notion DB.publora-scheduleOAuthSchedule a post across TikTok / IG / YouTube / X / LinkedIn in one call.publora-analyticsOAuthEngagement + reach per scheduled post.brandfetch-fetchAPI keyPull official palette / fonts / logo from a brand's domain.figma-exportOAuthHand off final carousel to a Figma file.frameio-uploadOAuthPush final cut to Frame.io for review.meta-graph-insightsOAuthDirect Meta Graph analytics (Publora fallback).tiktok-businessOAuthDirect TikTok Business API for analytics + boost.salesforce-queryOAuthPull customer segments for per-segment campaigns.hubspot-contactsOAuthAudience pull for marketing automation.discord-postOAuthDrop generated assets into Discord channels.buffer-scheduleOAuthBuffer scheduling as Publora alternative.unity-asset-importAPI keyDrop .glb + spritesheet into a Unity project (Splash 2).bynder-publishOAuthDAM integration for enterprise brands.patreon-postOAuthPaid-tier delivery for artist creators.artstation-publishOAuthPortfolio publishing for artists (Splash 4).Read the full architecture page — control flow, payment tickets, and the extension guide.