notque/vexjoy-agent / skills/content/video-editing
video-editing
VerifiedVideo editing pipeline: cut footage, assemble clips via FFmpeg and Remotion.
Video Editing Skill
Overview
This skill implements a 6-layer pipeline where AI handles judgment tasks (what to keep, what to cut, highlight selection) and FFmpeg/Remotion handle mechanical execution deterministically.
| Layer | Name | Mechanism | Primary Tool |
|---|---|---|---|
| 1 | CAPTURE | Inventory source footage | Bash + Glob |
| 2 | AI STRUCTURE | Transcript to EDL | LLM judgment |
| 3 | FFMPEG CUTS | EDL to segment files | FFmpeg (deterministic) |
| 4 | REMOTION COMPOSITION | Segments to TSX composition | Remotion + TypeScript |
| 5 | AI GENERATION | Fill gaps with generated assets | ElevenLabs / fal.ai (conditional) |
| 6 | FINAL POLISH | Human taste layer | Human + NLE |
Deep References
| Signal | Load | Content |
|---|---|---|
| Shell commands for each phase, gate checks | references/phase-commands.md | Full command blocks for Phases 1-6 |
| FFmpeg recipes: cutting, concat, proxy, detection | references/ffmpeg-commands.md | Timestamp extraction, batch cutting, audio normalization, scene/silence detection |
| Remotion composition scaffold | references/remotion-scaffold.md | TSX composition template, render command, reuse patterns |
| Image-to-video pipeline | references/image-to-video.md | Validate, prepare, encode, verify workflow |
| FFmpeg filter graphs for visualization | references/ffmpeg-filters.md | Scale/pad, showwaves, showspectrum, overlay, complete filter graphs |
Instructions
Preflight (Run before Phase 1)
Hard requirements (BLOCK if missing): ffmpeg (all phases), node (Remotion / npx).
Soft requirements (WARN if missing): remotion (only required for Phase 4).
which ffmpeg >/dev/null 2>&1 || { echo "ERROR: ffmpeg not found. Install: brew install ffmpeg (macOS) | apt install ffmpeg (Linux)"; exit 1; }
which node >/dev/null 2>&1 || { echo "ERROR: node not found. Install: https://nodejs.org or via nvm"; exit 1; }
npx remotion --version >/dev/null 2>&1 || echo "WARNING: remotion CLI not found. Phase 4 unavailable. Install: npm install @remotion/cli"
Phase 1: CAPTURE
Goal: Inventory all source footage and confirm files exist on disk before any processing.
Constraint: Source files are read-only. All FFmpeg commands write to new files only. Never overwrite source footage.
Steps: locate source files (find), inspect each with ffprobe, generate proxies for files >10 min (see references/ffmpeg-commands.md -> Proxy Generation), create working directories (segments/, assembled/). Full command block: references/phase-commands.md -> Phase 1.
Gate: Source files confirmed on disk. File list written to source-inventory.txt. Proceed only when gate passes.
Phase 2: AI STRUCTURE
Goal: Analyze content and produce a written EDL (cuts.txt) that drives all downstream cutting.
Constraint: cuts.txt is the contract. The EDL file is the only source of truth for downstream phases. Do not hand-edit FFmpeg commands — generate them from the EDL.
Constraint: Before writing the EDL manually, run FFmpeg scene/silence detection. Detection output informs judgment, not replaces it.
Steps: transcribe with whisper/AssemblyAI, run scene/silence detection, apply judgment (what advances narrative, what is filler, target duration), write cuts.txt in EDL format START_TIME,END_TIME,LABEL, review for overlap and total duration. Full command block and EDL format example: references/phase-commands.md -> Phase 2.
Gate: transcript.txt exists. cuts.txt written to disk. Proceed only when both files exist.
Phase 3: FFMPEG CUTS
Goal: Execute the EDL deterministically — one FFmpeg cut per segment in cuts.txt.
Constraint: Batch-cut from EDL using a loop. Do not create individual FFmpeg commands per cut. This ensures reproducibility and review capability as a list.
Constraint: Always generate concat-list.txt from cuts.txt order, not from shell glob. Shell glob (segments/*.mp4) sorts alphabetically, not by EDL order.
Steps: batch-cut from EDL with while loop (libx264/aac, -avoid_negative_ts make_zero), verify segments, generate concat-list.txt in EDL order, concat with -f concat -safe 0 -c copy. Full command block: references/phase-commands.md -> Phase 3. See also references/ffmpeg-commands.md -> Batch Cutting.
Gate: All segment files exist. assembled/rough-cut.mp4 written to disk. Proceed only when gate passes.
Phase 4: REMOTION COMPOSITION
Goal: Wrap segments in a Remotion TSX composition for programmatic overlays, titles, or transitions.
When to use: Only when rough-cut.mp4 requires programmatic elements (animated titles, lower thirds, caption tracks, brand overlays). If rough-cut.mp4 is sufficient, skip to Phase 6.
Constraint: Layer 4 requires TypeScript/React. Hand off to typescript-frontend-engineer for TSX work; return to python-general-engineer for Phase 5 onward.
Steps: initialize Remotion (npm create video@latest first time; otherwise npm install @remotion/cli @remotion/player remotion), scaffold composition (see references/remotion-scaffold.md), render with npx remotion render. Full command block: references/phase-commands.md -> Phase 4.
Gate: assembled/remotion-output.mp4 exists. Proceed only when gate passes.
Phase 5: AI GENERATION
Goal: Fill genuine gaps in source material with generated assets — only when needed.
Constraint: Check whether existing footage covers the gap before generating anything. Generate only what doesn't exist.
Decision tree: cut around the gap → update cuts.txt and re-run Phase 3; voiceover → ElevenLabs (authorization required); music → fal.ai (defer to fal-ai-media); b-roll → fal.ai (defer to fal-ai-media). ElevenLabs Python helper, authorization pattern, and save-to-disk flow: references/phase-commands.md -> Phase 5.
Gate: All required generated assets saved to assets/ directory before proceeding.
Phase 6: FINAL POLISH
Goal: Deliver assembled output and hand off taste-layer work to human.
Constraint: Layer 6 is human territory. The skill assembles; the human finishes. The following require human judgment and should not be attempted programmatically:
- Color grading and color matching between clips
- Music timing and volume ducking
- Caption style, font, positioning
- Transition timing and style
- Final audio mix levels
Handoff template (handoff-notes.txt with source list, EDL, rough-cut path, remaining-for-human checklist): references/phase-commands.md -> Phase 6.
Gate: assembled/rough-cut.mp4 (or assembled/remotion-output.mp4) exists. handoff-notes.txt written to disk.
Video Transcript Extraction
Pull a video's transcript as readable paragraphs. Prefer uploader subtitles; fall back to auto-generated.
# Path 1: uploader subtitles (accurate)
yt-dlp --skip-download --write-subs --sub-langs en --sub-format vtt -o '<work-dir>/%(id)s' '<URL>'
# Path 2: auto-generated (fallback when path 1 writes no file)
yt-dlp --skip-download --write-auto-subs --sub-langs en --sub-format vtt -o '<work-dir>/%(id)s' '<URL>'
# Clean VTT to paragraphs
python3 ${CLAUDE_SKILL_DIR}/scripts/vtt_to_paragraph.py <work-dir>/<id>.en.vtt
Options: --timestamps for [mm:ss] markers, --keep-brackets for cue tags, -o FILE for file output. Other languages: change --sub-langs. List available: yt-dlp --list-subs '<URL>'.
Error Handling
| Error | Cause | Solution |
|---|---|---|
| "No such file or directory" on source | Path in cuts.txt doesn't match filename | Run cat source-inventory.txt. Quote paths with spaces. |
| FFmpeg "Invalid option" or codec errors | Codec unavailable or flag syntax error | ffmpeg -codecs | grep libx264. Fall back to -c:v copy. |
| Remotion "Could not find composition" | Composition ID mismatch | Check src/index.ts for registered ID. Match exactly in render command. |
| Segments concat in wrong order | Shell glob sorts alphabetically, not by EDL | Generate concat-list.txt from cuts.txt order, not from glob. |
| ElevenLabs 401 | ELEVENLABS_API_KEY not set | export ELEVENLABS_API_KEY=your_key before Phase 5. |
| No .vtt file from transcript extraction | Video has no subtitles in requested language | yt-dlp --list-subs '<URL>' and pick an available language. |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |