puntorigen/avatar-skills / video-scene-analysis
video-scene-analysis
VerifiedAnalyze local video files (mp4, mov, webm) into scene sequences: scene-change detection, scene type (talking head vs B-roll), split-screen / screen-composition detection (B-roll band + presenter, picture-in-picture, graphic overlays), B-roll kind (archival footage of a recognizable/known person vs generic material, naming the people), presenter background (real set/location vs animated/cartoons/motion graphics), zoom in/out vs previous scene, faster-whisper transcription with timecodes, SFX/music-bed detection per scene, a representative frame per scene, and per-scene focus/emotion summaries. Outputs .analysis.json, .analysis.md, and a frames folder. Use when the user asks to analyze a video, detect scenes, camera angle, split-screen layouts, archival/known-person footage, animated vs real backgrounds, transcribe a local video, detect SFX or background music, build a shot list from footage, or understand reel/avatar video structure.
| File | Kind | Size |
|---|---|---|
| REFERENCE.md | markdown | 5.6 KB |
| SKILL.md | skill md | 14.6 KB |
| scripts/_audio_events.py | script | 7.8 KB |
| scripts/_camera_vocabulary.py | script | 1.2 KB |
| scripts/_face.py | script | 4.4 KB |
| scripts/analyze_video.py | script | 33.3 KB |
| scripts/export_talking_profile.py | script | 4.1 KB |
| scripts/render_report.py | script | 1010 B |
| scripts/requirements.txt | manifest | 85 B |
| scripts/setup_models.shexecutable | script | 682 B |
REFERENCE.md
View on GitHub# Video Scene Analysis — Reference
## JSON top-level fields
```json
{
"version": 1,
"video_name": "clip.mp4",
"video_path": "/abs/path/clip.mp4",
"analyzed_at": "ISO-8601 UTC",
"video_info": {
"duration": 124.572,
"duration_fmt": "00:02:04.57",
"fps": 30.0,
"width": 480,
"height": 854,
"has_audio": true
},
"scene_detection": {
"mode": "detect | interval | interval_fallback",
"interval_sec": 6.0,
"min_scene_duration_sec": 2.5,
"adaptive_threshold": 3.0
},
"transcript": {
"language": "es",
"duration": 124.484,
"segments": [{ "start": 0.0, "end": 8.72, "text": "...", "words": [...] }],
"words": [{ "word": "hola", "start": 0.0, "end": 0.4 }]
},
"overview": "One-line summary string",
"avatar_profile": {
"mannerisms_summary": "Recurring facial behavior across talking-head scenes",
"video_prompt": "Concise p-video-avatar prompt (identity-consistent)",
"negative_prompt": "brief, comma-separated failure modes to avoid"
},
"frames_dir": "clip_frames",
"scenes": [ ... ]
}
```
`avatar_profile` is `null` after `analyze_video.py`; the agent synthesizes it
from the talking-head scenes' `mannerisms`. Export it to the avatar folder with
`export_talking_profile.py` → `<avatar>/talking_profile.json` (consumed by the
**avatar-talking-video** skill).
## Scene object
```json
{
"index": 0,
"start": 0.0,
"end": 2.97,
"duration": 2.97,
"scene_type": "main_character_solo",
"zoom_from_previous": {
"type": "none | zoom_in | zoom_out | hard_cut",
"confidence": 0.75,
"scale": 1.583
},
"visual": {
"face_count": 1,
"faces": [{ "confidence": 0.93, "area_ratio": 0.18, "bbox": [x, y, w, h] }],
"motion_score": 0.42,
"blur_score": 0.61,
"edge_ratio": 0.08,
"brightness": 0.55
},
"representative_frame": {
"file": "clip_frames/scene_01.jpg",
"timestamp": 1.485,
"timestamp_fmt": "00:00:01.48",
"sharpness": 842.5
},
"camera": {
"angle": "high_angle",
"framing": "medium_close_up",
"description": "Selfie vertical frontal, encuadre pecho arriba"
},
"layout": {
"type": "fullscreen | split_horizontal | split_vertical | pip | overlay_graphics",
"regions": [
{ "position": "top | bottom | left | right | inset",
"content": "broll | main_character | screen | graphics",
"description": "qué se ve en esa región" }
],
"hint": "fullscreen | possible_split_horizontal | possible_split_vertical",
"notes": "agent-written, optional"
},
"broll_kind": "archival_known_person | archival_footage | stock_generic | screen_recording | graphics_animation | other | null",
"known_people": ["Anthony Bourdain"],
"background": {
"type": "real_set | animated | mixed | plain | virtual | unknown",
"elements": "qué hay detrás del personaje (real o animado)",
"notes": "optional"
},
"transcript": "Text spoken during this scene window",
"audio": {
"audio_profile": "speech_with_sfx",
"has_speech": true,
"has_sfx": true,
"has_music_bed": false,
"non_speech_ratio": 0.42,
"sfx_event_count": 2,
"sfx_events": [{ "start": 12.4, "end": 12.65, "strength": 0.08 }],
"levels": { "full_rms": 0.05, "speech_rms": 0.04, "non_speech_rms": 0.03 }
},
"summary": {
"focus": "Motivation / purpose — written by the Cursor agent",
"emotion": "curiosidad"
},
"mannerisms": "Talking-head only: how the face/head moves (agent-written); null for B-roll"
}
```
`summary`, `camera`, `mannerisms`, `broll_kind`, `known_people`, `background`
(and top-level `avatar_profile`) are `null` right after `analyze_video.py`;
`layout` has only its script-written `hint` (`type`/`regions` are `null`/empty).
The **active session agent** must fill them before delivery by viewing the
frames (`mannerisms`/`avatar_profile`/`background` only when talking-head scenes
exist; `broll_kind`/`known_people` only on B-roll or split B-roll regions).
### Composition fields (agent-written)
- **`layout`** — screen composition. `type: "fullscreen"` for an ordinary single
shot; `split_horizontal`/`split_vertical` when ONE scene shows B-roll in one
band and the presenter in another; `pip` for a small inset; `overlay_graphics`
for graphics over the shot. `regions[]` lists each band's `position`+`content`.
The script pre-fills a conservative `hint` (seam + half-histogram heuristic).
- **`broll_kind`** — for B-roll / supplementary scenes (and a split's B-roll
region): `archival_known_person` (pre-recorded footage of a recognizable
person), `archival_footage`, `stock_generic`, `screen_recording`,
`graphics_animation`, `other`. Pairs with **`known_people`** (array of names /
short descriptions). These flag footage best sourced via the `broll-finder`
skill rather than synthesized.
- **`background`** — presenter scenes only: is the backdrop a `real_set` or
`animated` (drawings/cartoons/motion graphics), `mixed`, `plain`, `virtual`,
or `unknown`, with an `elements` description.
## Downstream use cases
- **Avatar reel pipeline**: map `main_character_solo` scenes to talking-head segments; `supplementary_material` to B-roll slots.
- **EDL / edit decision list**: use `start`/`end` + `zoom_from_previous` for Ken Burns presets (`push_in`, `push_out`, cut).
- **Script alignment**: join `transcript.segments` with scene windows for caption burn-in.
## Limitations
- Zoom detection compares **midpoint keyframes** between scenes; slow Ken Burns within one scene is not tracked.
- `hard_cut` means a strong visual discontinuity, not necessarily an editorial mistake.
- Whisper quality depends on model size and background noise.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |