Skip to content

puntorigen/avatar-skills / broll-actor-copy

broll-actor-copy

Verified

Generate a B-roll clip where OUR avatar COPIES the motion of a driving video — same movement, gestures, expressions and lip movements — via ByteDance DreamActor M2.0 (bytedance/dreamactor-m2.0 on Replicate). Takes an avatar, a location/look (or its default) and a driving video, resolves the look's hero image, and transfers the video's performance onto it. Works with humans, cartoons and animals. The driving video must show exactly ONE animated character; multi-character sources are split into single-character segments and stitched. A human driver should match a human avatar's gender (else warn + ask: the face distorts) and the reference scene should include the objects the driver interacts with. Output keeps the reference image's resolution (9:16 hero → 9:16 clip); the clip is muted and avatar-reel-composer re-lays narration. Use when the user wants the avatar to mimic a video's movement, a motion copy / performance transfer / actor copy / "copiar el movimiento del video" B-roll, or mentions dreamactor.

FileKindSize
REFERENCE.mdmarkdown8.9 KB
SKILL.mdskill md12.3 KB
scripts/_common.pyscript19.0 KB
scripts/make_actor_copy.pyscript18.7 KB
scripts/requirements.txtmanifest239 B
scripts/setup_key.pyscript1.6 KB

REFERENCE.md

View on GitHub
# B-roll Actor Copy — reference

Schema, resolution rules, look resolution and examples for the
`bytedance/dreamactor-m2.0` motion-transfer path.

## bytedance/dreamactor-m2.0 — input schema (verified)

DreamActor M2.0 animates a character from ONE reference image by copying the
motion of a driving video. It learns motion from raw pixels (no human skeleton
extraction), so humans, cartoons, animals and stylized avatars all work.

| Input | Type | Default | Notes |
|---|---|---|---|
| `image` | string(file) | — | **required** — reference image of the subject (the avatar's look hero). JPEG/JPG/PNG, max **4.7 MB**, resolution **480×480 … 1920×1080**. **The output keeps this image's resolution.** |
| `video` | string(file) | — | **required** — driving/template video whose **motion, facial expressions and lip movements** are copied onto the subject. **Exactly ONE animated character** (person or animal) must be visible. MP4/MOV/WebM, **max 30 s**, resolution **200×200 … 2048×1440**. |
| `cut_first_second` | bool | `true` | Crop the first second of the output (removes the model's ~1 s lead-in transition). The script defaults to cutting it; `--keep-first-second` keeps it. |

**Output:** a single video URI. **No `seed`, no `resolution`, no prompt** — the
appearance is fully set by `image` and the motion fully by `video`.

## Single-subject rule (important)

DreamActor copies **one** subject. The driving video must show **exactly one
animated character** — a person OR an animal — visible in scene. Our reference is
a single-avatar hero, so it must be paired with a one-character driver.

If the source video has more than one character:
1. **Inform** the user that the driving video must have a single animated
   character to copy (model limitation).
2. **Ask** the user (AskQuestion) to either (a) supply a single-character clip, or
   (b) split into single-character time segments and stitch them.

For (b), pass the ranges to `--segments` (see below). To find where each character
is alone on screen, inspect the video visually or with the `video-scene-analysis`
skill.

## Driver gender rule (human ↔ human)

DreamActor also copies the driver's **facial gestures and feature placement**.
With a human driver and a human avatar of **different genders**, the avatar's
face distorts / drifts off-identity (observed: female driver → male avatar).

- Human driver + human avatar → **same apparent gender**, or **inform the user
  and ask (AskQuestion)** whether to continue anyway or swap the driving clip.
- Human driver + **animal** avatar → fine, no gender constraint (the
  human-to-human face matching is what breaks).

## Reference scene & framing rules

- **Interactable elements.** If the driving actor interacts with objects (bench,
  mat, bar, chair, wall…), the reference image's scene must include a **coherent
  counterpart** the avatar could use, roughly where the motion needs it. It does
  NOT have to look like the driver's prop — just fit the avatar's location. A
  mismatched scene makes DreamActor hallucinate props mid-clip and contact points
  (hands/feet on surfaces) look wrong.
- **Half-body reference for face-critical work.** DreamActor invents whatever
  lies outside the reference frame to complete the motion. Since face + torso
  carry the identity, frame the reference **mid-thigh up** (with a hint of the
  lower clothing so the model infers the rest). A far full-body reference makes
  the face tiny → mushy/distorted output; details invented below frame (pants,
  shoes) matter much less than a faithful face.

## Segment → process → stitch (multi-character sources)

`--segments "start-end,start-end,…"` takes single-character time ranges (seconds,
`M:SS`, or `H:MM:SS`). Each range is:
- trimmed from the source with ffmpeg (respecting the 30 s cap),
- animated separately through DreamActor with the **same** avatar hero,
- saved as a muted intermediate under `_work/`,
then all intermediates are **concatenated** (ffmpeg concat filter, resolution is
identical because they share the hero) into one muted `<NNN>_<slug>.mp4`.

Rules: each range ≤ 30 s and must contain a single character; `--segments` is
mutually exclusive with `--trim-start`/`--trim-duration`; the stitched output is
always muted (`--keep-audio` is ignored in this mode). The manifest entry records
`mode: "segments"` and a per-segment list (`start`, `duration`, `source_url`, …).

### Auto-fit (make_actor_copy.py, only when needed)
- **Reference image**: converted/resized only if it violates a constraint —
  upscaled if the short side < 480, downscaled to fit the 1920×1080 envelope
  (either orientation), and re-encoded to JPEG if > 4.7 MB. A compliant hero is
  **passed through untouched**, so its resolution flows straight to the clip.
  `--no-fit-image` disables this.
- **Driving video**: trimmed to ≤ 30 s (or the `--trim-start`/`--trim-duration`
  segment) and downscaled into the 2048×1440 box only if larger; otherwise sent
  as-is.

## Look → hero image resolution

Mirrors `avatar-location`'s layout. The avatar folder `<avatar>` is a bare name
routed to `avatares/<name>` (or an explicit path); slug = the folder name:

| `--location` | Hero used (first that exists) |
|---|---|
| `default` (or omitted) | `refs/<slug>_hero.png` → `refs/<slug>_hero_master.png` → a `angles/*_916.png` → `frames/frame_0001.png` |
| `<loc>` | `locations/<loc>/refs/<slug>__<loc>_hero.png` → `…_hero_master.png` → `locations/<loc>/angles/*_916.png` |

`--image PATH` bypasses this entirely (use any image; its resolution sets the
output resolution). The default look = the avatar's top-level `scene.json` +
`refs/` and is never modified.

## Output & manifest

Clips land in `<avatar>/broll/actor-copy/`:

| File | What it is |
|---|---|
| `<NNN>_<slug>.mp4` | The muted motion-copied clip (auto-numbered) |
| `manifest.json` | `items[]` mapping each clip → `avatar`, `location`, `reference_image`, `driving_video`, `mode` (`single`/`segments`), `segments[]`, `trim_start`, `trim_duration`, `cut_first_second`, `width`/`height`, `duration_actual`, `source_url`, … |
| `_work/` | Prepared (fitted image / trimmed video) intermediates |

## Worked examples

A bare avatar name routes under `./avatares/<name>` (override with `AVATARES_ROOT`);
an explicit path is used as-is.

```bash
# Default look — copy a dance video onto the avatar (9:16 hero -> 9:16 clip)
python3 broll-actor-copy/scripts/make_actor_copy.py nora \
  --video downloads/dance_take.mp4 --slug nora-copies-dance

# A specific location/look
python3 broll-actor-copy/scripts/make_actor_copy.py nora \
  --location studio_night --video downloads/host_moves.mp4

# Use only an 8s segment starting at 12s of a long driver
python3 broll-actor-copy/scripts/make_actor_copy.py nora \
  --video downloads/long_take.mp4 --trim-start 12 --trim-duration 8

# Multi-character source -> single-character segments, processed + stitched
python3 broll-actor-copy/scripts/make_actor_copy.py nora \
  --video downloads/interview.mp4 --segments "0-8,15-22,0:40-0:52" --slug nora-copies-host

# Explicit reference image (skip look resolution), keep the generated audio for QA
python3 broll-actor-copy/scripts/make_actor_copy.py \
  --image avatares/nora/refs/nora_hero.png --video downloads/clip.mp4 --keep-audio

# Preview the plan without spending (prints resolved hero + prepared sizes)
python3 broll-actor-copy/scripts/make_actor_copy.py nora \
  --video downloads/clip.mp4 --dry-run
```

## Recipes / lessons

- **9:16 in, 9:16 out.** The output resolution equals the reference image's, so
  crop the hero to the reel ratio first (or pass a 9:16 `--image`).
- **The driver IS the direction.** There is no text motion prompt — pick a
  driving clip that already performs the movement/expression you want.
- **Clean inputs → clean output.** A clear reference subject and a clean,
  well-lit driving clip give the best identity + motion fidelity. Portrait and
  full-body both work; the model adapts scale.
- **One character only** in the driving video (see the single-subject rule
  above); for multi-character sources, segment + stitch with `--segments`.
- **Match driver gender (human→human).** A female driver on a male avatar (or
  vice versa) distorts the face — warn + ask before proceeding. Human→animal is
  fine.
- **Scene must afford the motion.** Include the driver's interaction objects
  (bench/mat/bar…) as coherent counterparts in the reference scene, and prefer a
  half-body (mid-thigh up) reference when the face/torso are what matter — the
  model fills in the rest below frame.
- **Muted broll.** The clip is muted (ffmpeg `-an`); `avatar-reel-composer`
  re-lays the single master narration. Use `--keep-audio` only for a standalone
  lip-sync/motion QA preview.
- **When to use a sibling instead:** for an action described in words (no driving
  video) use `broll-avatar-camera` (start frame + `video_prompt`); for a stylized
  illustrated vignette use `broll-story` (seedance-2); for a precise start→end
  object move use seedance-2 start+end frames.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 196kUpdated Claude CodeCodexVerifiedInstall