0x0funky/agent-sprite-forge / skills/video2dsprite
video2dsprite
VerifiedTurn one approved master still into a character's whole animated sprite set - one image-to-video clip per action (idle, walk, run, attack, jump, hurt, cast and so on), all from that still, generated through generate2dmedia route_media.py (the xAI API when a key is configured, else the local Grok CLI) - then gate each take, key it with a soft matte, register it to the master, pick loops or retime actions, finish (HD by default, pixel on request) and package fixed-canvas frames, alpha WebM, packed MP4 for iPhone and a PNG fallback with engine metadata and a JS runtime. Also processes a supplied clip. Use for character and creature animation, fluid motion and any video-to-sprite work. Not for FX, icon or prop sheets (generate2dsprite), maps or plate loops (generate2dmap), code-drawn animation (codeart2d, only on request) or calling a video API by itself (generate2dmedia).
Video to 2D Animation
An approved still sets identity and geometry, a video model supplies motion, and deterministic tools prepare it for the engine. Video is not promised to be smaller, seamless or cheaper than sheets.
Use when
- Animating a character or creature: every action comes from one approved master still, one clip per action.
- Fluid motion from an accepted master: idle breathing, walks, runs, attacks, casts, hair, capes, creatures.
- A supplied clip must become transparent, registered, looped frames and engine video.
Do not use when
- FX, icon or prop sheets: generate2dsprite.
- Water, fire or mist inside a still scene plate: generate2dmap. Only an API or CLI call: generate2dmedia.
- Code-drawn animation the user asked for: codeart2d.
Capability check (once per session)
Run python "<skill-dir>/../generate2dmedia/scripts/forge_doctor.py" --host-tools <tools> --save <output>/doctor.json, where <tools> lists the media tools in your own tool list (image_gen, image_edit, image_to_video) or none. Its ROUTES block shows which API keys are configured (yes or no), the local readiness and the resolved order. If encoding.stdout fails, set PYTHONUTF8=1. python "<skill-dir>/scripts/video2dsprite.py" doctor checks ffmpeg (5.1+, VP9 alpha round trip).
Art source
The master still comes from generate2dsprite (image generation, approved by the user) or from the user. Motion is image-to-video, one clip per action from that master; record the route as art_source and name it in your reply:
- API, when a key is configured: xAI Grok Imagine Video (
XAI_API_KEY), BytePlus Seedance (ARK_API_KEY) or fal.ai (FAL_KEY: Kling v3, Veo 3.1, Luma Ray 3.2, MiniMax H3, Wan 3.0, Vidu Q3, LTX) (route-media.md). The configured key is the owner's consent, so there is no per-call question.--last-frame <still>pins idle and attack clips to end on their first pose. - Local: your own image-to-video tool (Grok's native
image_to_video) when you have one, otherwise the user's Grok CLI in ACP mode (it cannot pin a last frame; the result sayslastFrameUsed). - A clip the user supplies (no account needed).
- codeart2d only when the user explicitly asks for code-drawn animation, or when
route_media.pyprintsno-route(exit 3); otherwise prepare the job and prompt, explain the gap, and never invent a successful generation.
Every clip goes through one command: python "<skill-dir>/../generate2dmedia/scripts/route_media.py" video --prompt-file <job>/prompt.txt --reference <job>/input.png --last-frame <job>/input.png --duration 6 --resolution 720p --out-dir <new>. It prints one JSON line with route, artifact, sha256, estimateUsd and lastFrameUsed. The finish is HD by default; the pixel finish is an option when the user asks for pixel art.
A whole sprite set from one still
For a character's game set, use sprite_set.py: one approved master (generate2dsprite master.json) in, one image-to-video clip per action out, keyed, gated, registered, cut, finished and packaged. For one clip or a supplied video, use the pipeline below. Details: sprite-set.md. Prompts and fix clauses: motion-prompts.md.
- Plan.
python "<skill-dir>/scripts/sprite_set.py" plan --master <art>/master.json --output-dir <set>. Defaults:--actions idle,walk,run,attack,jump,hurt,--finishfrom the master (hd default; pixel or hibit),--target-height256 hd or 80 pixel and hibit. Jump gets a 3:4 canvas with 34% headroom and attack a 16:9 one; idle and attack pin the end frame to the master. Read each action'spromptin<set>/set_plan.json, and editmotion,negativesorgatesthere before the run if needed. - Run.
python "<skill-dir>/scripts/sprite_set.py" run --plan <set>/set_plan.json. Clips come from generate2dmediaroute_media.py, then:- soft-matte keying;
- numeric gates: area 0.72-1.32, feet, identity (a warning unless the design is lost), zoom, turning around, edge, background, extra objects, motion, end pose, colour bleed, one-shot timing;
- up to 3 takes, each retake carrying the fix clause for the gate that failed; if every take fails, the best usable window is kept;
- registration, then a loop, or a one-shot retimed to game length from its motion energy (attack 0.7 s with the hit at 40%, jump 0.9 s, hurt 0.5 s, cast 0.9 s; set in the plan's
timing); finish_frames.pyhd, pixel or hibit, with the colour lock to the master on by default for hd and pixel (plan --no-colour-lock; hibit never locks);- png, webm and packed packages, verified. Run it again to resume; nothing is generated twice.
- No route (exit 3). Make the clip with your own image-to-video tool from the input and prompt that the summary names, save it as the named
media/clip.mp4, and run again. - Review.
python "<skill-dir>/scripts/sprite_set.py" review --plan <set>/set_plan.json, then LOOK at every-sheet.png, every-final.pngand thelineup(Read in Claude Code, view_image in Codex). The numbers cannot see identity, facing, a changed weapon, text or a covered face. - Decide each action.
- Approve:
accept --plan <set>/set_plan.json --action <id> --take <n>. Add--window START:ENDto keep part of a failed take. The verdict goes into the package: an already packaged take is updated in place (engine_export.py review:animation.jsonreviewStatusbecomesaccepted), otherwise the nextrunpackages it with--review.review,acceptandretakemay run while arunworks: every verb reads and writesset_state.jsonunderset_state.lock(so doesforge_index.py scan) and the run keeps their decisions. - Reject:
retake --plan <set>/set_plan.json --action <id> --fix <clause id or text>, thenrunagain. Clause ids include never-turn, weapon-shape, face-visible, no-text, no-flares and keep-colours.
- Approve:
- Report.
python "<skill-dir>/scripts/sprite_set.py" report --plan <set>/set_plan.jsonlists the routes, takes, QC numbers, loops, timing and package paths. In your reply, name the route that made each clip. Never call an action approved before you have looked at its sheet.
Host notes
- Codex: no native image-to-video tool;
route_media.pytakes the xAI API or the Grok CLI. Look at review images withview_image. - Claude Code: no media tools;
route_media.pytakes the route. Look at review PNGs and contact sheets with Read; run tools with Bash;<skill-dir>is${CLAUDE_SKILL_DIR}. - Grok: its native
image_to_videotool is your own tool; the samegrokexecutable is the "Grok (local CLI)" route for other hosts.
Commands
Run each tool as one line from the user's project root: python "<skill-dir>/scripts/<tool>.py" .... <skill-dir> is this skill's folder; sibling skills sit beside it. Keep inputs and outputs inside the project. Every verb writes a new --output-dir: it refuses an existing one and publishes only after its checks (--strict fails on residue). Success prints one JSON line; errors print error: ... and exit 1 (nothing usable is published); usage errors exit 2; a result published for inspection with QA status fail exits 4 (sprite_set.py run: some actions packaged, others failed). sprite_set.py run, finish_frames.py hd, pixel and hibit, and engine_export.py package and verify take --progress-file <jsonl> (or FORGE_PROGRESS_FILE): one JSON line per stage for a host, never on stdout. Needs Python 3.10+, numpy, Pillow and ffmpeg 5.1+. --help lists every flag.
Pipeline (one clip)
- Key plan.
video2dsprite.py key-plan --master <master.png>: pastebackground_sentenceinto the prompt; protect or recolourdesign_colours_at_risk. - Prepare.
prepare_i2v_input.py prepare --master <art> --action <idle|walk|run|attack|cast|guard|hurt|victory|defeat|ambient|fx> --output-dir <job>; sendinput.pngandprompt.txt;registration_job.jsonkeeps the transform. An opaque master on a flat magenta, green or blue backdrop is keyed first (--master-keynames another backdrop; the job recordsmasterKeying).--subjectis the whole identity phrase; the background line names its head ("adventurer with a scarf" gives "behind the adventurer"). Check a hand-written prompt withprepare_i2v_input.py lint. - Generate with
route_media.py video(Art source above). - Accept the take.
register_clip.py qc --job <job>/registration_job.json --video <take>; thenvideo2dsprite.py triage --video <take> --output-dir <new>. A border touch inside the action means regenerate, never pad. - Key.
video2dsprite.py process --video <take> --output-dir <new> --reference <master.png>(soft matte); readframes-clean/matte-report.jsonand look at frames over light and dark. Same keying for every clip of a character:--matte-profile <character-profile.json>. A design colour near the key:--protect-color #rrggbb. Already decoded frames (frame_*.pngorraw_*.png, one frame keys a still):video2dsprite.py clean --raw-dir <dir> --output-dir <new>. - Register.
register_clip.py apply --job <job>/registration_job.json --frames <out>/frames-clean --output-dir <new>; feet drift or jumps:--lock feet; spell or hit FX:--profile fx; one scale per character:register_clip.py profile, then--character-profile. - Loop or retime. Walks, runs, idles:
gait_loop.py select --frames-dir <reg>/frames --fps N --output-dir <new>(--kind idleorhover); playaids/loop3x.gif. Attacks, casts, FX:retime.py --frames-dir <reg>/frames --fps N --output-dir <new> --spans <spans>(impact, hold, ticks), or for a slow-motion one-shot--range 0:N --auto-oneshot --duration 700 --key hit=<impact>@0.4+80(cuts frozen holds, pins the hit, compresses to game length); walk cadence:gait_loop.py measure-stride. - Finish and package. Finish the registered frames (Finish, below), then
engine_export.py package --clean-dir <fin>/frames --output-dir <new> --selection <selection.json> --source-size W,H --source-anchor X,Y --formats png,webm,packed --tiers actor(W,H and X,Y fromfinish.json). Refused for key residue: re-key;--allow-key-residueonly with a recorded override. - Verify.
engine_export.py verify --package <dir>, thenvalidate_animation.py <character dir> --require-states idle,walk --require-verify.--require-statesmatches each package's--name(a package namedhero-runneeds--require-states hero-run, notrun).
Details: pipeline.md, keying matte.md, review and cuts animation-review.md, prompts prompt-rules.md.
Finish (HD default, pixel option)
Finish every registered action before packaging. Finished frames keep the source names, so gait_loop and retime selections still apply. Details: finishing.md.
- HD (default):
python "<skill-dir>/scripts/finish_frames.py" hd --frames <reg> --output-dir <new> --target-height <px> --role hero --character <id>. The rest pose (frame 0 or--rest-frame) sets one scale. Box downscale with premultiplied alpha on a grid pinned to the feet, then clean alpha. Never nearest, never an upscale.--display-sizes 1x,2xrebuilds each size from the source. - Same character, other actions:
--scale-ref <idle finish dir>instead of--target-height(scale x sqrt(rest-area ratio), clamped to +-3%). A warning means the rest frame is not the base still. - Pixel (on request): first
finish_frames.py palette build --frames <every action's reg dir> --colors 32 [--reserve #hex] --out <palette.json>, then per actionfinish_frames.py pixel ... --target-height <1/8 to 1/12 of the source> --palette <palette.json>. The default is bold pixel art: a crisp 4x4-cluster downscale (no blurred in-between colours; lines and eyes survive), a light contrast and saturation lift, at most--colors(32) colours per frame, lone-pixel cleanup that keeps eyes, a 1 px selective outline (--outline dark|none), temporal hysteresis, binary alpha, no dither.--canvas 48x64gives fixed cells. Use--loop-policy oneshotfor attacks and--indexed-sheetfor an indexed sheet. - Hibit (on request, the hi-bit look): the master is flat lineless art (
master_still.py --finish hibit). Per actionfinish_frames.py hibit --frames <reg> --output-dir <new> --target-height 80: one character palette learned from the source pixels (--colors16), every output pixel the palette colour covering most of its 8x8 sub-pixel cell (a majority vote, never a blend;--coverage0.45), hysteresis, lone-pixel cleanup that keeps eyes, binary alpha and a 1 px outline whose colour comes from the sprite (--outline auto, or#RRGGBB,selective,dark,none). The next action reuses the first one'spalette.jsonwith--palette(same colours, same outline). No colour lock.sprite_set.py plan --finish hibitdoes all of this (80 px, 12 fps). Plates and pixelfx FX for the same game: generate2dspritereferences/hibit.md. - Pixel looks (only on request): a 1x sprite made by generate2dsprite
pixel_look.pydoes not go through register_clip or finish_frames. Idle:python "<skill-dir>/scripts/pixel_motion.py" breath --sprite <sprite.png> --name <clip> --output-dir <new>(no clip: the sprite and the sprite with its upper part one cell lower). Other actions:pixel_motion.py prepare --sprite <sprite.png> --action attack --facing right --subject "..." --extra "..." --output-dir <job>(the sprite enlarged by a whole number with its feet as the root, a prompt that tells the model to keep the pixels), the clip from<job>/prompt.txtand<job>/input.pngthrough route_media.py,pixel_motion.py frames --clip <mp4> --job <job> --output-dir <frames>(every frame back on the sprite's own grid and palette; look atframes-sheet.png) andpixel_motion.py finish --frames-dir <frames> --name <clip> --output-dir <new>(one cycle, an action or named frames; the head lock; ananimation.jsonpackage with nearest sampling). A walk is made from a side view and read first:pixel_motion.py measure --frames-dir <frames> --output-dir <new>gives the footfalls, the step, the ground speed and the drawings to take, andlegs.pngto look at. The whole route: generate2dspritereferences/pixel-looks.md. - Colour lock:
--colour-lock <keyed master.png>keeps every design colour on the master's in OKLab (chroma only, lightness kept: no posterising) and smooths still pixels over time;colour_lock.py measurereports hue flips and region spread. - Cast check:
finish_frames.py lineup --sets <finished dirs> --out <lineup.png>. Roles come from--roleorROLE=DIR; the default rules are hero:1.0, mob:<=1.0, boss:~2.0, spirit:~1.0 (by area). Look at the x1/x3 sheet; a failed rule writes its report and exits 4. - Review: look at
review-contact.pngand readqain the JSON line (flipsPerPair,specksRemoved,alphaBinary). The status staysneeds-visual-review. - Package:
engine_export.py package --clean-dir <fin>/frames --source-size W,H --source-anchor X,Y, taking W,H and X,Y fromsizeandanchorinfinish.json. Do not pass--registration; pixel and hibit finishes add--pixel-art --sampling nearest. A review made after packaging:engine_export.py review --package <dir> --review <verdict.json>(no re-encode).
Motion and geometry rules
- One clip, one action. The game owns translation, hit timing, damage and state changes.
- One fixed transform per clip; never crop or resize frames to their own alpha bounds (it pumps size and pins airborne feet). Never freeze the still's alpha over moving RGB.
- All actions of one character share one body height, taken from the master's rest pose; never upscale a finished sprite.
- Choose cycles with
gait_loop.py select, never by eye alone, then confirm by eye. Walks never ping-pong; attacks never recover by playing frames backwards; retime actions to their impact and hold. Walks ship cadence and stride. - Keying claims quote the matte report's numbers and stay
needs-visual-review.--matte binary --despill-mode offreproduces the cfed170 keyer.
Runtime and acceptance
- Acceptance order: package, then verify, then
validate_animation.py. Deliveranimation.json(3.0, keeps every 2.0 key; whole-msdurationsMs, exactfpsRational), QA, poster, PNG fallback and the requested transports. Geometry stays in source units (sourceSize,sourceAnchor,sourceRect). - Packed H.264 stores RGB left and alpha right; it is not a transparent MP4. Draw the compositor's
lease.drawable(packed-alpha-runtime.js overreferences/runtime/packed-alpha-webgl.mjs); show the poster until a decoded frame exists. Distance-driven walks, hit-stop and transitions:references/runtime/forge-runtime.mjs. - Inspect at game scale: face and silhouette stability, contact, moving alpha edges over light and dark, loop seam, readability. Verify real iPhone playback before claiming alpha or frame rate; compare bytes and decode cost before replacing sheets.
Data contracts: references/schemas/video.schema.json.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementation | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirements | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |