Skip to content

oliverv/caro / .agents/skills/oma-video

oma-video

Unverified

Create short, explainer, or recorded-demo videos through the OMA video CLI. Use for scripts, narration, assets, composition, and video delivery.

resources/execution-protocol.md

View on GitHub
# Video Agent - Execution Protocol

## Step -1: Clarify / Infer Mode (agent-side, before `oma video generate`)

Run the **Clarification Protocol** in `SKILL.md` before shelling out. Infer the
mode from keywords (shorts/reels/쇼츠/릴스 -> shorts; README/code/data/explain/설명
-> explainer; demo/walkthrough/capture/데모 -> demo) and show the user the inferred
plan when the brief is a one-liner.

## Step 0: Parse Request

1. Extract the brief and flags from the invocation.
2. Resolve defaults from shipped code defaults -> the `video:` section of `.agents/oma-config.yaml` -> env vars -> CLI flags (lowest to highest precedence). `config/video-config.yaml` is not consulted.
3. Validate:
   - `mode` ∈ {`shorts`, `explainer`, `demo`}.
   - `aspect` ∈ {`9:16`, `16:9`, `1:1`, `auto`} (`auto` snaps to the mode default: shorts -> 9:16, explainer/demo -> 16:9).
   - `captions` ∈ {`tiktok`, `lower-third`, `none`}; `visual` ∈ {`auto`, `generate`, `stock`, `aigc`, `slide`}.
   - `music` ∈ {`upbeat`, `calm`, `cinematic`, `lofi`, `piano`, `none`}; `compositor` ∈ {`hyperframes`, `mpt`}.
   - `duration` ≤ `limits.max_duration_sec` (180); resulting `scenes` ≤ `limits.max_scenes` (40).
   - `out` is inside `$PWD` unless `--allow-external-out`.
   - For `demo`: `--capture` (if given) exists, is absolute + `$PWD`-guarded, and is a valid video format.
4. If invalid: exit code 4 with a message identifying the offending field.

## Step 0.5: Mode Routing

- **shorts** (9:16): synthetic from the topic. Default visual = oma-image stills + Ken Burns.
- **explainer** (16:9 / 9:16): from README / code / data. Default visual = oma-slide frames + oma-image diagrams.
- **demo** (16:9): from a screen capture. If no `--capture` and Cap is unavailable -> emit the **guided capture protocol** (Step 4b) and stop.

## Step 1: Provider Availability + Selection

1. Call `available()` on every registered provider in parallel (`oma video provider list`).
2. For each capability, walk `providers.<capability>.order`:
   - The first available provider wins.
   - Paid providers (`pexels`, `pixelle`) are skipped unless their env key is present (`enabled` gate).
   - On chain exhaustion for a required capability -> exit 5 naming the capability.
3. Log `using: <provider>` per capability to stderr before generation.

## Step 2: Cost Guardrail

1. Estimate cost = sum of each selected provider's `estimateCost()` (most are `$0`; Pixelle/RunningHub credits are non-zero).
2. If `--dry-run`: emit `script.json` / `render-spec.json` / `manifest.json`, skip rendering, exit 0.
3. If estimate ≥ `cost.guardrail_usd` (or `--max-usd`) and not `--yes` / `OMA_VIDEO_YES=1`:
   - Prompt on stderr: `Estimated cost $X.XX. Proceed? (y/N)`. Decline -> exit 1.

## Step 3: Cancellation Setup

1. Install `SIGINT` / `SIGTERM` handlers that call `AbortController.abort()`.
2. Thread the signal into every provider call and into the render subprocess.

## Step 4: Pipeline (asset bus)

```
brief ─► [ScriptProvider] ─► script.json {scenes[], narration[], onScreenText[]}
script.json ─► parallel:
   ├► [VoiceProvider]   ─► audio/narration-01.wav (single joined track) + timing.json
   ├► [VisualProvider]  ─► visuals/scene-NN.*
   └► [CaptionProvider] ─► captions.srt / captions.vtt
all assets ─► render-spec.json ─► [Compositor: HyperFrames] ─► <mode>-<slug>.mp4
```

1. **Script**: AgentScriptProvider writes `script.json` (start of the determinism boundary).
2. **Voice**: oma-voice synthesizes narration -> a **single** `audio/narration-01.wav` (all scene lines joined into one track; per-line offsets live in `timing.json`). Fallback: estimated timing (no wav).
3. **Visuals**: walk the visual chain. oma-image stills (key-free default) / oma-slide frames (explainer) / Pexels (key) / Pixelle (key). Aspect -> 16-multiple size; HyperFrames crops to the exact frame.
4. **Captions**: oma-captions builds `captions.srt` + `captions.vtt` from `timing.json`. For a non-source locale, translate via oma-translation (key-free); absent -> warn + keep source.
5. **render-spec**: compose `render-spec.json` (the deterministic compute boundary) from the assets + seed.
6. **Render**: the compositor consumes `render-spec.json`.

### Step 4b: Guided Capture Protocol (demo, no capture)

State plainly to the user: **"Demo capture is performed by a human."** Then:
1. Instruct: record the walkthrough with Cap (or any recorder) at 16:9.
2. Ask the user to re-run with `--capture <absolute-path>.mp4`.
3. Stop without rendering (capture-required is a guided stop, not a hard error).

## Step 5: Compositor Render

<!-- oma-docs:ignore-start -->
- **HyperFrames (default, live)**: `generate` emits the asset bus and a pending HTML project at `<runDir>/hyperframes/`. Read its `AUTHORING.md`, the listed upstream API skills, and `resources/hyperframes-authoring/<mode>.md`. Author `index.html` with local assets and seekable animation. Run `oma video render <runDir> --output json`: lint → strict render → ffprobe stream/dimensions/duration checks → manifest update. Fix reported failures and re-render.
<!-- oma-docs:ignore-end -->
- **Failure**: a missing toolchain, render failure, missing video stream, or non-positive duration is a failure with diagnostics. Keep the run directory and recovery artifacts; do not write a placeholder MP4. `OMA_VIDEO_MOCK=1` permits deterministic placeholders for tests only.
- **MPT (`--compositor mpt`)**: inject the agent-written script (custom-script mode); keys env-only + log masking. It requires the installed checkout, venv, and ffmpeg; setup or render failures fail with diagnostics.

## Step 6: Write Artifacts

1. Copy every external asset into the run dir and hash it (`sha256`); no URL refs.
2. Validate each asset-bus schema (`script` / `timing` / `render-spec` / `manifest`) — `schemaVersion` must be `"1.0"`.
3. Build `manifest.json`: `runId`, `mode`, `providers{...}`, `assets[{path,sha256,bytes,seed}]`, `outputs{video,durationSec,sha256}`, `cost{usd,breakdown}`, `warnings[]`, `exitCode`.
4. If `--no-brief-in-manifest`: replace `prompt` with `promptSha256`.

## Step 7: Report

1. Print a one-line status per capability to stderr:
   - `[oma video] <capability> <provider> ok (Xs)`
   - `[oma video] <capability> <provider> fallback -> <fallback>`
2. Print the run-dir path and, only on success, the validated mp4 path.
3. For `--format json`: write `{exitCode, runDir, manifestPath, scriptPath, renderSpecPath, warnings, error}` to stdout as one JSON object (no `outputs` key — read output/asset paths from the manifest at `manifestPath`).

## Step 8: Exit Code Aggregation (aligned with `oma search fetch`)

- Success (playable mp4 with a video stream and positive duration + valid manifest) -> exit 0. Asset-provider fallbacks are recorded in `warnings`.
- Otherwise pick the most specific code:
  - `safety-refused` -> 2
  - `not-found` (profile/asset) -> 3
  - `schema-validation` / invalid input -> 4
  - `provider-unavailable` / `auth-required` -> 5
  - `timeout` -> 6
  - otherwise -> 1

## On Error

| Situation | Action |
|-----------|--------|
| No provider for a required capability | Exit 5, print `Run: oma video doctor` |
| HyperFrames or MPT toolchain not bootstrapped | Exit 1 + `oma video doctor` remediation; retain render-spec and authored composition for recovery |
| Voicebox MCP down | Fall back to estimated timing; still emit captions (whisper.cpp hop deferred: `TODO(oma-deferred): whisper-cpp`) |
| Pexels / Pixelle key absent | Skip provider; fall through to oma-image stills; annotate coverage in `warnings` |
| `demo` with no capture + no Cap | Guided protocol (Step 4b); stop without rendering |
| `--capture` outside `$PWD` or wrong format | Exit 4 with the path/format problem |
| Cost over guardrail, declined | Exit 1 |
| Timeout | Exit 6, manifest records `after_ms` |
| Cancelled (Ctrl+C) | Exit 130 (signal); no manifest if abort was pre-write |
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall