Skip to content

norahe0304-art/30x-video / cli/skill

30x-web-to-video

Flagged

Use when a user gives a brand URL and wants a premium launch, demo, or marketing video. Zero-manual workflow — Claude runs the V2 orchestrator itself (scrapes brand evidence, auto-installs missing tools, generates the first-cut Remotion project). User only sees the URL and the Studio preview.

FileKindSize
.gitignoreother103 B
AGENTS.mdmarkdown2.5 KB
FUSION-REPORT.mdmarkdown12.3 KB
README.mdmarkdown3.6 KB
SKILL.mdskill md30.7 KB
benchmarks/AGENTS.mdmarkdown773 B
benchmarks/README.mdmarkdown2.8 KB
benchmarks/manifest.jsonconfig5.9 KB
examples/INDEX.mdmarkdown1.8 KB
examples/caylent/MainVideo.tsxother6.0 KB
examples/caylent/scenes-v2/Act1V2.tsxother1.9 KB
examples/caylent/scenes-v2/Act2V2.tsxother2.5 KB
examples/caylent/scenes-v2/Act3V2.tsxother11.3 KB
examples/caylent/scenes-v2/Act4V2.tsxother8.6 KB
examples/caylent/scenes-v2/Act5V2.tsxother1.0 KB
examples/caylent/scenes/Act1.tsxother6.0 KB
examples/caylent/scenes/Act2.tsxother2.5 KB
examples/caylent/scenes/Act3.tsxother7.4 KB
examples/caylent/scenes/Act4.tsxother7.1 KB
examples/caylent/scenes/Act5.tsxother2.2 KB
examples/caylent/theme.tsscript1.2 KB
examples/cofounder/MainVideo.tsxother5.5 KB
examples/cofounder/scenes/AGENTS.mdmarkdown1.4 KB
examples/cofounder/scenes/Act1.tsxother2.3 KB
examples/cofounder/scenes/Act2.tsxother2.4 KB
examples/cofounder/scenes/Act3a.tsxother6.7 KB
examples/cofounder/scenes/Act3b.tsxother8.5 KB
examples/cofounder/scenes/Act3c.tsxother6.8 KB
examples/cofounder/scenes/Act4.tsxother4.4 KB
examples/cofounder/scenes/Act5.tsxother3.3 KB
examples/cofounder/scenes/World.tsxother4.0 KB
examples/cofounder/theme.tsscript1.5 KB
examples/happy-model/MainVideo.tsxother6.8 KB
examples/happy-model/scenes/AGENTS.mdmarkdown1.7 KB
examples/happy-model/scenes/Act1.tsxother7.9 KB
examples/happy-model/scenes/Act2.tsxother10.2 KB
examples/happy-model/scenes/Act3.tsxother15.6 KB
examples/happy-model/scenes/Act4.tsxother16.9 KB
examples/happy-model/scenes/Act5.tsxother9.0 KB
examples/happy-model/scenes/copy.tsscript588 B
examples/happy-model/theme.tsscript1.2 KB
examples/mana/MainVideo.tsxother4.9 KB
examples/mana/scenes/AGENTS.mdmarkdown926 B
examples/mana/scenes/Act1Open.tsxother2.8 KB
examples/mana/scenes/Act2Hero.tsxother3.3 KB
examples/mana/scenes/Act3Flavors.tsxother9.0 KB
examples/mana/scenes/Act4Label.tsxother7.3 KB
examples/mana/scenes/Act5Close.tsxother4.7 KB
examples/mana/scenes/World.tsxother7.2 KB
examples/mana/theme.tsscript2.6 KB
examples/parker/MainVideo.tsxother6.2 KB
examples/parker/scenes/AGENTS.mdmarkdown1.1 KB
examples/parker/scenes/Act1Hook.tsxother3.2 KB
examples/parker/scenes/Act2Meet.tsxother2.3 KB
examples/parker/scenes/Act2Problem.tsxother3.6 KB
examples/parker/scenes/Act3Steps.tsxother4.7 KB
examples/parker/scenes/Act4Proof.tsxother3.2 KB
examples/parker/scenes/Act5Close.tsxother2.6 KB
examples/parker/scenes/PaperWorld.tsxother6.8 KB
examples/parker/theme.tsscript1.6 KB
examples/perplexity-pc/MainVideo.tsxother4.5 KB
examples/perplexity-pc/scenes/Act1.tsxother2.1 KB
examples/perplexity-pc/scenes/Act2.tsxother2.7 KB
examples/perplexity-pc/scenes/Act3.tsxother3.1 KB
examples/perplexity-pc/scenes/Act4.tsxother1.6 KB
examples/perplexity-pc/scenes/Act5.tsxother2.0 KB
examples/perplexity-pc/theme.tsscript2.5 KB
references/AGENTS.mdmarkdown767 B
references/animations.mdmarkdown15.1 KB
references/audio.mdmarkdown8.6 KB
references/components.mdmarkdown9.5 KB
references/lottie.mdmarkdown5.1 KB
references/visual-effects.mdmarkdown5.0 KB
rules/AGENTS.mdmarkdown3.0 KB
rules/archetypes.mdmarkdown6.0 KB
rules/artifact-catalog.mdmarkdown10.8 KB
rules/beat-sync.mdmarkdown5.7 KB
rules/cards.mdmarkdown2.5 KB
rules/cinematic.mdmarkdown6.7 KB
rules/color.mdmarkdown2.2 KB
rules/composition.mdmarkdown13.2 KB
rules/creative-moves.mdmarkdown3.8 KB
rules/data-viz.mdmarkdown1.9 KB
rules/finish-gate.mdmarkdown5.2 KB
rules/layout.mdmarkdown6.6 KB
rules/media-resolve.mdmarkdown7.1 KB
rules/motion.mdmarkdown9.5 KB
rules/narration-sync.mdmarkdown5.5 KB
rules/narrative-templates.mdmarkdown10.4 KB
rules/narrative.mdmarkdown7.3 KB
rules/qc-gates.mdmarkdown11.3 KB
rules/taste.mdmarkdown12.6 KB
rules/transitions.mdmarkdown2.7 KB
rules/typography.mdmarkdown4.1 KB
rules/ui-mockups.mdmarkdown5.6 KB
rules/workflow.mdmarkdown53.2 KB
scaffold/.prettierrcother66 B
scaffold/AGENTS.mdmarkdown1.1 KB
scaffold/eslint.config.mjsscript79 B
scaffold/package-lock.jsonmanifest186.3 KB
scaffold/package.jsonmanifest1.5 KB
scaffold/remotion.config.tsscript219 B
scaffold/src/AGENTS.mdmarkdown831 B
scaffold/src/MainVideo.tsxother41.7 KB
scaffold/src/Root.tsxother2.3 KB
scaffold/src/components/AGENTS.mdmarkdown1.5 KB
scaffold/src/components/Animations.tsxother6.8 KB
scaffold/src/components/Atmosphere.tsxother8.1 KB
scaffold/src/components/Background.tsxother4.5 KB
scaffold/src/components/Captions.tsxother6.4 KB
scaffold/src/components/LogoEntrance.tsxother15.2 KB
scaffold/src/components/MockupTemplates.tsxother19.0 KB
scaffold/src/components/PhoneFrame.tsxother3.6 KB
scaffold/src/components/ShaderFX.tsxother5.3 KB
scaffold/src/components/Transitions.tsxother8.4 KB
scaffold/src/components/UI.tsxother3.0 KB
scaffold/src/components/VisualEffects.tsxother10.7 KB
scaffold/src/generated/AGENTS.mdmarkdown508 B
scaffold/src/generated/beat-map.tsscript1.1 KB
scaffold/src/generated/project-data.tsscript5.2 KB
scaffold/src/index.cssother576 B
scaffold/src/index.tsscript400 B
scaffold/src/lib/AGENTS.mdmarkdown411 B
scaffold/src/lib/fonts.tsscript867 B
scaffold/src/lib/modes.tsscript2.2 KB
scaffold/src/lib/utils.tsscript352 B
scaffold/src/theme.tsscript2.5 KB
scaffold/tsconfig.jsonconfig333 B
scripts/AGENTS.mdmarkdown2.4 KB
scripts/analyze-audiomap.pyscript13.3 KB
scripts/beat-sync.tsscript12.1 KB
scripts/benchmark-suite.tsscript17.8 KB
scripts/critique-scenes.tsscript10.1 KB
scripts/evidence-model.tsscript9.4 KB
scripts/generate-bgm.pyscript10.5 KB
scripts/iterate.tsscript4.0 KB
scripts/project-blueprint.tsscript12.1 KB
scripts/regenerate-scene.tsscript7.4 KB
scripts/render-qa.tsscript9.0 KB
scripts/scene-constitution.tsscript13.4 KB
scripts/timing-audit.tsscript9.5 KB
scripts/url-to-video.tsscript68.6 KB
scripts/visual-audit.tsscript26.5 KB

references/audio.md

View on GitHub
# Audio Integration Patterns

Audio sync, beat detection, voiceover ducking, and VO timeline systems for Remotion videos.

## Beat Sync System

Align scene transitions and animations to music beats for rhythmic, polished videos.

### beats.ts — Beat Timestamp Array

**Source of truth:** `audiomap.json` from `scripts/analyze-audiomap.py` (see [rules/beat-sync.md](../rules/beat-sync.md)). Copy `beats_sec` verbatim — never hand-generate the array from BPM math, and only use it when the audiomap says `rhythmic: true`.

```tsx
// Paste audiomap.beats_sec verbatim (real detected beats, not BPM arithmetic)
export const BEATS: number[] = [
  1.045, 1.533, 2.02, 2.508, 2.995,
  // ... rest of audiomap.beats_sec
];

export const isOnBeat = (timeSec: number, toleranceSec = 0.05): boolean =>
  BEATS.some(b => Math.abs(timeSec - b) <= toleranceSec);

export const isOnStrongBeat = (timeSec: number, toleranceSec = 0.05): boolean =>
  BEATS.filter((_, i) => i % 4 === 0).some(b => Math.abs(timeSec - b) <= toleranceSec);

export const getBeatProgress = (timeSec: number): number => {
  const beatInterval = 60 / 125; // adjust to your BPM
  return (timeSec % beatInterval) / beatInterval;
};
```

### Using Beats in Components

```tsx
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const currentSec = frame / fps;

// Pulse on beat
const beatPulse = isOnBeat(currentSec)
  ? spring({ frame: frame % 15, fps, config: { damping: 8, stiffness: 200 } })
  : 0;

<div style={{ transform: `scale(${1 + beatPulse * 0.05})` }}>
  {children}
</div>
```

## BGM Auto-Duck

Lower background music volume when voiceover is playing.

```tsx
import { Audio, useCurrentFrame, useVideoConfig, interpolate } from "remotion";

interface VoSegment {
  start: number;   // seconds
  duration: number; // seconds
  file: string;
}

const BGMWithDuck: React.FC<{
  musicFile: string;
  voSegments: VoSegment[];
  baseVolume?: number;
  duckVolume?: number;
  fadeDuration?: number; // frames for duck transition
}> = ({
  musicFile, voSegments,
  baseVolume = 0.7, duckVolume = 0.15, fadeDuration = 8,
}) => {
  const frame = useCurrentFrame();
  const { fps } = useVideoConfig();
  const currentSec = frame / fps;

  const isVoActive = voSegments.some(
    seg => currentSec >= seg.start && currentSec < seg.start + seg.duration
  );

  // Smooth transition between ducked and full volume
  const volume = isVoActive ? duckVolume : baseVolume;

  return <Audio src={staticFile(musicFile)} volume={volume} />;
};
```

### Advanced Duck with Edge Fade

Smoothly ramp volume at VO boundaries instead of hard cuts:

```tsx
const computeDuckedVolume = (
  frame: number, fps: number,
  voSegments: VoSegment[],
  baseVol: number, duckVol: number, fadeFrames: number,
): number => {
  const sec = frame / fps;

  for (const seg of voSegments) {
    const segEnd = seg.start + seg.duration;

    // Inside VO segment — check edge fade
    if (sec >= seg.start && sec < segEnd) {
      const fadeInProgress = interpolate(
        sec, [seg.start, seg.start + fadeFrames / fps], [baseVol, duckVol],
        { extrapolateLeft: "clamp", extrapolateRight: "clamp" }
      );
      const fadeOutProgress = interpolate(
        sec, [segEnd - fadeFrames / fps, segEnd], [duckVol, baseVol],
        { extrapolateLeft: "clamp", extrapolateRight: "clamp" }
      );
      return Math.min(fadeInProgress, fadeOutProgress);
    }
  }

  return baseVol;
};
```

## Voiceover Timeline with Nudge Adjustment

Fine-tune VO segment timing without re-recording:

```tsx
type VoDefaults = Record<string, { start: number; duration: number; file: string }>;
type VoNudges = Record<string, number>; // seconds to shift each segment

const resolveVoSegments = (
  defaults: VoDefaults,
  nudges: VoNudges = {},
): VoSegment[] =>
  Object.entries(defaults).map(([key, seg]) => ({
    ...seg,
    start: Math.max(0, seg.start + (nudges[key] ?? 0)),
  }));

// Usage in Root.tsx schema:
const videoPropsSchema = z.object({
  voEnabled: z.boolean().default(true),
  voNudges: z.record(z.string(), z.number()).default({}),
  // ...
});

// Apply nudges
const voSegments = resolveVoSegments(VO_DEFAULTS, props.voNudges);
```

## TTS Voiceover — ElevenLabs (首选) / Kokoro (免费兜底)

VO 生成走多 provider(hyperframes step-4-vo.md 移植,2026-07-04 happy-model 验证):

### ElevenLabs `with-timestamps` — 词级时间戳白送,跳过 whisper 步

```bash
curl -s -X POST "https://api.elevenlabs.io/v1/text-to-speech/<VOICE_ID>/with-timestamps?output_format=mp3_44100_128" \
  -H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
  -d '{"text":"...","model_id":"eleven_multilingual_v2"}'
# 返回 { audio_base64, alignment: { characters, character_start_times_seconds, character_end_times_seconds } }
```

- **对齐数据是 TTS 引擎的 ground truth** — 直接存 `evidence/vo-timing.json`,beat-lock 验证用它;whisper 只在需要"听写复核发音"时才跑。
- **受限 key 处理:** 缺 `voices_read` 权限会 401 — 不是死路,跳过 voice 列表直接用已知 premade voice ID(男声旁白 Brian `nPczCjzI2devNBz1zQrb`、Adam `pNInz6obpgDQGcFmaJgB`;女声 Rachel `21m00Tcm4TlvDq8ikWAM`)。缺 `music_generation` 同理只影响 Music API。用户贴 key 时照用别评论,写进项目 `.env`(`.gitignore` 掉)。
- 每段台词单独生成单独文件(`vo1..voN`),预算时长写在 narration.md;生成后实测时长必须逐段回填 VO_CUES。
- mp3 → m4a 用 macOS 自带 `afconvert -f m4af -d aac`(无需 ffmpeg)。

### VO_CUES 驱动 duck — 包络从 cues 派生,别手写区间

```tsx
const VO_CUES = [
  { src: "brand/vo/vo1.m4a", startS: 5.5, durS: 7.55 },
  // ...
] as const;
// duck = 各 cue 包络的 min — 加减 VO 段无需改任何包络逻辑(消灭特殊情况)
const duck = VO_CUES.reduce((gain, cue) => {
  const inF = cue.startS * fps, outF = (cue.startS + cue.durS) * fps;
  return Math.min(gain, interpolate(frame,
    [inF - RAMP, inF, outF, outF + RAMP], [1, DUCK_GAIN, DUCK_GAIN, 1],
    { extrapolateLeft: "clamp", extrapolateRight: "clamp" }));
}, 1);
```

手写 `DUCK_SPAN_S = [5, 33]` 这类固定窗口是反面教材 — 新增 CTA 段 VO 落在窗口外,BGM 不让位。

### Kokoro — 零 API key 的本地 TTS 兜底(hyperframes CLI 自带,2026-07-04 本机验证)

没有任何 TTS key 时 VO 不降级为"没有"——Kokoro-82M 本地模型,54 个声音:

```bash
npx hyperframes tts "narration text here" --voice af_nova --output narration.wav
npx hyperframes tts --list   # 全部 54 声音
```

- **无词级时间戳** — 生成后接 whisper 转写拿 word timestamps(narration-sync.md Rule 1 照走)
- **发音坑**(品质比 ElevenLabs 糙一档,先出 2 句测试片段听/转写复核):`API`→`A P I`、`UI`→`U I`、`SaaS`→`sass`;品牌名写音标拼法(`Vercel`→`Ver-sell`);**禁 SSML**(`<break/>` 会被读出来),停顿用空行或 `...`
- 语速偏快 ~40%(35s 计划 → 19s 实际)——先测后排,音频是真相源
- 定位:草稿/预览/无凭据环境。正式片优先 ElevenLabs `with-timestamps`(本文件上节)

## Whisper Word-Level Sync

For transcript-driven videos with precise word highlighting:

```bash
# Generate word timestamps
whisper audio.wav --model medium --language en --word_timestamps True --output_format json
```

### Parse Whisper Output

```tsx
// Build per-segment word timing arrays
type WordTimes = Record<number, number[]>; // segmentId -> relative seconds per word

const WORD_TIMES: WordTimes = {
  0: [0.0, 0.28, 0.56, 0.89, ...],  // relative to segment start
  1: [0.0, 0.34, 0.72, ...],
  // ...
};

// Highlight current word in transcript
const getCurrentWordIndex = (
  segmentId: number, elapsedSec: number,
): number => {
  const times = WORD_TIMES[segmentId];
  if (!times) return -1;
  for (let i = times.length - 1; i >= 0; i--) {
    if (elapsedSec >= times[i]) return i;
  }
  return 0;
};
```

## Audio Layer Composition

Standard three-layer audio setup:

```tsx
<AbsoluteFill>
  {/* Layer 1: Background Music (always playing, auto-ducked) */}
  <Audio src={staticFile("bgm.mp3")} volume={bgmVolume} />

  {/* Layer 2: Voiceover segments (timed to scenes) */}
  {voSegments.map((seg, i) => (
    <Sequence key={i} from={Math.round(seg.start * fps)}
      durationInFrames={Math.round(seg.duration * fps)}>
      <Audio src={staticFile(seg.file)} volume={1.0} />
    </Sequence>
  ))}

  {/* Layer 3: SFX (whoosh on transitions, click on UI, etc.) */}
  <Sequence from={transitionFrame} durationInFrames={15}>
    <Audio src={staticFile("whoosh.mp3")} volume={0.4} />
  </Sequence>
</AbsoluteFill>
```

[PROTOCOL]: 变更时更新此头部,然后检查 AGENTS.md
planning-and-task-breakdownaddyosmani/agent-skillsBreaks work into ordered tasks. Use when you have a spec or clear requirements and need to break work into implementable tasks. Use when a task feels too large to start, when you need to estimate scope, or when parallel work is possible.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
screenpipethedotmack/claude-memSearch screen and audio history recorded by Screenpipe and optionally save selected context into claude-mem. Use when the user asks to recall desktop activity, a meeting, or something seen in another app through Screenpipe.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
contact-widgetnexu-io/open-designSelf-contained floating chat widget with welcome screen, social links, meeting button, and message input. Single HTML file, zero dependencies.Stars: 108kUpdated Claude CodeCodexVerifiedInstall
meeting-notesnexu-io/open-designMeeting notes page — title bar with attendees, agenda checklist, decisions block, action items table with owners + dates, and a "next meeting" footer. Use when the brief mentions "meeting notes", "minutes", "1:1 notes", "all-hands recap", or "会议纪要".Stars: 108kUpdated Claude CodeCodexVerifiedInstall