Skip to content

ahmedmaknessi/sum-of-parts / .claude/skills/remotion-best-practices

remotion-best-practices

Verified

Router for all Remotion skills

FileKindSize
SKILL.mdskill md4.1 KB
agents/openai.yamlconfig311 B
assets/remotion-icon.svgother1.3 KB
remotion-captions/REFERENCE.mdmarkdown1.1 KB
remotion-captions/agents/openai.yamlconfig305 B
remotion-captions/assets/remotion-icon.svgother1.3 KB
remotion-captions/display-captions.mdmarkdown3.3 KB
remotion-captions/import-srt-captions.mdmarkdown2.2 KB
remotion-captions/transcribe-captions.mdmarkdown5.1 KB
remotion-create/REFERENCE.mdmarkdown2.9 KB
remotion-create/agents/openai.yamlconfig310 B
remotion-create/assets/remotion-icon.svgother1.3 KB
remotion-create/tailwind.mdmarkdown386 B
remotion-create/video-layout.mdmarkdown498 B
remotion-docs/REFERENCE.mdmarkdown1.3 KB
remotion-docs/agents/openai.yamlconfig307 B
remotion-docs/assets/remotion-icon.svgother1.3 KB
remotion-interactivity/REFERENCE.mdmarkdown18.3 KB
remotion-interactivity/agents/openai.yamlconfig320 B
remotion-interactivity/assets/remotion-icon.svgother1.3 KB
remotion-interactivity/inline-effects.mdmarkdown993 B
remotion-maps/REFERENCE.mdmarkdown1.0 KB
remotion-maps/agents/openai.yamlconfig293 B
remotion-maps/assets/remotion-icon.svgother1.3 KB
remotion-maps/techniques/cesium/TECHNIQUE.mdmarkdown3.8 KB
remotion-maps/techniques/cesium/assets/CesiumFlythrough.tsxother8.7 KB
remotion-maps/techniques/cesium/assets/cesium-path.jsonconfig12.0 KB
remotion-maps/techniques/cesium/assets/city-path.jsonconfig223 B
remotion-maps/techniques/cesium/assets/example-Root.tsxother923 B
remotion-maps/techniques/cesium/assets/flight-path.tsscript1.2 KB
remotion-maps/techniques/cesium/assets/sample-river.geojsonother6.3 KB
remotion-maps/techniques/cesium/references/3d-data-sources.mdmarkdown1.4 KB
remotion-maps/techniques/cesium/references/3d-flyover-architecture.mdmarkdown7.7 KB
remotion-maps/techniques/cesium/references/3d-troubleshooting.mdmarkdown6.6 KB
remotion-maps/techniques/cesium/scripts/prep-cesium-path.mjsscript5.4 KB
remotion-maps/techniques/mapbox/TECHNIQUE.mdmarkdown12.8 KB
remotion-maps/techniques/mapbox/references/render-stability.mdmarkdown3.6 KB
remotion-maps/techniques/maplibre/TECHNIQUE.mdmarkdown13.5 KB
remotion-maps/techniques/maplibre/references/render-stability.mdmarkdown3.6 KB
remotion-maps/techniques/maptiler/TECHNIQUE.mdmarkdown4.4 KB
remotion-maps/techniques/maptiler/assets/CountryLabel.tsxother1.8 KB
remotion-maps/techniques/maptiler/assets/MapTilerVectorElement.tsscript2.1 KB
remotion-maps/techniques/maptiler/assets/RiverReveal.tsxother10.6 KB
remotion-maps/techniques/maptiler/assets/example-Root.tsxother1.0 KB
remotion-maps/techniques/maptiler/assets/sample-data/country-meta.jsonconfig325.3 KB
remotion-maps/techniques/maptiler/assets/sample-data/yarlung-flow.jsonconfig12.8 KB
remotion-maps/techniques/maptiler/assets/tokens.tsscript1.2 KB
remotion-maps/techniques/maptiler/references/map-data-sources.mdmarkdown4.7 KB
remotion-maps/techniques/maptiler/references/map-explainer-architecture.mdmarkdown6.7 KB
remotion-maps/techniques/maptiler/references/map-geo-prep.mdmarkdown4.0 KB
remotion-maps/techniques/maptiler/references/render-stability.mdmarkdown3.6 KB
remotion-maps/techniques/maptiler/scripts/prep-geo.mjsscript7.4 KB
remotion-maps/techniques/static-map/TECHNIQUE.mdmarkdown1.1 KB
remotion-markup/3d.mdmarkdown2.2 KB
remotion-markup/REFERENCE.mdmarkdown14.7 KB
remotion-markup/agents/openai.yamlconfig310 B
remotion-markup/assets/remotion-icon.svgother1.3 KB
remotion-markup/audio-visualization.mdmarkdown4.8 KB
remotion-markup/audio.mdmarkdown4.5 KB
remotion-markup/calculate-metadata.mdmarkdown3.4 KB
remotion-markup/compositions.mdmarkdown4.9 KB
remotion-markup/connected-compositions.mdmarkdown5.0 KB
remotion-markup/cropping.mdmarkdown1.3 KB
remotion-markup/effects.mdmarkdown8.7 KB
remotion-markup/embedding-videos.mdmarkdown3.7 KB
remotion-markup/ffmpeg.mdmarkdown1.1 KB
remotion-markup/gifs.mdmarkdown3.7 KB
remotion-markup/google-fonts.mdmarkdown1.7 KB
remotion-markup/html-in-canvas.mdmarkdown2.9 KB
remotion-markup/images.mdmarkdown1.5 KB
remotion-markup/light-leaks.mdmarkdown3.1 KB
remotion-markup/local-fonts.mdmarkdown1.5 KB
remotion-markup/lottie.mdmarkdown1.8 KB
remotion-markup/measuring-dom-nodes.mdmarkdown995 B
remotion-markup/measuring-text.mdmarkdown2.7 KB
remotion-markup/motion-blur.mdmarkdown2.0 KB
remotion-markup/multi-scene-video.mdmarkdown3.1 KB
remotion-markup/parameters.mdmarkdown2.3 KB
remotion-markup/remotion-maps/REFERENCE.mdmarkdown1.0 KB
remotion-markup/remotion-maps/agents/openai.yamlconfig293 B
remotion-markup/remotion-maps/assets/remotion-icon.svgother1.3 KB
remotion-markup/remotion-maps/techniques/cesium/TECHNIQUE.mdmarkdown3.8 KB
remotion-markup/remotion-maps/techniques/cesium/assets/CesiumFlythrough.tsxother8.7 KB
remotion-markup/remotion-maps/techniques/cesium/assets/cesium-path.jsonconfig12.0 KB
remotion-markup/remotion-maps/techniques/cesium/assets/city-path.jsonconfig223 B
remotion-markup/remotion-maps/techniques/cesium/assets/example-Root.tsxother923 B
remotion-markup/remotion-maps/techniques/cesium/assets/flight-path.tsscript1.2 KB
remotion-markup/remotion-maps/techniques/cesium/assets/sample-river.geojsonother6.3 KB
remotion-markup/remotion-maps/techniques/cesium/references/3d-data-sources.mdmarkdown1.4 KB
remotion-markup/remotion-maps/techniques/cesium/references/3d-flyover-architecture.mdmarkdown7.7 KB
remotion-markup/remotion-maps/techniques/cesium/references/3d-troubleshooting.mdmarkdown6.6 KB
remotion-markup/remotion-maps/techniques/cesium/scripts/prep-cesium-path.mjsscript5.4 KB
remotion-markup/remotion-maps/techniques/mapbox/TECHNIQUE.mdmarkdown12.8 KB
remotion-markup/remotion-maps/techniques/mapbox/references/render-stability.mdmarkdown3.6 KB
remotion-markup/remotion-maps/techniques/maplibre/TECHNIQUE.mdmarkdown13.5 KB
remotion-markup/remotion-maps/techniques/maplibre/references/render-stability.mdmarkdown3.6 KB
remotion-markup/remotion-maps/techniques/maptiler/TECHNIQUE.mdmarkdown4.4 KB
remotion-markup/remotion-maps/techniques/maptiler/assets/CountryLabel.tsxother1.8 KB
remotion-markup/remotion-maps/techniques/maptiler/assets/MapTilerVectorElement.tsscript2.1 KB
remotion-markup/remotion-maps/techniques/maptiler/assets/RiverReveal.tsxother10.6 KB
remotion-markup/remotion-maps/techniques/maptiler/assets/example-Root.tsxother1.0 KB
remotion-markup/remotion-maps/techniques/maptiler/assets/sample-data/country-meta.jsonconfig325.3 KB
remotion-markup/remotion-maps/techniques/maptiler/assets/sample-data/yarlung-flow.jsonconfig12.8 KB
remotion-markup/remotion-maps/techniques/maptiler/assets/tokens.tsscript1.2 KB
remotion-markup/remotion-maps/techniques/maptiler/references/map-data-sources.mdmarkdown4.7 KB
remotion-markup/remotion-maps/techniques/maptiler/references/map-explainer-architecture.mdmarkdown6.7 KB
remotion-markup/remotion-maps/techniques/maptiler/references/map-geo-prep.mdmarkdown4.0 KB
remotion-markup/remotion-maps/techniques/maptiler/references/render-stability.mdmarkdown3.6 KB
remotion-markup/remotion-maps/techniques/maptiler/scripts/prep-geo.mjsscript7.4 KB
remotion-markup/remotion-maps/techniques/static-map/TECHNIQUE.mdmarkdown1.1 KB
remotion-markup/sequencing.mdmarkdown2.7 KB
remotion-markup/sfx.mdmarkdown1.8 KB
remotion-markup/silence-detection.mdmarkdown2.5 KB
remotion-markup/text-highlights.mdmarkdown1.8 KB
remotion-markup/timing-props.mdmarkdown3.5 KB
remotion-markup/timing.mdmarkdown3.2 KB
remotion-markup/transitions.mdmarkdown7.4 KB
remotion-markup/video-editing.mdmarkdown5.1 KB
remotion-markup/voiceover.mdmarkdown3.6 KB
remotion-multimedia/REFERENCE.mdmarkdown705 B
remotion-multimedia/agents/openai.yamlconfig317 B
remotion-multimedia/assets/remotion-icon.svgother1.3 KB
remotion-multimedia/get-audio-duration.mdmarkdown1.5 KB
remotion-multimedia/get-video-dimensions.mdmarkdown1.8 KB
remotion-multimedia/get-video-duration.mdmarkdown1.4 KB
remotion-render/REFERENCE.mdmarkdown711 B
remotion-render/agents/openai.yamlconfig286 B
remotion-render/assets/remotion-icon.svgother1.3 KB
remotion-render/transparent-videos.mdmarkdown2.2 KB
remotion-saas/REFERENCE.mdmarkdown1.0 KB
remotion-saas/agents/openai.yamlconfig283 B
remotion-saas/assets/remotion-icon.svgother1.3 KB
remotion-saas/framework.mdmarkdown908 B
remotion-saas/player.mdmarkdown942 B
remotion-saas/rendering.mdmarkdown3.3 KB
remotion-studio/REFERENCE.mdmarkdown1.6 KB
remotion-studio/agents/openai.yamlconfig237 B
remotion-studio/assets/remotion-icon.svgother1.3 KB
remotion-upgrade/REFERENCE.mdmarkdown1.9 KB
remotion-upgrade/agents/openai.yamlconfig282 B
remotion-upgrade/assets/remotion-icon.svgother1.3 KB

remotion-captions/transcribe-captions.md

View on GitHub
# Transcribing audio

To transcribe audio to generate captions in Remotion, use the [`transcribe()`](https://www.remotion.dev/docs/whisper-webgpu/transcribe.md) function from the [`@remotion/whisper-webgpu`](https://www.remotion.dev/docs/whisper-webgpu.md) package.
It runs Whisper locally on the GPU and works in both Node.js and the browser.

## Prerequisites

Install the required packages if they are not installed:

```bash
npx remotion add @remotion/whisper-webgpu @huggingface/transformers mediabunny @mediabunny/server # If project uses npm
bunx remotion add @remotion/whisper-webgpu @huggingface/transformers mediabunny @mediabunny/server # If project uses bun
yarn remotion add @remotion/whisper-webgpu @huggingface/transformers mediabunny @mediabunny/server # If project uses yarn
pnpm exec remotion add @remotion/whisper-webgpu @huggingface/transformers mediabunny @mediabunny/server # If project uses pnpm
```

A compatible GPU is required. ONNX Runtime does not work on Linux arm64.

## Transcribing

Make a Node.js script that decodes the audio to a 16kHz mono waveform, downloads a model, and transcribes it.

```ts
import { registerMediabunnyServer } from "@mediabunny/server";
import {
  WHISPER_WEBGPU_SAMPLE_RATE,
  canUseWhisperWebGpu,
  downloadWhisperModel,
  loadWhisperModel,
  toCaptions,
  transcribe,
} from "@remotion/whisper-webgpu";
import {
  ALL_FORMATS,
  Conversion,
  FilePathSource,
  Input,
  NullTarget,
  Output,
  WavOutputFormat,
} from "mediabunny";
import { writeFile } from "node:fs/promises";

registerMediabunnyServer();

const support = await canUseWhisperWebGpu();
if (!support.supported) {
  throw new Error(support.detailedReason);
}

type WaveformChunk = {
  startFrame: number;
  waveform: Float32Array;
};
const chunks: WaveformChunk[] = [];

using input = new Input({
  formats: ALL_FORMATS,
  source: new FilePathSource("public/video123.mp4"),
});
const audioTrack = await input.getPrimaryAudioTrack();
if (audioTrack === null) {
  throw new Error("The media does not contain an audio track.");
}

const conversion = await Conversion.init({
  input,
  output: new Output({
    format: new WavOutputFormat(),
    target: new NullTarget(),
  }),
  video: { discard: true },
  audio: (track) => {
    if (track.id !== audioTrack.id) {
      return { discard: true };
    }

    return {
      codec: "pcm-f32",
      forceTranscode: true,
      numberOfChannels: 1,
      sampleFormat: "f32",
      sampleRate: WHISPER_WEBGPU_SAMPLE_RATE,
      process: (sample) => {
        const waveform = new Float32Array(
          sample.allocationSize({ format: "f32", planeIndex: 0 }) /
            Float32Array.BYTES_PER_ELEMENT,
        );
        sample.copyTo(waveform, { format: "f32", planeIndex: 0 });
        chunks.push({
          startFrame: Math.round(sample.timestamp * WHISPER_WEBGPU_SAMPLE_RATE),
          waveform,
        });
        return sample;
      },
    };
  },
});

if (!conversion.isValid) {
  throw new Error("The audio track cannot be decoded.");
}

await conversion.execute();

const waveformLength = chunks.reduce(
  (max, chunk) => Math.max(max, chunk.startFrame + chunk.waveform.length),
  0,
);
const channelWaveform = new Float32Array(waveformLength);
for (const chunk of chunks) {
  const destinationStart = Math.max(0, chunk.startFrame);
  const sourceStart = Math.max(0, -chunk.startFrame);
  const availableLength = Math.min(
    chunk.waveform.length - sourceStart,
    channelWaveform.length - destinationStart,
  );

  if (availableLength > 0) {
    channelWaveform.set(
      chunk.waveform.subarray(sourceStart, sourceStart + availableLength),
      destinationStart,
    );
  }
}

const model = "small.en";
await downloadWhisperModel({ model });
await using modelHandle = await loadWhisperModel({ model });
const transcription = await transcribe({ channelWaveform, model });
const { captions } = toCaptions({ whisperWebGpuOutput: transcription });

// Write it to a file so the captions can be inlined into the Remotion code
await writeFile("captions123.json", JSON.stringify(captions, null, 2));
```

## Choosing a model

`small.en` is the recommended default for English.  
For other languages, use a multilingual model such as `small` and pass the `language` option to `transcribe()` - automatic language detection is not supported.  
See [`getAvailableModels()`](https://www.remotion.dev/docs/whisper-webgpu/get-available-models.md) for all models.

## Transcribing in the browser

In the browser, use [`resampleTo16Khz()`](https://www.remotion.dev/docs/whisper-webgpu/resample-to-16khz.md) to get the waveform from a `File` instead of using Mediabunny:

```ts
import {
  downloadWhisperModel,
  resampleTo16Khz,
  toCaptions,
  transcribe,
} from "@remotion/whisper-webgpu";

export const transcribeFile = async (file: File) => {
  await downloadWhisperModel({ model: "small.en" });
  const channelWaveform = await resampleTo16Khz({ file });
  const transcription = await transcribe({
    channelWaveform,
    model: "small.en",
  });

  const { captions } = toCaptions({ whisperWebGpuOutput: transcription });
  return captions;
};
```

Transcribe each clip individually.

See [Displaying captions](display-captions.md) for how to display the captions in Remotion.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 323kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 323kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 323kUpdated Claude CodeCodexVerifiedInstall