Skip to content

guanyang/open-agent-hub / skills/remotion

remotion-best-practices

Verified

Best practices and domain knowledge for building videos programmatically with Remotion (videos in React/TypeScript, rendered to MP4). Use whenever writing or editing Remotion code — scaffolding a video project, animating with interpolate/spring over useCurrentFrame(), sequencing scenes and transitions, captions/subtitles, audio, GIFs/Lottie/3D, charts, visual effects, dynamic duration via calculateMetadata, or rendering. Consult before reaching for a Remotion API, since its components and props evolve (e.g. @remotion/media).

FileKindSize
SKILL.mdskill md12.0 KB
rules/3d.mdmarkdown2.2 KB
rules/animations.mdmarkdown1.1 KB
rules/assets.mdmarkdown1.6 KB
rules/assets/charts-bar-chart.tsxother3.3 KB
rules/assets/text-animations-typewriter.tsxother2.1 KB
rules/assets/text-animations-word-highlight.tsxother2.3 KB
rules/audio-visualization.mdmarkdown4.8 KB
rules/audio.mdmarkdown3.5 KB
rules/calculate-metadata.mdmarkdown3.2 KB
rules/can-decode.mdmarkdown1.6 KB
rules/charts.mdmarkdown2.9 KB
rules/compositions.mdmarkdown3.3 KB
rules/display-captions.mdmarkdown5.3 KB
rules/effects.mdmarkdown7.9 KB
rules/extract-frames.mdmarkdown5.4 KB
rules/ffmpeg.mdmarkdown1.1 KB
rules/fonts.mdmarkdown3.4 KB
rules/get-audio-duration.mdmarkdown1.3 KB
rules/get-video-dimensions.mdmarkdown1.6 KB
rules/get-video-duration.mdmarkdown1.3 KB
rules/gifs.mdmarkdown3.6 KB
rules/google-fonts.mdmarkdown1.7 KB
rules/html-in-canvas.mdmarkdown3.5 KB
rules/images.mdmarkdown1.4 KB
rules/import-srt-captions.mdmarkdown2.2 KB
rules/light-leaks.mdmarkdown2.3 KB
rules/local-fonts.mdmarkdown1.5 KB
rules/lottie.mdmarkdown1.8 KB
rules/mapbox.mdmarkdown11.0 KB
rules/maplibre.mdmarkdown12.2 KB
rules/maps.mdmarkdown11.0 KB
rules/measuring-dom-nodes.mdmarkdown973 B
rules/measuring-text.mdmarkdown2.7 KB
rules/parameters.mdmarkdown2.3 KB
rules/sequencing.mdmarkdown3.2 KB
rules/sfx.mdmarkdown1.8 KB
rules/silence-detection.mdmarkdown2.5 KB
rules/subtitles.mdmarkdown922 B
rules/tailwind.mdmarkdown420 B
rules/text-animations.mdmarkdown700 B
rules/timing.mdmarkdown5.4 KB
rules/transcribe-captions.mdmarkdown1.9 KB
rules/transitions.mdmarkdown5.7 KB
rules/transparent-videos.mdmarkdown2.2 KB
rules/trimming.mdmarkdown1.2 KB
rules/video-layout.mdmarkdown4.0 KB
rules/videos.mdmarkdown3.4 KB
rules/voiceover.mdmarkdown3.2 KB

rules/extract-frames.md

View on GitHub
---
name: extract-frames
description: Extract frames from videos at specific timestamps using Mediabunny
metadata:
  tags: frames, extract, video, thumbnail, filmstrip, canvas
---

# Extracting frames from videos

Use Mediabunny to extract frames from videos at specific timestamps. This is useful for generating thumbnails, filmstrips, or processing individual frames.

## The `extractFrames()` function

This function can be copy-pasted into any project.

```tsx
import {
  ALL_FORMATS,
  Input,
  UrlSource,
  VideoSample,
  VideoSampleSink,
} from "mediabunny";

type Options = {
  track: { width: number; height: number };
  container: string;
  durationInSeconds: number | null;
};

export type ExtractFramesTimestampsInSecondsFn = (
  options: Options,
) => Promise<number[]> | number[];

export type ExtractFramesProps = {
  src: string;
  timestampsInSeconds: number[] | ExtractFramesTimestampsInSecondsFn;
  onVideoSample: (sample: VideoSample) => void;
  signal?: AbortSignal;
};

export async function extractFrames({
  src,
  timestampsInSeconds,
  onVideoSample,
  signal,
}: ExtractFramesProps): Promise<void> {
  using input = new Input({
    formats: ALL_FORMATS,
    source: new UrlSource(src),
  });

  const [durationInSeconds, format, videoTrack] = await Promise.all([
    input.computeDuration(),
    input.getFormat(),
    input.getPrimaryVideoTrack(),
  ]);

  if (!videoTrack) {
    throw new Error("No video track found in the input");
  }

  if (signal?.aborted) {
    throw new Error("Aborted");
  }

  const timestamps =
    typeof timestampsInSeconds === "function"
      ? await timestampsInSeconds({
          track: {
            width: videoTrack.displayWidth,
            height: videoTrack.displayHeight,
          },
          container: format.name,
          durationInSeconds,
        })
      : timestampsInSeconds;

  if (timestamps.length === 0) {
    return;
  }

  if (signal?.aborted) {
    throw new Error("Aborted");
  }

  const sink = new VideoSampleSink(videoTrack);

  for await (using videoSample of sink.samplesAtTimestamps(timestamps)) {
    if (signal?.aborted) {
      break;
    }

    if (!videoSample) {
      continue;
    }

    onVideoSample(videoSample);
  }
}
```

## Basic usage

Extract frames at specific timestamps:

```tsx
await extractFrames({
  src: "https://remotion.media/video.mp4",
  timestampsInSeconds: [0, 1, 2, 3, 4],
  onVideoSample: (sample) => {
    const canvas = document.createElement("canvas");
    canvas.width = sample.displayWidth;
    canvas.height = sample.displayHeight;
    const ctx = canvas.getContext("2d");
    sample.draw(ctx!, 0, 0);
  },
});
```

## Creating a filmstrip

Use a callback function to dynamically calculate timestamps based on video metadata:

```tsx
const canvasWidth = 500;
const canvasHeight = 80;
const fromSeconds = 0;
const toSeconds = 10;

await extractFrames({
  src: "https://remotion.media/video.mp4",
  timestampsInSeconds: async ({ track, durationInSeconds }) => {
    const aspectRatio = track.width / track.height;
    const amountOfFramesFit = Math.ceil(
      canvasWidth / (canvasHeight * aspectRatio),
    );
    const segmentDuration = toSeconds - fromSeconds;
    const timestamps: number[] = [];

    for (let i = 0; i < amountOfFramesFit; i++) {
      timestamps.push(
        fromSeconds + (segmentDuration / amountOfFramesFit) * (i + 0.5),
      );
    }

    return timestamps;
  },
  onVideoSample: (sample) => {
    console.log(`Frame at ${sample.timestamp}s`);

    const canvas = document.createElement("canvas");
    canvas.width = sample.displayWidth;
    canvas.height = sample.displayHeight;
    const ctx = canvas.getContext("2d");
    sample.draw(ctx!, 0, 0);
  },
});
```

## Cancellation with AbortSignal

Cancel frame extraction after a timeout:

```tsx
const controller = new AbortController();

setTimeout(() => controller.abort(), 5000);

try {
  await extractFrames({
    src: "https://remotion.media/video.mp4",
    timestampsInSeconds: [0, 1, 2, 3, 4],
    onVideoSample: (sample) => {
      using frame = sample;
      const canvas = document.createElement("canvas");
      canvas.width = frame.displayWidth;
      canvas.height = frame.displayHeight;
      const ctx = canvas.getContext("2d");
      frame.draw(ctx!, 0, 0);
    },
    signal: controller.signal,
  });

  console.log("Frame extraction complete!");
} catch (error) {
  console.error("Frame extraction was aborted or failed:", error);
}
```

## Timeout with Promise.race

```tsx
const controller = new AbortController();

const timeoutPromise = new Promise<never>((_, reject) => {
  const timeoutId = setTimeout(() => {
    controller.abort();
    reject(new Error("Frame extraction timed out after 10 seconds"));
  }, 10000);

  controller.signal.addEventListener("abort", () => clearTimeout(timeoutId), {
    once: true,
  });
});

try {
  await Promise.race([
    extractFrames({
      src: "https://remotion.media/video.mp4",
      timestampsInSeconds: [0, 1, 2, 3, 4],
      onVideoSample: (sample) => {
        using frame = sample;
        const canvas = document.createElement("canvas");
        canvas.width = frame.displayWidth;
        canvas.height = frame.displayHeight;
        const ctx = canvas.getContext("2d");
        frame.draw(ctx!, 0, 0);
      },
      signal: controller.signal,
    }),
    timeoutPromise,
  ]);

  console.log("Frame extraction complete!");
} catch (error) {
  console.error("Frame extraction was aborted or failed:", error);
}
```
brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 322kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 198kUpdated Claude CodeCodexVerifiedInstall