Skip to content

gentleeduck/duck-ui / .claude/skills/vitest

vitest

Verified

Vitest fast unit testing framework powered by Vite with Jest-compatible API. Use when writing tests, mocking, configuring coverage, or working with test filtering and fixtures.

references/features-benchmarking.md

View on GitHub
---
name: benchmarking
description: Write benchmarks with the v5 bench test-context fixture (Tinybench)
---

# Benchmarking (v5)

In v5 the benchmark API was rewritten: `bench` is **no longer a top-level import**. It is a [test-context fixture](features-context.md) used inside a regular `test()`, available only in files matched by `benchmark.include` (default `**/*.{bench,benchmark}.?(c|m)[jt]s?(x)`). Benchmarks are powered by [Tinybench](https://github.com/tinylibs/tinybench).

## Defining & Running

```ts
import { expect, test } from 'vitest'

test('parse performance', async ({ bench }) => {
  // bench() registers; .run() executes and returns the result
  const result = await bench('parse', () => {
    const data = JSON.parse('{"key":"value"}')
    use(data) // consume the result — engines may eliminate dead code
  }).run()

  expect(result.throughput.mean).toBeGreaterThan(10_000)
})
```

Run benchmarks:

```bash
vitest bench           # only benchmarks (implicitly enables them)
vitest bench parser    # filter by filename
vitest bench -t JSON   # filter by test name
```

Set `benchmark: { enabled: true }` to run them alongside regular tests in a separate isolated group.

## Comparing Implementations

```ts
test('compare parsers', async ({ bench }) => {
  const result = await bench.compare(
    bench('JSON.parse', () => { JSON.parse(input) }),
    bench('custom', { beforeEach: () => reset() }, () => { customParse(input) }),
    { iterations: 100, time: 1000 }, // shared Tinybench options (last arg)
  )

  // Assertion matchers (delta avoids flaky failures)
  expect(result.get('JSON.parse')).toBeFasterThan(result.get('custom'), { delta: 0.1 })
  expect(result.get('custom')).toBeSlowerThan(result.get('JSON.parse'))
})
```

`bench.compare` interleaves iterations to reduce environmental bias and prints a comparison table after the test.

## Storing & Replaying Baselines

```ts
test('compare against baseline', async ({ bench }) => {
  await bench.compare(
    bench('current', { writeResult: './benchmarks/parse.json' }, () => parse(input)),
    bench.from('previous', './benchmarks/parse.json'),       // reads a stored result, no run
    bench.from('remote', () => fetch(url).then(r => r.json())),
  )
})
```

- `writeResult` overwrites the JSON file on every successful run (no skip-when-cached).
- `bench.from(name, source)` reads a stored result without invoking any function.
- For multi-project workspaces, pass `{ perProject: true }` and use `${projectName}` in `writeResult` paths to collect a cross-project comparison table.

## Stability Notes

- Benchmark files run sequentially and never in parallel; `retry` and the `delta` option reduce flakiness.
- Consume the result inside the bench fn — JS engines eliminate side-effect-free code.
- In Node mode every imported binding goes through Vite's module-runner getter; store hot references locally (`const _parse = parse`), benchmark the built package, or disable `experimental.viteModuleRunner` for the bench project.

## v5 Migration

- `bench` top-level import → `({ bench })` from the test context
- `bench.skip/only/todo` removed → use `test.skip/only/todo` on the surrounding test
- `benchmark.reporters`/`outputFile`/`compare`/`outputJson` and `--compare`/`--outputJson` removed → use `--reporter=json --outputFile` (JSON now has a `benchmarks` field)

## Key Points

- Benchmarks live in `*.bench.ts` files and run inside `test()` via `{ bench }`
- Use `bench.compare` + `toBeFasterThan`/`toBeSlowerThan` (with `delta`) for relative perf
- Persist baselines with `writeResult` and replay with `bench.from`

<!--
Source references:
- https://vitest.dev/guide/benchmarking
- https://vitest.dev/guide/test-context#bench
-->
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall