testdino-hq/playwright-skill / ci
playwright-ci
VerifiedProduction-ready CI/CD configurations for Playwright — GitHub Actions, GitLab CI, CircleCI, Azure DevOps, Jenkins, Docker, parallel sharding, reporting, code coverage, and global setup/teardown.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 2.1 KB |
| ci-github-actions.md | markdown | 18.0 KB |
| ci-gitlab.md | markdown | 10.8 KB |
| ci-other.md | markdown | 15.1 KB |
| docker-and-containers.md | markdown | 14.3 KB |
| global-setup-teardown.md | markdown | 17.7 KB |
| parallel-and-sharding.md | markdown | 16.1 KB |
| projects-and-dependencies.md | markdown | 19.1 KB |
| reporting-and-artifacts.md | markdown | 20.1 KB |
| test-coverage.md | markdown | 13.8 KB |
reporting-and-artifacts.md
View on GitHub# Reporting and Artifacts
> **When to use**: Configuring test output for local debugging, CI dashboards, and team visibility. Every project needs a reporting strategy from day one.
## Quick Reference
```bash
# View the last HTML report
npx playwright show-report
# Run with specific reporter
npx playwright test --reporter=html
npx playwright test --reporter=dot # minimal CI output
npx playwright test --reporter=line # one line per test
npx playwright test --reporter=json # machine-readable
npx playwright test --reporter=junit # CI integration
# Multiple reporters via CLI
npx playwright test --reporter=dot,html
# Merge shard reports
npx playwright merge-reports --reporter=html ./blob-report
```
## Patterns
### Pattern 1: Multi-Reporter Configuration
**Use when**: Every project. You always want at least two reporters: one for humans, one for CI.
**Avoid when**: Never -- always configure reporters.
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [
// CI: machine-readable + human-readable + CI annotations
['dot'], // minimal console output
['html', { open: 'never' }], // browsable report (uploaded as artifact)
['junit', { outputFile: 'test-results/junit.xml' }], // CI test tab integration
['github'], // PR annotations (GitHub Actions only)
]
: [
// Local: detailed console + auto-opening report
['list'], // verbose console output
['html', { open: 'on-failure' }], // auto-open on failure
],
});
```
### Pattern 2: Built-in Reporters in Detail
**Use when**: Choosing the right reporter for your context.
| Reporter | Output | Best For |
|---|---|---|
| `list` | One line per test with pass/fail | Local development |
| `line` | Updates a single line as tests complete | Local, less verbose |
| `dot` | Single dot per test: `.` pass, `F` fail | CI logs (minimal) |
| `html` | Interactive HTML page with traces | Post-run analysis |
| `json` | Machine-readable JSON to stdout or file | Custom tooling, dashboards |
| `junit` | JUnit XML | CI platforms (Azure DevOps, Jenkins, CircleCI) |
| `github` | GitHub Actions annotations | GitHub PRs |
| `blob` | Binary archive for shard merging | Sharded CI runs |
**JSON reporter -- write to file:**
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['json', { outputFile: 'test-results/results.json' }],
],
});
```
**JUnit reporter -- customize output:**
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['junit', {
outputFile: 'test-results/junit.xml',
stripANSIControlSequences: true,
includeProjectInTestName: true,
}],
],
});
```
### Pattern 2b: Trace Retention For Flaky Tests (Playwright 1.59+)
**Use when**: A test sometimes fails and sometimes passes on retry, and you need artifacts from both attempts to compare behavior.
**Avoid when**: Trace size matters more than diagnosis depth. In that case, keep using `'on-first-retry'`.
Playwright 1.59 adds a new trace mode, `'retain-on-failure-and-retries'`, which records each run and keeps all traces when any attempt fails.
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
retries: process.env.CI ? 2 : 0,
use: {
trace: process.env.CI
? 'retain-on-failure-and-retries'
: 'on-first-retry',
},
});
```
This is excellent for flaky-test analysis because you can compare the failing run against a passing retry instead of guessing what changed between attempts.
### Pattern 2c: Scoped HAR Artifacts For a Flow (Playwright 1.60+)
**Use when**: You want a network archive (HAR) attached to the report for one specific flow — e.g. a flaky third-party integration — without recording HAR for the entire context.
**Avoid when**: A trace already gives you the network waterfall you need (traces include network detail). Reach for a dedicated HAR only when you need a portable `.har` to replay or hand to another tool.
Playwright 1.60 adds `tracing.startHar()` / `tracing.stopHar()`, which record HAR on demand inside the test. Capture only the suspect flow and attach it to the report:
```ts
import { test } from '@playwright/test';
test('payment integration', async ({ context, page }, testInfo) => {
const harPath = testInfo.outputPath('payment.har');
await context.tracing.startHar(harPath, { urlFilter: '**/payments/**' });
await page.goto('/checkout');
await page.getByRole('button', { name: 'Pay' }).click();
await context.tracing.stopHar();
await testInfo.attach('payment-network', { path: harPath, contentType: 'application/json' });
});
```
The attached HAR shows up alongside traces and screenshots in the HTML report and CI artifacts. See [core/network-mocking.md](../core/network-mocking.md#on-demand-har-recording-in-tracing-playwright-160) for the full API and `await using` cleanup.
### Pattern 2d: Aria and Screen Snapshots in Traces (Playwright 1.63+)
Traces recorded DOM snapshots only. 1.63 lets you choose which snapshot kinds are captured.
```typescript
export default defineConfig({
use: {
trace: {
mode: 'on',
snapshots: { dom: true, aria: true, screen: true }
},
},
});
```
`aria` snapshots make a trace readable by an agent or a script without rendering the DOM, which is the difference between a human opening Trace Viewer and a bot diagnosing a failure in CI. `screen` captures rendered frames, which is the most expensive of the 3.
**Turn `screen` off for routine CI capture.** With `trace: 'on-first-retry'` on a large suite, screen snapshots are the line item that pushes artifacts past upload limits. Keep `dom` and `aria`, add `screen` when investigating a specific visual failure.
### Pattern 2e: Appending Reporters and New Reporter Options (Playwright 1.62 to 1.63)
`--reporter` on the CLI **replaces** the configured reporter list, which silently drops a custom reporter you rely on. 1.63 adds an append form:
```bash
npx playwright test --add-reporter json
```
Use `--add-reporter` in any pipeline where the config already registers a reporter that must keep running — a quarantine reporter using `preprocess()`, for instance. See [core/dynamic-test-selection.md](../core/dynamic-test-selection.md).
Other reporter changes worth knowing:
| Feature | Version | What it does |
|---|---|---|
| `perfetto` reporter | 1.63 | Emits Trace Event Format, openable in Perfetto UI for timing analysis across a run |
| `omitTags` option | 1.63 | Suppresses auto-appended tags in reporter output |
| `mergeFiles` (HTML) | 1.62 | Groups test blocks in the HTML report: `reporter: [['html', { mergeFiles: true }]]` |
| Duration waterfall | 1.63 | HTML report shows step durations as a waterfall alongside the step list |
### Pattern 3: Custom Reporter
**Use when**: Built-in reporters don't meet your needs -- you want Slack notifications, database logging, or custom dashboards.
**Avoid when**: A built-in reporter or existing third-party reporter covers your case.
```ts
// reporters/slack-reporter.ts
import type {
FullConfig,
FullResult,
Reporter,
Suite,
TestCase,
TestResult,
} from '@playwright/test/reporter';
class SlackReporter implements Reporter {
private passed = 0;
private failed = 0;
private skipped = 0;
private failures: string[] = [];
onTestEnd(test: TestCase, result: TestResult) {
switch (result.status) {
case 'passed':
this.passed++;
break;
case 'failed':
case 'timedOut':
this.failed++;
this.failures.push(`${test.title}: ${result.error?.message?.split('\n')[0]}`);
break;
case 'skipped':
this.skipped++;
break;
}
}
async onEnd(result: FullResult) {
const total = this.passed + this.failed + this.skipped;
const emoji = this.failed > 0 ? ':red_circle:' : ':large_green_circle:';
const text = [
`${emoji} *Playwright Tests*: ${result.status}`,
`Passed: ${this.passed} | Failed: ${this.failed} | Skipped: ${this.skipped} | Total: ${total}`,
`Duration: ${(result.duration / 1000).toFixed(1)}s`,
];
if (this.failures.length > 0) {
text.push('', '*Failures:*');
this.failures.slice(0, 5).forEach((f) => text.push(` - ${f}`));
if (this.failures.length > 5) {
text.push(` ...and ${this.failures.length - 5} more`);
}
}
const webhookUrl = process.env.SLACK_WEBHOOK_URL;
if (webhookUrl) {
await fetch(webhookUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ text: text.join('\n') }),
});
}
}
}
export default SlackReporter;
```
**Register the custom reporter:**
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['dot'],
['html', { open: 'never' }],
['./reporters/slack-reporter.ts'],
],
});
```
### Pattern 4: Trace File Management
**Use when**: Debugging test failures. Traces capture a complete timeline of actions, network requests, DOM snapshots, and console logs.
**Avoid when**: Never disable traces entirely in CI -- the on-first-retry setting has minimal overhead.
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
retries: process.env.CI ? 2 : 0,
use: {
// 'on-first-retry': records trace only when a test fails and retries.
// Minimal overhead on passing tests, full debugging on failures.
trace: 'on-first-retry',
},
});
```
**Trace options:**
| Value | Records trace | When | Overhead |
|---|---|---|---|
| `'off'` | Never | -- | None |
| `'on'` | Every test | Always | High (large files) |
| `'on-first-retry'` | On first retry after failure | Retries only | Minimal |
| `'retain-on-failure'` | Every test, keeps only failures | Failures | Medium |
| `'retain-on-first-failure'` | Every test, keeps only first failure | First failure | Medium |
**Viewing traces:**
```bash
# Open trace viewer locally
npx playwright show-trace test-results/my-test/trace.zip
# Open trace from HTML report (click "Traces" tab in the report)
npx playwright show-report
# Online trace viewer (upload trace.zip)
# https://trace.playwright.dev
```
### Pattern 5: Screenshot and Video Configuration
**Use when**: Visual evidence of test failures is valuable for debugging or bug reports.
**Avoid when**: Never disable screenshots in CI -- the on-failure setting is cheap.
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
use: {
// Screenshots
screenshot: 'only-on-failure', // capture final state on failure
// Video
video: 'retain-on-failure', // record all, keep only failures
// Video size (optional -- smaller = less disk)
video: {
mode: 'retain-on-failure',
size: { width: 1280, height: 720 },
},
},
});
```
**Screenshot options:**
| Value | Captures | Disk cost |
|---|---|---|
| `'off'` | Never | None |
| `'on'` | Every test (at end) | High |
| `'only-on-failure'` | Failed tests only | Low |
**Video options:**
| Value | Records | Keeps | Disk cost |
|---|---|---|---|
| `'off'` | Never | -- | None |
| `'on'` | Every test | All | Very high |
| `'on-first-retry'` | On retry | Retried tests | Low |
| `'on-all-retries'` (1.61+) | On every retry | All retries | Low-medium |
| `'retain-on-failure'` | Every test | Failed only | Medium |
| `'retain-on-first-failure'` (1.61+) | First run of every test (not retries) | Failed first runs | Medium |
| `'retain-on-failure-and-retries'` (1.61+) | Every run incl. retries | Failures + their retries | Medium |
The 1.61 modes mirror the `trace` modes, so video and trace policies can finally match. `'retain-on-failure-and-retries'` is the best default for flaky-test hunting: you get video of both the original failure *and* the retry, so you can compare the two runs.
**Video quality and styling (Playwright 1.64+)**: `video` accepts an object with `fps`, and action decorations are styled with plain CSS:
```typescript
use: {
video: {
mode: 'retain-on-failure',
size: { width: 1280, height: 720 },
fps: 15, // lower fps for CI artifacts; 30 to 60 for demos
},
},
```
Videos are now encoded with **VP9 instead of VP8**: less CPU and smaller files at the same or better quality, so existing `retain-on-failure` setups get cheaper with no config change. Firefox and WebKit currently capture up to 25 fps regardless of the setting. For demo-quality recordings with styled action callouts, see [screenshots-and-media.md](../playwright-cli/screenshots-and-media.md#annotated-screencasts).
### Pattern 6: Artifact Organization for CI
**Use when**: Keeping test artifacts organized and accessible in CI.
**Recommended directory structure:**
```
test-results/ # Playwright's default output directory
├── my-test-chromium/
│ ├── trace.zip # Trace file
│ ├── test-failed-1.png # Screenshot
│ └── video.webm # Video recording
├── another-test-firefox/
│ ├── trace.zip
│ └── test-failed-1.png
└── junit.xml # JUnit report (if configured)
playwright-report/ # HTML report directory
├── index.html
└── data/
└── ...
blob-report/ # Blob report for shard merging
└── report-1.zip
```
**GitHub Actions artifact upload:**
```yaml
# Upload HTML report (always -- useful even when tests pass)
- uses: actions/upload-artifact@v7
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 14
# Upload traces and screenshots (only on failure -- saves storage)
- uses: actions/upload-artifact@v7
if: failure()
with:
name: test-traces
path: |
test-results/**/trace.zip
test-results/**/*.png
test-results/**/*.webm
retention-days: 7
```
### Pattern 7: Allure Integration
**Use when**: Your team uses Allure for test reporting across multiple test frameworks.
**Avoid when**: The built-in HTML reporter meets your needs (it usually does).
```bash
# Install Allure reporter
npm install -D allure-playwright
```
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['line'],
['allure-playwright', {
detail: true,
outputFolder: 'allure-results',
suiteTitle: true,
}],
],
});
```
```bash
# Generate and view Allure report
npx allure generate allure-results -o allure-report --clean
npx allure open allure-report
# Or use Allure CLI
allure serve allure-results
```
**Add Allure metadata to tests:**
```ts
import { test, expect } from '@playwright/test';
import { allure } from 'allure-playwright';
test('checkout flow', async ({ page }) => {
await allure.epic('E-Commerce');
await allure.feature('Checkout');
await allure.story('Credit Card Payment');
await allure.severity('critical');
await page.goto('/checkout');
// ... test implementation
});
```
## Decision Guide
| Scenario | Reporter Configuration | Why |
|---|---|---|
| Local development | `[['list'], ['html', { open: 'on-failure' }]]` | Verbose console + auto-opening report on failure |
| GitHub Actions | `[['dot'], ['html'], ['github']]` | Minimal logs + report artifact + PR annotations |
| GitLab CI | `[['dot'], ['html'], ['junit']]` | Minimal logs + report artifact + test tab |
| Azure DevOps / Jenkins | `[['dot'], ['html'], ['junit']]` | JUnit for native test results integration |
| Sharded CI | `[['blob'], ['github']]` | Blob for merging; github for PR annotations |
| Team uses Allure | `[['line'], ['allure-playwright']]` | Cross-framework reporting consistency |
| Custom dashboard | `[['json', { outputFile: '...' }]]` + custom reporter | JSON for data, custom for notifications |
| Artifact | When to Collect | Retention | Upload Condition |
|---|---|---|---|
| HTML report | Always | 14 days | `if: ${{ !cancelled() }}` |
| Traces (`.zip`) | On failure | 7 days | `if: failure()` |
| Screenshots (`.png`) | On failure | 7 days | `if: failure()` |
| Videos (`.webm`) | On failure | 7 days | `if: failure()` |
| JUnit XML | Always | 14 days | `if: ${{ !cancelled() }}` |
| Blob report | Always (sharded) | 1 day | `if: ${{ !cancelled() }}` |
## Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
| No reporter configured | Default `list` only; no persistent report | Always configure `html` + one CI reporter |
| `trace: 'on'` in CI | Massive artifacts (50-100 MB per test), slow uploads | Use `trace: 'on-first-retry'` |
| `video: 'on'` in CI | Enormous storage cost; slows test execution | Use `video: 'retain-on-failure'` |
| Only uploading artifacts on failure | No report when tests pass; can't verify results | Upload with `if: ${{ !cancelled() }}` (always) |
| No retention limits on artifacts | CI storage fills up within weeks | Set `retention-days: 7-14` |
| Using only `dot` reporter with no HTML | Can't drill into failures after the run | Always pair `dot` with `html` in CI |
| JUnit output to stdout | Interferes with console output; hard to parse | Write to file: `['junit', { outputFile: 'results/junit.xml' }]` |
| Custom reporter that blocks `onEnd` | Slow Slack/HTTP calls delay pipeline completion | Use `Promise.race` with a timeout in custom reporters |
## Troubleshooting
### HTML report is empty or missing tests
**Cause**: Another reporter is conflicting, or `outputFolder` was overridden to a non-default path.
**Fix**: Check your reporter config. The HTML report defaults to `playwright-report/`:
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [['html', { outputFolder: 'playwright-report', open: 'never' }]],
});
```
### Traces are too large for CI artifact upload
**Cause**: `trace: 'on'` records every test, even passing ones.
**Fix**: Switch to `'on-first-retry'` and ensure `retries > 0` in CI:
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
retries: process.env.CI ? 2 : 0,
use: {
trace: 'on-first-retry',
},
});
```
### JUnit XML not recognized by CI platform
**Cause**: Output path doesn't match what the CI task expects, or the file is empty.
**Fix**: Ensure the path matches your CI configuration:
```ts
// playwright.config.ts -- the outputFile path
reporter: [['junit', { outputFile: 'test-results/junit.xml' }]],
```
```yaml
# GitHub Actions
- uses: dorny/test-reporter@v3
with:
name: Playwright Tests
path: test-results/junit.xml
reporter: java-junit
# Azure DevOps
- task: PublishTestResults@2
inputs:
testResultsFiles: 'test-results/junit.xml'
# Jenkins
junit 'test-results/junit.xml'
```
### `merge-reports` produces empty report
**Cause**: Shards are using `html` reporter instead of `blob`. Only `blob` output can be merged.
**Fix**: Use blob reporter for sharded runs:
```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['blob'], ['dot']] // blob for merge, dot for console
: [['html', { open: 'on-failure' }]],
});
```
### Screenshots not appearing in HTML report
**Cause**: `screenshot: 'off'` or screenshots are in `test-results/` but not linked to the report.
**Fix**: Enable screenshots and ensure both directories are available:
```ts
use: {
screenshot: 'only-on-failure',
},
```
The HTML report automatically embeds screenshots from `test-results/`. If you move or delete `test-results/`, screenshots will be missing from the report.
## Related
- [ci/ci-github-actions.md](ci-github-actions.md) -- artifact upload in GitHub Actions
- [ci/ci-gitlab.md](ci-gitlab.md) -- artifact configuration in GitLab
- [ci/parallel-and-sharding.md](parallel-and-sharding.md) -- blob reporter for sharded runs
- [core/configuration.md](../core/configuration.md) -- trace, screenshot, video settings
- [core/debugging.md](../core/debugging.md) -- using traces and screenshots for debugging
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |