Skip to content

ever-co/ever-works / .agents/skills/playwright-best-practices

playwright-best-practices

Verified

Use when writing Playwright tests, fixing flaky tests, debugging failures, implementing Page Object Model, configuring CI/CD, optimizing performance, mocking APIs, handling authentication or OAuth, testing accessibility (axe-core), file uploads/downloads, date/time mocking, WebSockets, geolocation, permissions, multi-tab/popup flows, mobile/responsive layouts, touch gestures, GraphQL, error handling, offline mode, multi-user collaboration, third-party services (payments, email verification), console error monitoring, global setup/teardown, test annotations (skip, fixme, slow), test tags (@smoke, @fast, @critical, filtering with --grep), project dependencies, security testing (XSS, CSRF, auth), performance budgets (Web Vitals, Lighthouse), iframes, component testing, canvas/WebGL, service workers/PWA, test coverage, i18n/localization, Electron apps, or browser extension testing. Covers E2E, component, API, visual, accessibility, security, Electron, and extension testing.

FileKindSize
LICENSE.mdmarkdown1.0 KB
README.mdmarkdown9.9 KB
SKILL.mdskill md25.7 KB
advanced/authentication-flows.mdmarkdown10.4 KB
advanced/authentication.mdmarkdown29.4 KB
advanced/clock-mocking.mdmarkdown9.4 KB
advanced/mobile-testing.mdmarkdown10.3 KB
advanced/multi-context.mdmarkdown8.2 KB
advanced/multi-user.mdmarkdown10.9 KB
advanced/network-advanced.mdmarkdown10.1 KB
advanced/third-party.mdmarkdown11.1 KB
architecture/pom-vs-fixtures.mdmarkdown11.4 KB
architecture/test-architecture.mdmarkdown14.3 KB
architecture/when-to-mock.mdmarkdown11.9 KB
browser-apis/browser-apis.mdmarkdown10.1 KB
browser-apis/iframes.mdmarkdown11.1 KB
browser-apis/service-workers.mdmarkdown13.1 KB
browser-apis/websockets.mdmarkdown9.9 KB
core/annotations.mdmarkdown10.4 KB
core/assertions-waiting.mdmarkdown9.1 KB
core/configuration.mdmarkdown13.7 KB
core/fixtures-hooks.mdmarkdown11.2 KB
core/global-setup.mdmarkdown12.4 KB
core/locators.mdmarkdown6.4 KB
core/page-object-model.mdmarkdown7.7 KB
core/projects-dependencies.mdmarkdown8.1 KB
core/test-data.mdmarkdown11.3 KB
core/test-suite-structure.mdmarkdown9.1 KB
core/test-tags.mdmarkdown6.7 KB
debugging/console-errors.mdmarkdown9.3 KB
debugging/debugging.mdmarkdown14.7 KB
debugging/error-testing.mdmarkdown9.2 KB
debugging/flaky-tests.mdmarkdown14.5 KB
frameworks/angular.mdmarkdown18.9 KB
frameworks/nextjs.mdmarkdown13.4 KB
frameworks/react.mdmarkdown17.5 KB
frameworks/vue.mdmarkdown17.3 KB
infrastructure-ci-cd/ci-cd.mdmarkdown10.4 KB
infrastructure-ci-cd/docker.mdmarkdown7.0 KB
infrastructure-ci-cd/github-actions.mdmarkdown15.1 KB
infrastructure-ci-cd/gitlab.mdmarkdown10.9 KB
infrastructure-ci-cd/other-providers.mdmarkdown14.2 KB
infrastructure-ci-cd/parallel-sharding.mdmarkdown11.6 KB
infrastructure-ci-cd/performance.mdmarkdown11.3 KB
infrastructure-ci-cd/reporting.mdmarkdown11.4 KB
infrastructure-ci-cd/test-coverage.mdmarkdown12.0 KB
testing-patterns/accessibility.mdmarkdown8.7 KB
testing-patterns/api-testing.mdmarkdown23.5 KB
testing-patterns/browser-extensions.mdmarkdown13.0 KB
testing-patterns/canvas-webgl.mdmarkdown12.3 KB
testing-patterns/component-testing.mdmarkdown11.0 KB
testing-patterns/drag-drop.mdmarkdown17.4 KB
testing-patterns/electron.mdmarkdown12.8 KB
testing-patterns/file-operations.mdmarkdown10.0 KB
testing-patterns/file-upload-download.mdmarkdown16.5 KB
testing-patterns/forms-validation.mdmarkdown21.3 KB
testing-patterns/graphql-testing.mdmarkdown8.6 KB
testing-patterns/i18n.mdmarkdown12.3 KB
testing-patterns/performance-testing.mdmarkdown11.5 KB
testing-patterns/security-testing.mdmarkdown11.5 KB
testing-patterns/visual-regression.mdmarkdown18.4 KB

infrastructure-ci-cd/reporting.md

View on GitHub
# Test Reports & Artifacts

## Table of Contents

1. [CLI Commands](#cli-commands)
2. [Reporter Configuration](#reporter-configuration)
3. [Custom Reporter](#custom-reporter)
4. [Trace Configuration](#trace-configuration)
5. [Screenshot & Video Settings](#screenshot--video-settings)
6. [Artifact Work Structure](#artifact-work-structure)
7. [CI Artifact Upload](#ci-artifact-upload)
8. [Decision Guide](#decision-guide)
9. [Anti-Patterns](#anti-patterns)
10. [Troubleshooting](#troubleshooting)

> **When to use**: Configuring test output for debugging, CI dashboards, and team visibility.

## CLI Commands

```bash
# Display last HTML report
npx playwright show-report

# Specify reporter
npx playwright test --reporter=html
npx playwright test --reporter=dot           # minimal CI output
npx playwright test --reporter=line          # one line per test
npx playwright test --reporter=json          # machine-readable
npx playwright test --reporter=junit         # CI integration

# Combine reporters
npx playwright test --reporter=dot,html

# Merge sharded reports
npx playwright merge-reports --reporter=html ./blob-report
```

## Reporter Configuration

### Environment-Based Setup

```typescript
// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
	reporter: process.env.CI
		? [['dot'], ['html', { open: 'never' }], ['junit', { outputFile: 'results/junit.xml' }], ['github']]
		: [['list'], ['html', { open: 'on-failure' }]]
});
```

### Reporter Types

| Reporter | Output                | Use Case            |
| -------- | --------------------- | ------------------- |
| `list`   | One line per test     | Local development   |
| `line`   | Single updating line  | Local, less verbose |
| `dot`    | `.` pass, `F` fail    | CI logs             |
| `html`   | Interactive HTML page | Post-run analysis   |
| `json`   | Machine-readable JSON | Custom tooling      |
| `junit`  | JUnit XML             | CI platforms        |
| `github` | PR annotations        | GitHub Actions      |
| `blob`   | Binary archive        | Shard merging       |

### JSON Output to File

```typescript
import { defineConfig } from '@playwright/test';

export default defineConfig({
	reporter: [['json', { outputFile: 'results/output.json' }]]
});
```

### JUnit Customization

```typescript
import { defineConfig } from '@playwright/test';

export default defineConfig({
	reporter: [
		[
			'junit',
			{
				outputFile: 'results/junit.xml',
				stripANSIControlSequences: true,
				includeProjectInTestName: true
			}
		]
	]
});
```

## Custom Reporter

Build custom reporters for Slack notifications, database logging, or dashboards.

```typescript
// reporters/notification-reporter.ts
import type { FullResult, Reporter, TestCase, TestResult } from '@playwright/test/reporter';

class NotificationReporter implements Reporter {
	private passed = 0;
	private failed = 0;
	private skipped = 0;
	private failures: string[] = [];

	onTestEnd(test: TestCase, result: TestResult) {
		switch (result.status) {
			case 'passed':
				this.passed++;
				break;
			case 'failed':
			case 'timedOut':
				this.failed++;
				this.failures.push(`${test.title}: ${result.error?.message?.split('\n')[0]}`);
				break;
			case 'skipped':
				this.skipped++;
				break;
		}
	}

	async onEnd(result: FullResult) {
		const total = this.passed + this.failed + this.skipped;
		const status = this.failed > 0 ? 'FAILED' : 'PASSED';
		const message = [
			`Tests ${status}`,
			`Passed: ${this.passed} | Failed: ${this.failed} | Skipped: ${this.skipped}`,
			`Duration: ${(result.duration / 1000).toFixed(1)}s`
		];

		if (this.failures.length > 0) {
			message.push('', 'Failures:');
			this.failures.slice(0, 5).forEach((f) => message.push(`  - ${f}`));
			if (this.failures.length > 5) {
				message.push(`  ...and ${this.failures.length - 5} more`);
			}
		}

		const webhookUrl = process.env.NOTIFICATION_WEBHOOK;
		if (webhookUrl) {
			const controller = new AbortController();
			const timeout = setTimeout(() => controller.abort(), 5000);
			try {
				await fetch(webhookUrl, {
					method: 'POST',
					headers: { 'Content-Type': 'application/json' },
					body: JSON.stringify({ text: message.join('\n') }),
					signal: controller.signal
				});
			} catch (error) {
				// Intentionally swallow notifier failures to avoid blocking test completion
				console.warn('Webhook notification failed:', error.message);
			} finally {
				clearTimeout(timeout);
			}
		}
	}
}

export default NotificationReporter;
```

**Register custom reporter:**

```typescript
import { defineConfig } from '@playwright/test';

export default defineConfig({
	reporter: [['dot'], ['html', { open: 'never' }], ['./reporters/notification-reporter.ts']]
});
```

## Trace Configuration

Traces capture actions, network requests, DOM snapshots, and console logs.

```typescript
import { defineConfig } from '@playwright/test';

export default defineConfig({
	retries: process.env.CI ? 2 : 0,
	use: {
		trace: 'on-first-retry'
	}
});
```

### Trace Options

| Value                       | Behavior                         | Overhead |
| --------------------------- | -------------------------------- | -------- |
| `'off'`                     | Never records                    | None     |
| `'on'`                      | Every test                       | High     |
| `'on-first-retry'`          | On first retry after failure     | Minimal  |
| `'retain-on-failure'`       | Records all, keeps failures      | Medium   |
| `'retain-on-first-failure'` | Records all, keeps first failure | Medium   |

### Viewing Traces

```bash
# Local trace viewer
npx playwright show-trace results/my-test/trace.zip

# From HTML report (click Traces tab)
npx playwright show-report

# Online viewer: https://trace.playwright.dev
```

## Screenshot & Video Settings

```typescript
import { defineConfig } from '@playwright/test';

export default defineConfig({
	use: {
		screenshot: 'only-on-failure',
		video: 'retain-on-failure'
	}
});
```

### Video with Custom Size

```typescript
use: {
  video: {
    mode: 'retain-on-failure',
    size: { width: 1280, height: 720 },
  },
},
```

### Screenshot Options

| Value               | Captures     | Disk Cost |
| ------------------- | ------------ | --------- |
| `'off'`             | Never        | None      |
| `'on'`              | Every test   | High      |
| `'only-on-failure'` | Failed tests | Low       |

### Video Options

| Value                 | Records    | Keeps   | Disk Cost |
| --------------------- | ---------- | ------- | --------- |
| `'off'`               | Never      | —       | None      |
| `'on'`                | Every test | All     | Very high |
| `'on-first-retry'`    | On retry   | Retried | Low       |
| `'retain-on-failure'` | Every test | Failed  | Medium    |

## Artifact Work Structure

```text
test-results/
├── checkout-test-chromium/
│   ├── trace.zip
│   ├── test-failed-1.png
│   └── video.webm
├── login-test-firefox/
│   ├── trace.zip
│   └── test-failed-1.png
└── junit.xml

playwright-report/
├── index.html
└── data/

blob-report/
└── report-1.zip
```

## CI Artifact Upload

### GitHub Actions

```yaml
- uses: actions/upload-artifact@v4
  if: ${{ !cancelled() }}
  with:
      name: playwright-report
      path: playwright-report/
      retention-days: 14

- uses: actions/upload-artifact@v4
  if: failure()
  with:
      name: test-traces
      path: |
          test-results/**/trace.zip
          test-results/**/*.png
          test-results/**/*.webm
      retention-days: 7
```

## Decision Guide

| Scenario               | Reporter Configuration                                |
| ---------------------- | ----------------------------------------------------- |
| Local development      | `[['list'], ['html', { open: 'on-failure' }]]`        |
| GitHub Actions         | `[['dot'], ['html'], ['github']]`                     |
| GitLab CI              | `[['dot'], ['html'], ['junit']]`                      |
| Azure DevOps / Jenkins | `[['dot'], ['html'], ['junit']]`                      |
| Sharded CI             | `[['blob'], ['github']]`                              |
| Custom dashboard       | `[['json', { outputFile: '...' }]]` + custom reporter |

| Artifact    | When to Collect  | Retention | Upload Condition          |
| ----------- | ---------------- | --------- | ------------------------- |
| HTML report | Always           | 14 days   | `if: ${{ !cancelled() }}` |
| Traces      | On failure       | 7 days    | `if: failure()`           |
| Screenshots | On failure       | 7 days    | `if: failure()`           |
| Videos      | On failure       | 7 days    | `if: failure()`           |
| JUnit XML   | Always           | 14 days   | `if: ${{ !cancelled() }}` |
| Blob report | Always (sharded) | 1 day     | `if: ${{ !cancelled() }}` |

## Anti-Patterns

| Anti-Pattern                        | Problem                                   | Solution                              |
| ----------------------------------- | ----------------------------------------- | ------------------------------------- |
| No reporter configured              | Default `list` only; no persistent report | Configure `html` + CI reporter        |
| `trace: 'on'` in CI                 | Massive artifacts, slow uploads           | Use `trace: 'on-first-retry'`         |
| `video: 'on'` in CI                 | Enormous storage, slower tests            | Use `video: 'retain-on-failure'`      |
| Upload artifacts only on failure    | No report when tests pass                 | Upload with `if: ${{ !cancelled() }}` |
| No retention limits                 | CI storage fills quickly                  | Set `retention-days: 7-14`            |
| Only `dot` reporter                 | Cannot drill into failures                | Pair `dot` with `html`                |
| JUnit to stdout                     | Interferes with console output            | Write to file                         |
| Blocking `onEnd` in custom reporter | Slow HTTP calls delay pipeline            | Use `Promise.race` with timeout       |

## Troubleshooting

### Empty HTML Report

Check reporter config. HTML report defaults to `playwright-report/`:

```typescript
import { defineConfig } from '@playwright/test';

export default defineConfig({
	reporter: [['html', { outputFolder: 'playwright-report', open: 'never' }]]
});
```

### Traces Too Large

Switch from `trace: 'on'` to `'on-first-retry'` with retries enabled:

```typescript
import { defineConfig } from '@playwright/test';

export default defineConfig({
	retries: process.env.CI ? 2 : 0,
	use: {
		trace: 'on-first-retry'
	}
});
```

### JUnit XML Not Recognized

Ensure path matches CI configuration:

```typescript
reporter: [['junit', { outputFile: 'results/junit.xml' }]],
```

```yaml
# GitHub Actions
- uses: dorny/test-reporter@latest
  with:
    path: results/junit.xml
    reporter: java-junit

# Azure DevOps
- task: PublishTestResults@latest
  inputs:
    testResultsFiles: 'results/junit.xml'

# Jenkins
junit 'results/junit.xml'
```

### Empty Merged Report

Use `blob` reporter for sharded runs (not `html`):

```typescript
import { defineConfig } from '@playwright/test';

export default defineConfig({
	reporter: process.env.CI ? [['blob'], ['dot']] : [['html', { open: 'on-failure' }]]
});
```

### Missing Screenshots in Report

Enable screenshots and keep both works:

```typescript
use: {
  screenshot: 'only-on-failure',
},
```

The HTML report embeds screenshots from `test-results/`. Deleting that work removes screenshots from the report.
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall