Skip to content

ever-co/ever-works / .agents/skills/playwright-best-practices

playwright-best-practices

Verified

Use when writing Playwright tests, fixing flaky tests, debugging failures, implementing Page Object Model, configuring CI/CD, optimizing performance, mocking APIs, handling authentication or OAuth, testing accessibility (axe-core), file uploads/downloads, date/time mocking, WebSockets, geolocation, permissions, multi-tab/popup flows, mobile/responsive layouts, touch gestures, GraphQL, error handling, offline mode, multi-user collaboration, third-party services (payments, email verification), console error monitoring, global setup/teardown, test annotations (skip, fixme, slow), test tags (@smoke, @fast, @critical, filtering with --grep), project dependencies, security testing (XSS, CSRF, auth), performance budgets (Web Vitals, Lighthouse), iframes, component testing, canvas/WebGL, service workers/PWA, test coverage, i18n/localization, Electron apps, or browser extension testing. Covers E2E, component, API, visual, accessibility, security, Electron, and extension testing.

FileKindSize
LICENSE.mdmarkdown1.0 KB
README.mdmarkdown9.9 KB
SKILL.mdskill md25.7 KB
advanced/authentication-flows.mdmarkdown10.4 KB
advanced/authentication.mdmarkdown29.4 KB
advanced/clock-mocking.mdmarkdown9.4 KB
advanced/mobile-testing.mdmarkdown10.3 KB
advanced/multi-context.mdmarkdown8.2 KB
advanced/multi-user.mdmarkdown10.9 KB
advanced/network-advanced.mdmarkdown10.1 KB
advanced/third-party.mdmarkdown11.1 KB
architecture/pom-vs-fixtures.mdmarkdown11.4 KB
architecture/test-architecture.mdmarkdown14.3 KB
architecture/when-to-mock.mdmarkdown11.9 KB
browser-apis/browser-apis.mdmarkdown10.1 KB
browser-apis/iframes.mdmarkdown11.1 KB
browser-apis/service-workers.mdmarkdown13.1 KB
browser-apis/websockets.mdmarkdown9.9 KB
core/annotations.mdmarkdown10.4 KB
core/assertions-waiting.mdmarkdown9.1 KB
core/configuration.mdmarkdown13.7 KB
core/fixtures-hooks.mdmarkdown11.2 KB
core/global-setup.mdmarkdown12.4 KB
core/locators.mdmarkdown6.4 KB
core/page-object-model.mdmarkdown7.7 KB
core/projects-dependencies.mdmarkdown8.1 KB
core/test-data.mdmarkdown11.3 KB
core/test-suite-structure.mdmarkdown9.1 KB
core/test-tags.mdmarkdown6.7 KB
debugging/console-errors.mdmarkdown9.3 KB
debugging/debugging.mdmarkdown14.7 KB
debugging/error-testing.mdmarkdown9.2 KB
debugging/flaky-tests.mdmarkdown14.5 KB
frameworks/angular.mdmarkdown18.9 KB
frameworks/nextjs.mdmarkdown13.4 KB
frameworks/react.mdmarkdown17.5 KB
frameworks/vue.mdmarkdown17.3 KB
infrastructure-ci-cd/ci-cd.mdmarkdown10.4 KB
infrastructure-ci-cd/docker.mdmarkdown7.0 KB
infrastructure-ci-cd/github-actions.mdmarkdown15.1 KB
infrastructure-ci-cd/gitlab.mdmarkdown10.9 KB
infrastructure-ci-cd/other-providers.mdmarkdown14.2 KB
infrastructure-ci-cd/parallel-sharding.mdmarkdown11.6 KB
infrastructure-ci-cd/performance.mdmarkdown11.3 KB
infrastructure-ci-cd/reporting.mdmarkdown11.4 KB
infrastructure-ci-cd/test-coverage.mdmarkdown12.0 KB
testing-patterns/accessibility.mdmarkdown8.7 KB
testing-patterns/api-testing.mdmarkdown23.5 KB
testing-patterns/browser-extensions.mdmarkdown13.0 KB
testing-patterns/canvas-webgl.mdmarkdown12.3 KB
testing-patterns/component-testing.mdmarkdown11.0 KB
testing-patterns/drag-drop.mdmarkdown17.4 KB
testing-patterns/electron.mdmarkdown12.8 KB
testing-patterns/file-operations.mdmarkdown10.0 KB
testing-patterns/file-upload-download.mdmarkdown16.5 KB
testing-patterns/forms-validation.mdmarkdown21.3 KB
testing-patterns/graphql-testing.mdmarkdown8.6 KB
testing-patterns/i18n.mdmarkdown12.3 KB
testing-patterns/performance-testing.mdmarkdown11.5 KB
testing-patterns/security-testing.mdmarkdown11.5 KB
testing-patterns/visual-regression.mdmarkdown18.4 KB

infrastructure-ci-cd/parallel-sharding.md

View on GitHub
# Sharding and Parallel Execution

## Table of Contents

1. [CLI Commands](#cli-commands)
2. [Patterns](#patterns)
3. [Decision Guide](#decision-guide)
4. [Anti-Patterns](#anti-patterns)
5. [Troubleshooting](#troubleshooting)

> **When to use**: Speeding up test suites by running tests concurrently on one machine (workers) or splitting across multiple CI jobs (sharding).

## CLI Commands

```bash
# Parallelism within one machine
npx playwright test --workers=4
npx playwright test --workers=50%

# Splitting across CI jobs
npx playwright test --shard=1/4
npx playwright test --shard=2/4

# Merging shard outputs
npx playwright merge-reports ./blob-report
npx playwright merge-reports --reporter=html,json ./blob-report

# Override config for single run
npx playwright test --fully-parallel
```

## Patterns

### Worker Configuration

**Use when**: Controlling concurrent test execution on a single machine.

```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
	// Tests WITHIN a file also run in parallel
	fullyParallel: true,

	// Worker count options:
	// - undefined: auto-detect (half CPU cores)
	// - number: fixed count
	// - string: percentage of cores
	workers: process.env.CI ? '50%' : undefined
});
```

**`fullyParallel` behavior:**

| Setting                          | Files parallel | Tests in file parallel |
| -------------------------------- | -------------- | ---------------------- |
| `fullyParallel: false` (default) | Yes            | No (serial)            |
| `fullyParallel: true`            | Yes            | Yes                    |

**Serial execution for specific files:**

```ts
// tests/checkout-flow.spec.ts
import { test, expect } from '@playwright/test';

test.describe.configure({ mode: 'serial' });

test('add items to cart', async ({ page }) => {
	// ...
});

test('complete payment', async ({ page }) => {
	// ...
});
```

### Sharding Across CI Machines

**Use when**: Suite exceeds 5 minutes even with maximum workers.

```bash
# Job 1            Job 2            Job 3            Job 4
--shard=1/4        --shard=2/4      --shard=3/4      --shard=4/4
```

**Config for sharded runs:**

```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
	fullyParallel: true,
	workers: process.env.CI ? '50%' : undefined,

	reporter: process.env.CI ? [['blob'], ['github']] : [['html', { open: 'on-failure' }]]
});
```

### Merging Shard Reports

**Use when**: Combining blob reports from multiple shards into a unified report.

```bash
# Merge all blobs into HTML
npx playwright merge-reports --reporter=html ./all-blob-reports

# Multiple formats
npx playwright merge-reports --reporter=html,json,junit ./all-blob-reports

# Custom output location
PLAYWRIGHT_HTML_REPORT=merged-report npx playwright merge-reports --reporter=html ./all-blob-reports
```

**GitHub Actions merge job:**

```yaml
merge-reports:
    if: ${{ !cancelled() }}
    needs: test
    runs-on: ubuntu-latest
    steps:
        - uses: actions/checkout@v4
        - run: npm ci

        - uses: actions/download-artifact@v4
          with:
              path: all-blob-reports
              pattern: blob-report-*
              merge-multiple: true

        - run: npx playwright merge-reports --reporter=html ./all-blob-reports

        - uses: actions/upload-artifact@v4
          with:
              name: playwright-report
              path: playwright-report/
              retention-days: 14
```

### Worker-Scoped Fixtures

**Use when**: Expensive resources (DB connections, auth tokens) should be created once per worker, not per test.

```ts
// fixtures.ts
import { test as base } from '@playwright/test';

type WorkerFixtures = {
	dbClient: DatabaseClient;
	apiToken: string;
};

export const test = base.extend<{}, WorkerFixtures>({
	dbClient: [
		async ({}, use) => {
			const client = await DatabaseClient.connect(process.env.DB_URL!);
			await use(client);
			await client.disconnect();
		},
		{ scope: 'worker' }
	],

	apiToken: [
		async ({}, use, workerInfo) => {
			const res = await fetch(`${process.env.API_URL}/auth`, {
				method: 'POST',
				headers: { 'Content-Type': 'application/json' },
				body: JSON.stringify({
					user: `test-user-${workerInfo.workerIndex}`,
					password: process.env.TEST_PASSWORD
				})
			});
			const { token } = await res.json();
			await use(token);
		},
		{ scope: 'worker' }
	]
});

export { expect } from '@playwright/test';
```

### Test Isolation for Parallelism

**Use when**: Preparing tests to run without interference.

Each test must create its own state. No test should depend on or modify shared state.

```ts
// BAD: Shared user causes race conditions
test('edit settings', async ({ page }) => {
	await page.goto('/users/test-user/settings');
	await page.getByLabel('Email').fill('new@example.com');
	await page.getByRole('button', { name: 'Save' }).click();
});

// GOOD: Unique user per test
test('edit settings', async ({ page, request }) => {
	const res = await request.post('/api/users', {
		data: { name: `user-${Date.now()}`, email: `${Date.now()}@test.com` }
	});
	const user = await res.json();

	await page.goto(`/users/${user.id}/settings`);
	await page.getByLabel('Email').fill('updated@example.com');
	await page.getByRole('button', { name: 'Save' }).click();
	await expect(page.getByLabel('Email')).toHaveValue('updated@example.com');

	await request.delete(`/api/users/${user.id}`);
});
```

**Using `testInfo` for unique identifiers:**

```ts
import { test, expect } from '@playwright/test';

test('submit order', async ({ page }, testInfo) => {
	const orderId = `order-${testInfo.workerIndex}-${Date.now()}`;
	await page.goto(`/orders/new?ref=${orderId}`);
	// ...
});
```

### Dynamic Shard Count

**Use when**: Automatically adjusting shards based on test count.

```yaml
# .github/workflows/playwright.yml
jobs:
    calculate-shards:
        runs-on: ubuntu-latest
        outputs:
            shard-count: ${{ steps.calc.outputs.count }}
            shard-matrix: ${{ steps.calc.outputs.matrix }}
        steps:
            - uses: actions/checkout@v4
            - run: npm ci
            - id: calc
              run: |
                  TEST_COUNT=$(npx playwright test --list --reporter=json 2>/dev/null | node -e "
                    const data = require('fs').readFileSync('/dev/stdin', 'utf8');
                    const parsed = JSON.parse(data);
                    console.log(parsed.suites?.reduce((acc, s) => acc + (s.specs?.length || 0), 0) || 0);
                  ")
                  # 1 shard per 20 tests, min 1, max 8
                  SHARDS=$(( (TEST_COUNT + 19) / 20 ))
                  SHARDS=$(( SHARDS > 8 ? 8 : SHARDS ))
                  SHARDS=$(( SHARDS < 1 ? 1 : SHARDS ))
                  MATRIX="["
                  for i in $(seq 1 $SHARDS); do
                    [ $i -gt 1 ] && MATRIX+=","
                    MATRIX+="\"$i/$SHARDS\""
                  done
                  MATRIX+="]"
                  echo "count=$SHARDS" >> $GITHUB_OUTPUT
                  echo "matrix=$MATRIX" >> $GITHUB_OUTPUT

    test:
        needs: calculate-shards
        runs-on: ubuntu-latest
        strategy:
            fail-fast: false
            matrix:
                shard: ${{ fromJson(needs.calculate-shards.outputs.shard-matrix) }}
        steps:
            - uses: actions/checkout@v4
            - run: npm ci
            - run: npx playwright install --with-deps
            - run: npx playwright test --shard=${{ matrix.shard }}
```

## Decision Guide

| Scenario                         | Workers        | Shards | Reason                                  |
| -------------------------------- | -------------- | ------ | --------------------------------------- |
| < 50 tests, < 5 min              | Auto (default) | None   | No optimization needed                  |
| 50-200 tests, 5-15 min           | `'50%'` in CI  | 2-4    | Balance speed and cost                  |
| 200+ tests, > 15 min             | `'50%'` in CI  | 4-8    | Keep feedback under 10 min              |
| Flaky due to resource contention | Reduce to 2    | Keep   | Less CPU/memory pressure                |
| Tests modify shared database     | 1 or isolate   | Useful | Sharding splits files; workers run them |
| CI has limited resources         | 1 or `'25%'`   | More   | Compensate with more machines           |

| Aspect         | Workers (in-process)      | Shards (across machines)   |
| -------------- | ------------------------- | -------------------------- |
| What it splits | Tests across CPU cores    | Test files across CI jobs  |
| Controlled by  | Config or `--workers` CLI | `--shard=X/Y` CLI flag     |
| Shares memory  | Yes                       | No                         |
| Report merging | Not needed                | Required (`merge-reports`) |
| Cost           | Free (same machine)       | More CI minutes            |

## Anti-Patterns

| Anti-Pattern                            | Problem                                  | Solution                                             |
| --------------------------------------- | ---------------------------------------- | ---------------------------------------------------- |
| `fullyParallel: false` without reason   | Tests in files run serially              | Set `fullyParallel: true` unless tests need serial   |
| `workers: 1` in CI "for safety"         | Negates parallelism                      | Fix isolation issues; use `workers: '50%'`           |
| Hardcoded shared user account           | Race conditions in parallel runs         | Each test creates unique data                        |
| Sharding without blob reporter          | Each shard produces separate HTML report | Configure `reporter: [['blob']]` for CI              |
| Sharding with 3 tests                   | Setup overhead exceeds time saved        | Only shard when suite > 5 minutes                    |
| `test.describe.serial()` everywhere     | Kills parallelism, creates dependencies  | Use only when tests genuinely need prior state       |
| Workers > CPU cores                     | Context switching overhead               | Use `'50%'` or auto-detect                           |
| Missing `fail-fast: false` in CI matrix | One shard failure cancels others         | Always set `fail-fast: false` for sharded strategies |

## Troubleshooting

### Tests pass solo but fail together

- **Shared state**. Make test data unique:
    ```ts
    test('create item', async ({ request }, ti) => {
    	await request.post('/api/items', {
    		data: { name: `Item-${ti.workerIndex}-${Date.now()}` }
    	});
    });
    ```

### "No tests found" in some shards

- **Too many shards**. Never exceed file count:
    ```bash
    npx playwright test --shard=1/10   # ok if 10 files
    npx playwright test --shard=1/20   # too many, some shards empty
    ```

### Merged report missing results

- **Blob reports collide**. Use unique names:
    ```yaml
    # Each shard
    - uses: actions/upload-artifact@v4
      with:
          name: blob-report-${{ strategy.job-index }}
          path: blob-report/
    # Merge step
    - uses: actions/download-artifact@v4
      with:
          pattern: blob-report-*
          merge-multiple: true
          path: all-blob-reports
    ```

### Worker-scoped fixture not working

- **Missing `{ scope: 'worker' }`**. Fix:
    ```ts
    export const test = base.extend({
    	resource: [
    		async ({}, use) => {
    			const r = await Resource.create();
    			await use(r);
    			await r.destroy();
    		},
    		{ scope: 'worker' }
    	]
    });
    ```

### More workers = Slower

- **Too many workers thrash**. Limit in CI:
    ```ts
    export default defineConfig({
    	workers: process.env.CI ? 2 : undefined
    });
    ```
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall