Skip to content

ever-co/ever-works / .agents/skills/playwright-best-practices

playwright-best-practices

Verified

Use when writing Playwright tests, fixing flaky tests, debugging failures, implementing Page Object Model, configuring CI/CD, optimizing performance, mocking APIs, handling authentication or OAuth, testing accessibility (axe-core), file uploads/downloads, date/time mocking, WebSockets, geolocation, permissions, multi-tab/popup flows, mobile/responsive layouts, touch gestures, GraphQL, error handling, offline mode, multi-user collaboration, third-party services (payments, email verification), console error monitoring, global setup/teardown, test annotations (skip, fixme, slow), test tags (@smoke, @fast, @critical, filtering with --grep), project dependencies, security testing (XSS, CSRF, auth), performance budgets (Web Vitals, Lighthouse), iframes, component testing, canvas/WebGL, service workers/PWA, test coverage, i18n/localization, Electron apps, or browser extension testing. Covers E2E, component, API, visual, accessibility, security, Electron, and extension testing.

FileKindSize
LICENSE.mdmarkdown1.0 KB
README.mdmarkdown9.9 KB
SKILL.mdskill md25.7 KB
advanced/authentication-flows.mdmarkdown10.4 KB
advanced/authentication.mdmarkdown29.4 KB
advanced/clock-mocking.mdmarkdown9.4 KB
advanced/mobile-testing.mdmarkdown10.3 KB
advanced/multi-context.mdmarkdown8.2 KB
advanced/multi-user.mdmarkdown10.9 KB
advanced/network-advanced.mdmarkdown10.1 KB
advanced/third-party.mdmarkdown11.1 KB
architecture/pom-vs-fixtures.mdmarkdown11.4 KB
architecture/test-architecture.mdmarkdown14.3 KB
architecture/when-to-mock.mdmarkdown11.9 KB
browser-apis/browser-apis.mdmarkdown10.1 KB
browser-apis/iframes.mdmarkdown11.1 KB
browser-apis/service-workers.mdmarkdown13.1 KB
browser-apis/websockets.mdmarkdown9.9 KB
core/annotations.mdmarkdown10.4 KB
core/assertions-waiting.mdmarkdown9.1 KB
core/configuration.mdmarkdown13.7 KB
core/fixtures-hooks.mdmarkdown11.2 KB
core/global-setup.mdmarkdown12.4 KB
core/locators.mdmarkdown6.4 KB
core/page-object-model.mdmarkdown7.7 KB
core/projects-dependencies.mdmarkdown8.1 KB
core/test-data.mdmarkdown11.3 KB
core/test-suite-structure.mdmarkdown9.1 KB
core/test-tags.mdmarkdown6.7 KB
debugging/console-errors.mdmarkdown9.3 KB
debugging/debugging.mdmarkdown14.7 KB
debugging/error-testing.mdmarkdown9.2 KB
debugging/flaky-tests.mdmarkdown14.5 KB
frameworks/angular.mdmarkdown18.9 KB
frameworks/nextjs.mdmarkdown13.4 KB
frameworks/react.mdmarkdown17.5 KB
frameworks/vue.mdmarkdown17.3 KB
infrastructure-ci-cd/ci-cd.mdmarkdown10.4 KB
infrastructure-ci-cd/docker.mdmarkdown7.0 KB
infrastructure-ci-cd/github-actions.mdmarkdown15.1 KB
infrastructure-ci-cd/gitlab.mdmarkdown10.9 KB
infrastructure-ci-cd/other-providers.mdmarkdown14.2 KB
infrastructure-ci-cd/parallel-sharding.mdmarkdown11.6 KB
infrastructure-ci-cd/performance.mdmarkdown11.3 KB
infrastructure-ci-cd/reporting.mdmarkdown11.4 KB
infrastructure-ci-cd/test-coverage.mdmarkdown12.0 KB
testing-patterns/accessibility.mdmarkdown8.7 KB
testing-patterns/api-testing.mdmarkdown23.5 KB
testing-patterns/browser-extensions.mdmarkdown13.0 KB
testing-patterns/canvas-webgl.mdmarkdown12.3 KB
testing-patterns/component-testing.mdmarkdown11.0 KB
testing-patterns/drag-drop.mdmarkdown17.4 KB
testing-patterns/electron.mdmarkdown12.8 KB
testing-patterns/file-operations.mdmarkdown10.0 KB
testing-patterns/file-upload-download.mdmarkdown16.5 KB
testing-patterns/forms-validation.mdmarkdown21.3 KB
testing-patterns/graphql-testing.mdmarkdown8.6 KB
testing-patterns/i18n.mdmarkdown12.3 KB
testing-patterns/performance-testing.mdmarkdown11.5 KB
testing-patterns/security-testing.mdmarkdown11.5 KB
testing-patterns/visual-regression.mdmarkdown18.4 KB

infrastructure-ci-cd/other-providers.md

View on GitHub
# CI: CircleCI, Azure DevOps, and Jenkins

> **When to use**: Running Playwright tests in CI platforms other than GitHub Actions or GitLab.

## Table of Contents

1. [Common Commands](#common-commands)
2. [Jenkins](#jenkins)
3. [CircleCI](#circleci)
4. [Azure DevOps](#azure-devops)
5. [JUnit Reporter Config](#junit-reporter-config)
6. [Platform Comparison](#platform-comparison)
7. [Troubleshooting](#troubleshooting)
8. [Anti-Patterns](#anti-patterns)

---

## Common Commands

```bash
npx playwright install --with-deps    # browsers + OS dependencies
npx playwright test --shard=1/4       # parallel sharding
npx playwright merge-reports ./blob-report  # combine shard results
npx playwright test --reporter=dot,html     # multiple reporters
```

## Jenkins

### Declarative Pipeline

```groovy
// Jenkinsfile
pipeline {
    agent {
        docker {
            image 'mcr.microsoft.com/playwright:v1.48.0-noble'
            args '-u root'
        }
    }

    environment {
        CI = 'true'
        HOME = '/root'
        npm_config_cache = "${WORKSPACE}/.npm"
    }

    options {
        timeout(time: 30, unit: 'MINUTES')
        disableConcurrentBuilds()
    }

    stages {
        stage('Install') {
            steps {
                sh 'npm ci'
            }
        }

        stage('Test') {
            steps {
                sh 'npx playwright test'
            }
            post {
                always {
                    junit allowEmptyResults: true,
                         testResults: 'results/junit.xml'
                    archiveArtifacts artifacts: 'pw-report/**',
                                     allowEmptyArchive: true
                    archiveArtifacts artifacts: 'results/**',
                                     allowEmptyArchive: true
                }
            }
        }
    }

    post {
        failure {
            echo 'Tests failed!'
        }
        cleanup {
            cleanWs()
        }
    }
}
```

### Parallel Shards

```groovy
// Jenkinsfile (sharded)
pipeline {
    agent none

    environment {
        CI = 'true'
        HOME = '/root'
    }

    options {
        timeout(time: 30, unit: 'MINUTES')
    }

    stages {
        stage('Test') {
            parallel {
                stage('Shard 1') {
                    agent {
                        docker {
                            image 'mcr.microsoft.com/playwright:v1.48.0-noble'
                            args '-u root'
                        }
                    }
                    steps {
                        sh 'npm ci'
                        sh 'npx playwright test --shard=1/4'
                    }
                    post {
                        always {
                            archiveArtifacts artifacts: 'blob-report/**',
                                             allowEmptyArchive: true
                        }
                    }
                }
                stage('Shard 2') {
                    agent {
                        docker {
                            image 'mcr.microsoft.com/playwright:v1.48.0-noble'
                            args '-u root'
                        }
                    }
                    steps {
                        sh 'npm ci'
                        sh 'npx playwright test --shard=2/4'
                    }
                    post {
                        always {
                            archiveArtifacts artifacts: 'blob-report/**',
                                             allowEmptyArchive: true
                        }
                    }
                }
                stage('Shard 3') {
                    agent {
                        docker {
                            image 'mcr.microsoft.com/playwright:v1.48.0-noble'
                            args '-u root'
                        }
                    }
                    steps {
                        sh 'npm ci'
                        sh 'npx playwright test --shard=3/4'
                    }
                    post {
                        always {
                            archiveArtifacts artifacts: 'blob-report/**',
                                             allowEmptyArchive: true
                        }
                    }
                }
                stage('Shard 4') {
                    agent {
                        docker {
                            image 'mcr.microsoft.com/playwright:v1.48.0-noble'
                            args '-u root'
                        }
                    }
                    steps {
                        sh 'npm ci'
                        sh 'npx playwright test --shard=4/4'
                    }
                    post {
                        always {
                            archiveArtifacts artifacts: 'blob-report/**',
                                             allowEmptyArchive: true
                        }
                    }
                }
            }
        }
    }
}
```

## CircleCI

### Basic Pipeline

```yaml
# .circleci/config.yml
version: 2.1

executors:
    pw:
        docker:
            - image: mcr.microsoft.com/playwright:v1.48.0-noble
        working_directory: ~/app

jobs:
    install:
        executor: pw
        steps:
            - checkout
            - restore_cache:
                  keys:
                      - deps-{{ checksum "package-lock.json" }}
            - run: npm ci
            - save_cache:
                  key: deps-{{ checksum "package-lock.json" }}
                  paths:
                      - node_modules
            - persist_to_workspace:
                  root: .
                  paths:
                      - node_modules

    test:
        executor: pw
        parallelism: 4
        steps:
            - checkout
            - attach_workspace:
                  at: .
            - run:
                  name: Run tests
                  command: |
                      npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
            - store_artifacts:
                  path: pw-report
                  destination: pw-report
            - store_artifacts:
                  path: results
                  destination: results
            - store_test_results:
                  path: results/junit.xml

workflows:
    test:
        jobs:
            - install
            - test:
                  requires:
                      - install
```

### Using Orbs

```yaml
# .circleci/config.yml
version: 2.1

orbs:
    node: circleci/node@latest

executors:
    pw:
        docker:
            - image: mcr.microsoft.com/playwright:v1.48.0-noble

jobs:
    e2e:
        executor: pw
        parallelism: 4
        steps:
            - checkout
            - node/install-packages
            - run:
                  name: Run tests
                  command: npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
            - store_artifacts:
                  path: pw-report
            - store_test_results:
                  path: results/junit.xml

workflows:
    main:
        jobs:
            - e2e
```

## Azure DevOps

### Basic Pipeline

```yaml
# azure-pipelines.yml
trigger:
    branches:
        include:
            - main

pr:
    branches:
        include:
            - main

pool:
    vmImage: 'ubuntu-latest'

variables:
    CI: 'true'
    npm_config_cache: $(Pipeline.Workspace)/.npm

steps:
    - task: NodeTool@0
      inputs:
          versionSpec: '20.x'
      displayName: 'Install Node.js'

    - task: Cache@2
      inputs:
          key: 'npm | "$(Agent.OS)" | package-lock.json'
          restoreKeys: |
              npm | "$(Agent.OS)"
          path: $(npm_config_cache)
      displayName: 'Cache npm'

    - script: npm ci
      displayName: 'Install dependencies'

    - script: npx playwright install --with-deps
      displayName: 'Install browsers'

    - script: npx playwright test
      displayName: 'Run tests'

    - task: PublishTestResults@2
      condition: always()
      inputs:
          testResultsFormat: 'JUnit'
          testResultsFiles: 'results/junit.xml'
          mergeTestResults: true
          testRunTitle: 'E2E Tests'
      displayName: 'Publish results'

    - task: PublishPipelineArtifact@1
      condition: always()
      inputs:
          targetPath: pw-report
          artifact: pw-report
          publishLocation: 'pipeline'
      displayName: 'Upload report'
```

### With Sharding

```yaml
# azure-pipelines.yml
trigger:
    branches:
        include:
            - main

pr:
    branches:
        include:
            - main

variables:
    CI: 'true'

stages:
    - stage: Test
      jobs:
          - job: E2E
            pool:
                vmImage: 'ubuntu-latest'
            strategy:
                matrix:
                    shard1:
                        SHARD: '1/4'
                    shard2:
                        SHARD: '2/4'
                    shard3:
                        SHARD: '3/4'
                    shard4:
                        SHARD: '4/4'
            steps:
                - task: NodeTool@0
                  inputs:
                      versionSpec: '20.x'

                - script: npm ci
                  displayName: 'Install dependencies'

                - script: npx playwright install --with-deps
                  displayName: 'Install browsers'

                - script: npx playwright test --shard=$(SHARD)
                  displayName: 'Run tests (shard $(SHARD))'

                - task: PublishPipelineArtifact@1
                  condition: always()
                  inputs:
                      targetPath: blob-report
                      artifact: blob-report-$(System.JobPositionInPhase)
                  displayName: 'Upload blob report'

    - stage: Report
      dependsOn: Test
      condition: always()
      jobs:
          - job: MergeReports
            pool:
                vmImage: 'ubuntu-latest'
            steps:
                - task: NodeTool@0
                  inputs:
                      versionSpec: '20.x'

                - script: npm ci
                  displayName: 'Install dependencies'

                - task: DownloadPipelineArtifact@2
                  inputs:
                      patterns: 'blob-report-*/**'
                      path: all-blob-reports
                  displayName: 'Download blob reports'

                - script: npx playwright merge-reports --reporter=html ./all-blob-reports
                  displayName: 'Merge reports'

                - task: PublishPipelineArtifact@1
                  inputs:
                      targetPath: pw-report
                      artifact: pw-report
                  displayName: 'Upload merged report'
```

## JUnit Reporter Config

All platforms benefit from JUnit output for native test result display:

```typescript
// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
	reporter: process.env.CI
		? [['dot'], ['html', { open: 'never' }], ['junit', { outputFile: 'results/junit.xml' }]]
		: [['html', { open: 'on-failure' }]]
});
```

## Platform Comparison

| Feature           | CircleCI                                        | Azure DevOps                     | Jenkins                |
| ----------------- | ----------------------------------------------- | -------------------------------- | ---------------------- |
| Docker support    | `docker:` executor                              | `vmImage` or container jobs      | Docker Pipeline plugin |
| Parallelism       | `parallelism: N` + `CIRCLE_NODE_INDEX`          | `strategy.matrix`                | `parallel` stages      |
| Artifact upload   | `store_artifacts`                               | `PublishPipelineArtifact@1`      | `archiveArtifacts`     |
| JUnit integration | `store_test_results`                            | `PublishTestResults@2`           | `junit` step           |
| Shard variable    | `$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL` | Define in matrix: `SHARD: '1/4'` | Hardcode per stage     |
| Cache key         | `checksum "package-lock.json"`                  | `Cache@2` with key template      | `stash`/`unstash`      |
| Secrets           | Context + env variables                         | Variable groups                  | Credentials plugin     |

## Troubleshooting

### Jenkins: "Browser closed unexpectedly"

Running as non-root in container causes sandbox issues.

```groovy
agent {
    docker {
        image 'mcr.microsoft.com/playwright:v1.48.0-noble'
        args '-u root'
    }
}
environment {
    HOME = '/root'
}
```

### CircleCI: "Executable doesn't exist"

Image version mismatch with `@playwright/test` version. Use `latest` tag or match versions:

```yaml
docker:
    - image: mcr.microsoft.com/playwright:v1.48.0-noble
```

### Azure DevOps: Test results not showing

Missing JUnit reporter or `PublishTestResults@2` task:

```typescript
reporter: [['junit', { outputFile: 'results/junit.xml' }]],
```

```yaml
- task: PublishTestResults@2
  condition: always()
  inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: 'results/junit.xml'
```

### Shard index off by one

CircleCI's `CIRCLE_NODE_INDEX` is 0-based, Playwright's `--shard` is 1-based:

```yaml
# CircleCI - add 1
command: npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
```

## Anti-Patterns

| Anti-Pattern                        | Problem                                   | Solution                                             |
| ----------------------------------- | ----------------------------------------- | ---------------------------------------------------- |
| Missing `--with-deps` on bare metal | OS libs missing, browser launch fails     | Use Playwright Docker image or `--with-deps`         |
| No JUnit reporter                   | CI can't display test results             | Add `['junit', { outputFile: 'results/junit.xml' }]` |
| No job timeout                      | Hung tests consume resources indefinitely | Set explicit timeout (20-30 min)                     |
| No artifact upload on success       | Can't verify passing results              | Always upload reports (`condition: always()`)        |
| Non-root in container without setup | Permission errors on browser binaries     | Run as root or configure permissions                 |
| Hardcoded shard count               | Must update multiple places               | Use CI-native variables                              |
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall