Skip to content
skillxone

CherryHQ/cherry-studio / .agents/skills/cherry-regression-test

cherry-regression-test

Verified

Run Cherry Studio critical-path system regression tasks through the repository-owned Playwright E2E workflow. Use for full regression, release acceptance, development-branch system validation, or a named cherry-regression-test task on GitHub-hosted macOS and Windows runners.

Cherry Regression Test

Run deterministic Playwright E2E tests against one driver-owned Cherry Studio process per platform. The tested product includes Chat, Agents, MCP, Skills, knowledge bases, translation, image generation, and code tools; an LLM test agent does not control the test run.

CI contract

Use .github/workflows/e2e-regression-test.yml as the entry point. It:

  1. Resolves a trusted branch or release tag.
  2. Initializes an isolated directory under the GitHub runner temporary folder.
  3. Installs the application and the code tools under test.
  4. Launches one owned Electron process with CDP enabled.
  5. Runs tests/e2e/regression/phases/*.test.ts in the manifest's phase order.
  6. Continues after a failed phase so later results are still collected.
  7. Produces English platform and aggregate reports, then enforces the verdict.
  8. Stops only the Electron process recorded in the isolated run directory.

Do not add an LLM tool loop, MCP control server, turn limit, or a second Electron launch for each test. A restart is allowed only where the case contract explicitly verifies persistence or switches from the clean startup profile to the authenticated shared profile.

Test organization

Read scenario organization and the controller contract before making changes.

Register each case from the manifest:

test(...caseDefinition('S-01'), async ({ mainWindow }) => {
  // Assert the user-visible outcome.
})

cases.ts owns case IDs, titles, task tags, phases, and capability requirements. The workflow accepts a task ID and delegates selection to the controller. Prefer accessible roles, labels, placeholders, test IDs, and visible text. Native dialogs and cross-application interactions must use the repository-owned helpers in systemAutomation.ts.

Record assertions in Playwright, not prose. The custom reporter writes case and phase status into run.json and feeds the English Markdown/JUnit reports. The fixture saves failure screenshots. Executor errors and interrupted phases block a passing verdict. Do not enable Playwright Trace for credential-bearing tests because action parameters can expose secrets. A passing result does not depend on a model's judgment.

Focused execution

With an initialized run directory and its owned Electron process running:

pnpm exec tsx scripts/e2e/regression/cli.ts run-phase \
  --run-dir /absolute/run-directory --phase basicFeatures

The task selected when initializing the run determines which cases execute. For a Notes-only run, initialize with --task notes. Enumeration is read-only: set CHERRY_TEST_RUN_DIR to an absolute path, but no initialized run directory or running Electron process is required because --list does not execute fixtures.

CHERRY_TEST_RUN_DIR=/tmp/cherry-regression-list pnpm test:e2e:regression --list

Do not call the regression cleanup command for an Electron instance owned by cherry-electron-dev; cleanup is only for an app record created by this driver.

Verification when changing the framework

Run the focused script suite and enumerate Playwright cases:

pnpm exec vitest run --project scripts scripts/e2e/regression
CHERRY_TEST_RUN_DIR=/tmp/cherry-regression-list \
  pnpm test:e2e:regression --list
pnpm typecheck:e2e
pnpm test:lint

Do not use pnpm test or pnpm build:check for this focused workflow change.

skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 195kUpdated Claude CodeCodexVerifiedInstall
webapp-testinganthropics/skillsToolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.Stars: 195kUpdated Claude CodeCodexVerifiedInstall
browser-testing-with-devtoolsaddyosmani/agent-skillsTests in real browsers via Chrome DevTools MCP. Use when building or debugging anything that runs in a browser. Use when you need to inspect the DOM, capture console errors, analyze network requests, profile performance, or verify visual output with real runtime data. Requires the chrome-devtools MCP server to be configured.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
ci-cd-and-automationaddyosmani/agent-skillsAutomates CI/CD pipeline setup. Use when setting up or modifying build and deployment pipelines. Use when you need to automate quality gates, configure test runners in CI, or establish deployment strategies.Stars: 118kUpdated Claude CodeCodexVerifiedInstall