Skip to content

netresearch/typo3-testing-skill / skills/typo3-testing

typo3-testing

Flagged

Use when reproducing defects as failing tests, testing template or rendered output changes, confirming changes on a running TYPO3 instance, or setting up TYPO3 extension tests: unit/functional/E2E, PHPUnit 11-13, mutation testing, final-class mocks (v14), CI matrix across TYPO3 12/13/14.3 LTS, typo3-ci-workflows meta-package, CI failures. Also: transaction rollback, worker crashes, competing requests, testing-framework setup, test matrix, integration/e2e testing, coverage, test generation.

FileKindSize
SKILL.mdskill md6.8 KB
assets/.php-cs-fixer.dist.phpscript7.0 KB
assets/AGENTS.mdmarkdown5.0 KB
assets/Build/Scripts/runTests.shexecutablescript27.5 KB
assets/Build/playwright/.nvmrcother6 B
assets/Build/playwright/package.jsonmanifest535 B
assets/Build/playwright/playwright.config.tsscript1.3 KB
assets/Build/playwright/tests/playwright/accessibility/modules.spec.tsscript2.0 KB
assets/Build/playwright/tests/playwright/config.tsscript712 B
assets/Build/playwright/tests/playwright/e2e/backend-module.spec.tsscript1.9 KB
assets/Build/playwright/tests/playwright/fixtures/setup-fixtures.tsscript2.7 KB
assets/Build/playwright/tests/playwright/helper/login.setup.tsscript975 B
assets/FunctionalTests.xmlconfig1.4 KB
assets/FunctionalTestsBootstrap.phpscript1.6 KB
assets/Makefileother2.3 KB
assets/UnitTests.xmlconfig890 B
assets/UnitTestsBootstrap.phpscript2.1 KB
assets/bootstrap.phpscript1.4 KB
assets/codecov.ymlconfig1.6 KB
assets/docker/codeception.ymlconfig979 B
assets/docker/docker-compose.ymlconfig1.4 KB
assets/example-tests/ExampleAcceptanceCest.phpscript2.2 KB
assets/example-tests/ExampleFunctionalTest.phpscript2.4 KB
assets/example-tests/ExampleUnitTest.phpscript1.6 KB
assets/fixtures/README.mdmarkdown2.0 KB
assets/fixtures/be_users.csvother361 B
assets/fixtures/pages.csvother257 B
assets/fixtures/sys_category.csvother228 B
assets/fixtures/tt_content.csvother338 B
assets/github-actions-e2e.ymlconfig8.8 KB
assets/github-actions-tests.ymlconfig6.3 KB
assets/infection.json5other2.8 KB
assets/phpat.neonother330 B
assets/phpat.phpscript8.4 KB
assets/phpstan-baseline.neonother365 B
assets/phpstan.neonother1.4 KB
assets/rector.phpscript2.9 KB
checkpoints.yamlconfig61.7 KB
references/accessibility-testing.mdmarkdown7.6 KB
references/architecture-testing.mdmarkdown4.4 KB
references/asset-templates-guide.mdmarkdown3.4 KB
references/backend-module-render-verification.mdmarkdown9.7 KB
references/backend-user-access-testing.mdmarkdown9.5 KB
references/captainhook-setup.mdmarkdown2.4 KB
references/ci-cd.mdmarkdown39.5 KB
references/ci-debugging.mdmarkdown20.0 KB
references/ci-workflows-meta-package.mdmarkdown11.6 KB
references/crash-and-concurrency-testing.mdmarkdown4.8 KB
references/crypto-testing.mdmarkdown16.6 KB
references/datahandler-silent-rewrites.mdmarkdown7.2 KB
references/ddev-testing.mdmarkdown1.3 KB
references/e2e-testing.mdmarkdown53.0 KB
references/enforcement-rules.mdmarkdown5.7 KB
references/event-dispatch-testing.mdmarkdown2.8 KB
references/framework-compat-gate.mdmarkdown4.9 KB
references/functional-test-patterns.mdmarkdown11.0 KB
references/functional-testing.mdmarkdown46.3 KB
references/fuzz-testing.mdmarkdown8.6 KB
references/integration-testing.mdmarkdown14.1 KB
references/javascript-testing.mdmarkdown29.3 KB
references/live-instance-verification.mdmarkdown8.6 KB
references/mock-validity.mdmarkdown15.7 KB
references/mutation-testing.mdmarkdown36.0 KB
references/performance-testing.mdmarkdown11.2 KB
references/quality-tools.mdmarkdown25.5 KB
references/release-workflow-validation.mdmarkdown3.6 KB
references/sonarcloud.mdmarkdown18.8 KB
references/synthetic-secret-fixtures.mdmarkdown3.8 KB
references/tdd-discipline.mdmarkdown9.1 KB
references/test-environment-guards.mdmarkdown18.5 KB
references/test-runners.mdmarkdown24.6 KB
references/typo3-ci-config-patterns.mdmarkdown9.0 KB
references/typo3-v14-final-classes.mdmarkdown16.5 KB
references/unit-testing.mdmarkdown59.1 KB
scripts/generate-test.shexecutablescript6.0 KB
scripts/setup-testing.shexecutablescript6.3 KB
scripts/validate-setup.shexecutablescript4.0 KB

references/tdd-discipline.md

View on GitHub
<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
<!-- SPDX-FileCopyrightText: Netresearch DTT GmbH -->

# TDD Discipline

The strict loop used for bug fixes and new features. Prevents "tested and verified" claims that turn out to be wishful thinking.

## The Non-Negotiable Loop

For every bug fix, follow this sequence. Do not deviate. Do not ask the user for input mid-loop until the loop is green or until you have tried 5 distinct implementation approaches.

1. **Reproduce** — write a failing test that captures the bug. Commit it on the working branch under a conventional-commit message like `test: reproduce <bug>`.
2. **Confirm the test fails for the expected reason** — not a setup error, not a bootstrapping error, not a missing fixture. The failure message must describe the actual bug.
3. **Implement the minimal fix.** Scope it narrowly to the smallest diff that turns the failing test green.
4. **Run the specific test** via `Build/Scripts/runTests.sh -s unit -- --filter <TestName>` (or functional, as appropriate). Must pass.
   Check the test **count**, not the exit code: a `--filter` that matches nothing prints `No tests executed!` and exits 0, so a typo turns the red step green (see [test-runners.md](test-runners.md) § A `--filter` That Matches Nothing Exits 0).
5. **Run the full suite** (`-s unit`, `-s functional`) and linters (`-s phpstan`, `-s cgl`). Must all pass.
6. **If anything fails, iterate.** Do not report the fix as done. Try up to 5 distinct approaches before escalating.
7. **Only then open the PR.** The PR description must include the name of the reproduction test and the one-line verification command.

## What the Evidence Establishes

Passing tests are evidence for their exercised cases. If a proposed bug
reproduction passes immediately, check whether it reaches the reported failure;
the baseline can still describe already-correct behavior. For persistence and
concurrency claims, choose the real boundary using
[crash-and-concurrency-testing.md](crash-and-concurrency-testing.md). A passing
exception test does not establish behavior when the process is killed.

## Forbidden Phrases Without Evidence

These words are banned from assistant output unless the same turn contains the command output backing the claim:

- "tested"
- "verified"
- "confirmed working"
- "passes"
- "all green"

If you cannot run the verification (no DDEV available, no Docker, sandbox constraints), say so explicitly: "I implemented the fix but did not run the test suite because <reason>. The validating command is: `<command>`."

## Evidence Required

"Evidence" means one of:

- Command output pasted in the same turn (stdout or stderr)
- A CI run URL with visible status
- A gist or artifact link with visible content
- A GitHub Actions summary screenshot (for E2E)

A bare assertion from the assistant is not evidence.

## Playwright Hard Timeout

Playwright sessions have hung for 2+ hours when waiting for a selector that never appeared. Set a hard ceiling. The snippet below shows the timeout-relevant fields only — merge into the project's existing `playwright.config.ts` alongside its `projects`, `webServer`, `use`, and reporter settings:

```typescript
// playwright.config.ts — merge into existing defineConfig({...})
import { defineConfig } from '@playwright/test';

export default defineConfig({
  timeout: 120_000,            // 2 min per test — HARD CAP
  expect: { timeout: 15_000 }, // 15 s per expectation
  globalTimeout: 1_800_000,    // 30 min total across all tests
  workers: 4,
  forbidOnly: !!process.env.CI,
  // ... keep existing fields: projects, webServer, use, reporter, etc.
});
```

Individual tests that legitimately need more time must document why in a comment and pass `test.setTimeout(...)` explicitly — never raise the global default.

### If a Playwright run hangs

1. Kill it. Do not wait longer than the configured timeout.
2. Re-run with `DEBUG=pw:api` to see which action stalled.
3. Inspect the trace: `npx playwright show-trace test-results/*.zip`.
4. Add an explicit `waitFor` with a short timeout + meaningful error message to the stalling selector.

## Cross-Version Test Worktree Authority

When testing across TYPO3 v11/v12/v13/v14, use a separate worktree per version — never switch branches in place. The `.bare/` bare-clone layout this assumes (setup commands, absolute-path rules, cache safety) is documented in [`typo3-extension-upgrade-skill/references/multi-version-worktrees.md`](https://github.com/netresearch/typo3-extension-upgrade-skill/blob/main/skills/typo3-extension-upgrade/references/multi-version-worktrees.md) — set that up first, then:

```bash
git -C .bare worktree add ../TYPO3_11 TYPO3_11
git -C .bare worktree add ../TYPO3_12 TYPO3_12
git -C .bare worktree add ../main main

# Run tests in the specific worktree
cd ../TYPO3_11 && Build/Scripts/runTests.sh -s functional -p 8.1
cd ../main     && Build/Scripts/runTests.sh -s functional -p 8.4
```

### Before declaring "tested on v14"

Verify the command ran in the correct worktree:

```bash
pwd                                                      # must match the intended worktree

# composer.json: typo3/cms-core may be declared in require OR require-dev
# depending on whether the extension treats core as a runtime or a dev
# dependency. typo3/testing-framework is typically require-dev. Search the
# union so both forms work.
jq -r '(.require + (."require-dev" // {}))["typo3/cms-core"]'          composer.json
jq -r '(.require + (."require-dev" // {}))["typo3/testing-framework"]' composer.json

# composer.lock: same packages may resolve under .packages or .packages-dev
# per the same runtime/dev split.
jq -r '(.packages + (."packages-dev" // []))[] | select(.name=="typo3/cms-core") | .version'           composer.lock
jq -r '(.packages + (."packages-dev" // []))[] | select(.name=="typo3/testing-framework") | .version' composer.lock
```

All four values should agree on the target major. If you ran tests in the v13 worktree but claimed v14, the claim is false.

## Prove the Test Can Fail — Name the Production Change

Step 2 of the loop only works when the test is written first. When the code
already exists — a review finding, a fix you wrote before thinking about the
test, a test added to cover old behaviour — there is no observed red state, and
"the suite is green" then says nothing about whether the test guards anything.

Before calling such a test done, answer one question in writing:

> Which single production change would make this test fail?

Then make that change, run the test, and confirm it fails. Revert. This is a
one-minute mutation probe and it is the only evidence that the test is load-bearing:

```bash
# 1. revert the fix (or invert the condition) in the production file
# 2. run only the test that is supposed to guard it
Build/Scripts/runTests.sh -s unit -- --filter <TestName>   # MUST fail
# 3. restore the production file
# 4. run the exact same command again
Build/Scripts/runTests.sh -s unit -- --filter <TestName>   # MUST pass
```

If you cannot name a change that breaks it, the test is decorative — delete it
or move it to the level where the behaviour actually lives.

**The failure mode this catches.** A fix moved a value from the wrong source to
the right one in a service, and the accompanying test asserted that the
*downstream setter* stored what it was handed. Green, and worthless: reverting
the actual fix — the caller passing the wrong value again — left the test green,
because it never involved the caller. The whole bug was restorable with a
passing suite. A reviewer found it; the probe above would have found it in a
minute.

Watch for the shape: a test that exercises a collaborator one layer *below* the
line you changed. Assert on the seam you actually moved.

## Anti-Patterns

| Anti-pattern | What it looks like | Why it's wrong |
|--------------|--------------------|----------------|
| "Looks fine, should work" | No command run | Zero evidence |
| Test written after the fix, never seen red | Green suite, no observed failure | Proves nothing; run the mutation probe above |
| Asserting on a setter instead of the changed seam | `assertSame($x, $entity->getX())` for a fix in the *caller* | Reverting the fix leaves the test green |
| "Tests pass locally" | No output pasted | Unverifiable claim |
| Sharing a mock DB in a multi-test setup | One DB fixture across tests | Test pollution; flaky failures |
| Same service instance reused across tests | `private static Service $sharedService` at class scope | State bleed between tests |
| Running tests without `-p <php-version>` | `Build/Scripts/runTests.sh -s unit` on multi-PHP project | Silently uses host PHP, misses compat bugs |
| Declaring green on one TYPO3 version, shipping to all | Single `-s functional` run | Different LTSes break differently |

## Reporting a Fix

After the loop completes, report in this format:

```
Fix summary: <one sentence>

Reproduction test: Tests/Unit/<Name>Test.php::<testMethod>
Verification:
  $ Build/Scripts/runTests.sh -s unit -- --filter <Name>
  <pasted output, last 10 lines>

Full suite: green on <versions tested>
Not tested: <any versions or suites not exercised, with reason>
```

The "Not tested" line is mandatory — if every suite ran, write "none".
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall