Skip to content

netresearch/typo3-testing-skill / skills/typo3-testing

typo3-testing

Flagged

Use when reproducing defects as failing tests, testing template or rendered output changes, confirming changes on a running TYPO3 instance, or setting up TYPO3 extension tests: unit/functional/E2E, PHPUnit 11-13, mutation testing, final-class mocks (v14), CI matrix across TYPO3 12/13/14.3 LTS, typo3-ci-workflows meta-package, CI failures. Also: transaction rollback, worker crashes, competing requests, testing-framework setup, test matrix, integration/e2e testing, coverage, test generation.

FileKindSize
SKILL.mdskill md6.8 KB
assets/.php-cs-fixer.dist.phpscript7.0 KB
assets/AGENTS.mdmarkdown5.0 KB
assets/Build/Scripts/runTests.shexecutablescript27.5 KB
assets/Build/playwright/.nvmrcother6 B
assets/Build/playwright/package.jsonmanifest535 B
assets/Build/playwright/playwright.config.tsscript1.3 KB
assets/Build/playwright/tests/playwright/accessibility/modules.spec.tsscript2.0 KB
assets/Build/playwright/tests/playwright/config.tsscript712 B
assets/Build/playwright/tests/playwright/e2e/backend-module.spec.tsscript1.9 KB
assets/Build/playwright/tests/playwright/fixtures/setup-fixtures.tsscript2.7 KB
assets/Build/playwright/tests/playwright/helper/login.setup.tsscript975 B
assets/FunctionalTests.xmlconfig1.4 KB
assets/FunctionalTestsBootstrap.phpscript1.6 KB
assets/Makefileother2.3 KB
assets/UnitTests.xmlconfig890 B
assets/UnitTestsBootstrap.phpscript2.1 KB
assets/bootstrap.phpscript1.4 KB
assets/codecov.ymlconfig1.6 KB
assets/docker/codeception.ymlconfig979 B
assets/docker/docker-compose.ymlconfig1.4 KB
assets/example-tests/ExampleAcceptanceCest.phpscript2.2 KB
assets/example-tests/ExampleFunctionalTest.phpscript2.4 KB
assets/example-tests/ExampleUnitTest.phpscript1.6 KB
assets/fixtures/README.mdmarkdown2.0 KB
assets/fixtures/be_users.csvother361 B
assets/fixtures/pages.csvother257 B
assets/fixtures/sys_category.csvother228 B
assets/fixtures/tt_content.csvother338 B
assets/github-actions-e2e.ymlconfig8.8 KB
assets/github-actions-tests.ymlconfig6.3 KB
assets/infection.json5other2.8 KB
assets/phpat.neonother330 B
assets/phpat.phpscript8.4 KB
assets/phpstan-baseline.neonother365 B
assets/phpstan.neonother1.4 KB
assets/rector.phpscript2.9 KB
checkpoints.yamlconfig61.7 KB
references/accessibility-testing.mdmarkdown7.6 KB
references/architecture-testing.mdmarkdown4.4 KB
references/asset-templates-guide.mdmarkdown3.4 KB
references/backend-module-render-verification.mdmarkdown9.7 KB
references/backend-user-access-testing.mdmarkdown9.5 KB
references/captainhook-setup.mdmarkdown2.4 KB
references/ci-cd.mdmarkdown39.5 KB
references/ci-debugging.mdmarkdown20.0 KB
references/ci-workflows-meta-package.mdmarkdown11.6 KB
references/crash-and-concurrency-testing.mdmarkdown4.8 KB
references/crypto-testing.mdmarkdown16.6 KB
references/datahandler-silent-rewrites.mdmarkdown7.2 KB
references/ddev-testing.mdmarkdown1.3 KB
references/e2e-testing.mdmarkdown53.0 KB
references/enforcement-rules.mdmarkdown5.7 KB
references/event-dispatch-testing.mdmarkdown2.8 KB
references/framework-compat-gate.mdmarkdown4.9 KB
references/functional-test-patterns.mdmarkdown11.0 KB
references/functional-testing.mdmarkdown46.3 KB
references/fuzz-testing.mdmarkdown8.6 KB
references/integration-testing.mdmarkdown14.1 KB
references/javascript-testing.mdmarkdown29.3 KB
references/live-instance-verification.mdmarkdown8.6 KB
references/mock-validity.mdmarkdown15.7 KB
references/mutation-testing.mdmarkdown36.0 KB
references/performance-testing.mdmarkdown11.2 KB
references/quality-tools.mdmarkdown25.5 KB
references/release-workflow-validation.mdmarkdown3.6 KB
references/sonarcloud.mdmarkdown18.8 KB
references/synthetic-secret-fixtures.mdmarkdown3.8 KB
references/tdd-discipline.mdmarkdown9.1 KB
references/test-environment-guards.mdmarkdown18.5 KB
references/test-runners.mdmarkdown24.6 KB
references/typo3-ci-config-patterns.mdmarkdown9.0 KB
references/typo3-v14-final-classes.mdmarkdown16.5 KB
references/unit-testing.mdmarkdown59.1 KB
scripts/generate-test.shexecutablescript6.0 KB
scripts/setup-testing.shexecutablescript6.3 KB
scripts/validate-setup.shexecutablescript4.0 KB

checkpoints.yaml

View on GitHub
# SPDX-License-Identifier: MIT
# SPDX-FileCopyrightText: Netresearch DTT GmbH

# Checkpoints for typo3-testing skill
# Focuses on PHPUnit, test structure, CI, PHPStan, and code coverage
#
# Supports two CI patterns:
#   1. Traditional: local Build/UnitTests.xml + runTests.sh + direct action usage
#   2. Shared workflow: netresearch/typo3-ci-workflows (reusable CI, handles
#      PHPUnit config naming, SHA pinning, coverage, and PHPStan internally)

version: 1
skill_id: typo3-testing

# Only run for TYPO3 extensions: the checks assume extension test infrastructure
# (runTests.sh, Build/ PHPUnit configs, typo3-ci-workflows). Without this gate
# /assess ran them against a Python repository and reported 45 failures.
preconditions:
  - type: json_path
    target: composer.json
    pattern: '.type == "typo3-cms-extension"'

mechanical:
  # === SHARED CI WORKFLOW DETECTION ===
  # Check first — many later checks can be relaxed when a shared CI workflow
  # handles PHPUnit config, SHA pinning, coverage, and PHPStan.

  - id: TT-00
    type: regex
    target: .github/workflows/*.yml
    pattern: 'uses:\s*netresearch/typo3-ci-workflows'
    severity: info
    desc: "Detects netresearch/typo3-ci-workflows reusable workflow (relaxes TT-01..05, TT-21, TT-24)"

  # === PHPUNIT CONFIGURATION ===
  # TYPO3 testing-framework convention: Build/UnitTests.xml + Build/FunctionalTests.xml
  # netresearch/typo3-ci-workflows also accepts: Build/phpunit.xml + Build/phpunit.functional.xml
  # Extensions that keep every PHPUnit config in one subdirectory use
  # Build/phpunit/UnitTests.xml + Build/phpunit/FunctionalTests.xml.
  # All three layouts are valid; TT-01/02/04/05/06 accept each of them.
  # When TT-00 passes, either naming convention is valid.

  - id: TT-01
    type: file_exists
    target: "{Build/UnitTests.xml,Build/phpunit.xml,Build/phpunit/UnitTests.xml}"
    severity: error
    desc: "PHPUnit unit test config must exist (Build/UnitTests.xml, Build/phpunit.xml or Build/phpunit/UnitTests.xml)"

  - id: TT-02
    type: file_exists
    target: "{Build/FunctionalTests.xml,Build/phpunit.functional.xml,Build/phpunit/FunctionalTests.xml}"
    severity: error
    desc: "PHPUnit functional test config must exist (Build/FunctionalTests.xml, Build/phpunit.functional.xml or Build/phpunit/FunctionalTests.xml)"

  - id: TT-03
    type: file_not_exists
    target: phpunit.xml
    severity: warning
    desc: "phpunit.xml in project root is discouraged; use Build/ directory instead"

  - id: TT-04
    type: regex
    target: "{Build/UnitTests.xml,Build/phpunit.xml,Build/phpunit/UnitTests.xml}"
    pattern: '<testsuite.*name="unit"'
    severity: error
    desc: "PHPUnit config must define unit test suite"

  - id: TT-05
    type: regex
    target: "{Build/FunctionalTests.xml,Build/phpunit.functional.xml,Build/phpunit/FunctionalTests.xml}"
    pattern: '<testsuite.*name="functional"'
    severity: error
    desc: "PHPUnit config must define functional test suite"

  # === PHPUNIT DEPRECATED ATTRIBUTES ===
  - id: TT-06
    type: regex_not
    target: "{Build/FunctionalTests.xml,Build/phpunit.xml,Build/UnitTests.xml,Build/phpunit.functional.xml,Build/phpunit/UnitTests.xml,Build/phpunit/FunctionalTests.xml}"
    pattern: 'displayDetailsOnIncompleteTests|displayDetailsOnSkippedTests'
    severity: warning
    desc: "PHPUnit configs must not use displayDetailsOnIncompleteTests or displayDetailsOnSkippedTests (removed in PHPUnit 12)"

  # === TEST DIRECTORY STRUCTURE ===
  - id: TT-10
    type: file_exists
    target: Tests/Unit/
    severity: error
    desc: "Tests/Unit/ directory must exist"

  - id: TT-11
    type: file_exists
    target: Tests/Functional/
    severity: warning
    desc: "Tests/Functional/ directory should exist"

  - id: TT-12
    type: regex
    target: Tests/Unit/**/*Test.php
    pattern: 'class.*Test.*extends.*TestCase'
    severity: error
    desc: "Unit tests must extend TestCase"

  - id: TT-13
    type: regex
    target: Tests/Functional/**/*Test.php
    pattern: 'class.*Test.*extends.*FunctionalTestCase'
    severity: warning
    desc: "Functional tests should extend FunctionalTestCase"

  # === CI WORKFLOW WITH TEST EXECUTION ===
  - id: TT-20
    type: file_exists
    target: .github/workflows/ci.yml
    severity: warning
    desc: "CI workflow should exist (ci.yml, ci.yaml, or tests.yml)"

  - id: TT-21
    type: regex
    target: .github/workflows/*.yml
    pattern: 'phpunit|composer.*test|runTests|typo3-ci-workflows'
    severity: error
    desc: "CI workflow must execute tests (directly or via reusable workflow)"

  - id: TT-22
    type: regex
    target: .github/workflows/*.yml
    pattern: 'php-version.*\[.*8\.[2-9]|php-versions'
    severity: warning
    desc: "CI should test multiple PHP versions (directly or via php-versions input)"

  - id: TT-23
    type: regex
    target: .github/workflows/*.yml
    pattern: 'typo3.*\[.*1[3-4]|typo3-versions'
    severity: warning
    desc: "CI should test multiple TYPO3 versions"

  - id: TT-24
    type: regex
    target: .github/workflows/*.yml
    pattern: 'uses:.*@[a-f0-9]{40}|uses:\s*netresearch/typo3-ci-workflows'
    severity: error
    desc: "GitHub Actions must be pinned to SHA (or use trusted reusable workflow)"

  # === PHPSTAN CONFIGURATION ===
  - id: TT-30
    type: file_exists
    target: "{phpstan.neon,Build/phpstan.neon,Build/phpstan/phpstan.neon}"
    severity: warning
    desc: "phpstan.neon configuration should exist"

  - id: TT-31
    type: file_exists
    target: "{phpstan-baseline.neon,Build/phpstan-baseline.neon,Build/phpstan/phpstan-baseline.neon}"
    severity: info
    desc: "phpstan-baseline.neon for gradual adoption (root or Build/ subdir)"

  - id: TT-32
    type: regex
    target: "{phpstan.neon,Build/phpstan.neon,Build/phpstan/phpstan.neon}"
    pattern: 'level:\s*(10|max)'
    severity: warning
    desc: "PHPStan must use level 10 (max) — this is the strict check for projects using typo3-ci-workflows"

  - id: TT-33
    type: regex
    target: "{phpstan.neon,Build/phpstan.neon,Build/phpstan/phpstan.neon}"
    pattern: "typo3|TYPO3"
    severity: info
    desc: "PHPStan should include TYPO3-specific configuration"

  - id: TT-34
    type: regex
    target: .github/workflows/*.yml
    pattern: 'phpstan|composer.*stan|composer.*analyze|typo3-ci-workflows'
    severity: warning
    desc: "CI should run PHPStan analysis (directly or via reusable workflow)"

  # Was a single-line `command:` chaining four alternatives with `||`, which the
  # runner rejects outright (command-chaining metacharacter), so it never ran and
  # reported a failure on every project. It also relied on an `expected:` field the
  # runner does not parse — the verdict has to be the exit status. Both are fixed
  # here by moving the logic into a screened script body.
  #
  # The level is read with `grep -oE` plus a bash suffix strip, not `grep -oP
  # …\K`: where PCRE is missing (BSD grep), the PCRE spelling captured nothing
  # from every candidate file and the loop then reported "no PHPStan
  # configuration declaring a level" — a false failure against a conformant
  # project on every such machine.
  #
  # Every candidate configuration is inspected before the verdict. A project
  # with Build/phpstan.neon at level 9 and phpstan.neon at level 10 satisfies
  # the rule; exiting at the first file below 10 reported it as non-conformant.
  - id: TT-35
    type: script
    command: |
      bad=""
      for f in Build/phpstan.neon Build/phpstan/phpstan.neon phpstan.neon phpstan.neon.dist; do
        [ -f "$f" ] || continue
        lvl=$(grep -m1 -oE 'level:[[:space:]]*([0-9]+|max)' "$f" 2>/dev/null)
        lvl="${lvl##*[[:space:]:]}"
        [ -z "$lvl" ] && continue
        if [ "$lvl" = "max" ] || [ "$lvl" = "10" ]; then
          exit 0
        fi
        bad="$bad $f=$lvl"
      done
      if [ -n "$bad" ]; then
        echo "FAIL: no PHPStan configuration declares level 10 or max; found:$bad"
        exit 1
      fi
      echo "FAIL: no PHPStan configuration declaring a level was found"
      exit 1
    severity: warning
    desc: "PHPStan must use level 10 (max) — this is the strict check for projects using typo3-ci-workflows"

  # === CODE COVERAGE ===
  - id: TT-40
    type: file_exists
    target: codecov.yml
    severity: info
    desc: "codecov.yml configuration for coverage settings"

  - id: TT-41
    type: regex
    target: .github/workflows/*.yml
    pattern: 'codecov|upload-coverage|typo3-ci-workflows'
    severity: warning
    desc: "CI should upload coverage (directly or via reusable workflow with upload-coverage: true)"

  - id: TT-42
    type: regex
    target: .github/workflows/{ci,tests}.{yml,yaml}
    pattern: 'coverage.*clover|XDEBUG_MODE.*coverage|--coverage|upload-coverage:\s*true'
    severity: warning
    desc: "CI should generate code coverage"

  - id: TT-43
    type: contains
    target: README.md
    pattern: "codecov.io"
    severity: info
    desc: "README should display Codecov badge"

  # === RUNTESTS.SH SCRIPT (mandatory for all Netresearch TYPO3 extensions) ===
  # TT-51/52/53/57/59 read the shared runner when the extension only carries
  # the stub. The Netresearch standard Build/Scripts/runTests.sh is a stub
  # (typo3-ci-workflows assets/Build/Scripts/runTests.stub.sh) that only execs
  # .Build/bin/runTests.sh, so grepping the stub measured nothing. That bin
  # file is not the runner either: it is Composer's ~1 KB bin proxy, which
  # execs the package file. When the stub is detected, the checks read the
  # package file directly, at <vendor-dir from composer.json, default
  # "vendor">/netresearch/typo3-ci-workflows/assets/Build/Scripts/runTests.sh.
  # When that file is absent (a clean checkout), the check passes by
  # delegation as long as composer.json requires
  # netresearch/typo3-ci-workflows, and fails when the stub has nothing to
  # hand over to.
  - id: TT-50
    type: file_exists
    target: Build/Scripts/runTests.sh
    severity: error
    desc: "Build/Scripts/runTests.sh must exist as the unified test runner for all TYPO3 extensions"

  - id: TT-51
    type: script
    command: |
      f=Build/Scripts/runTests.sh
      [ -f "$f" ] || exit 1
      if grep -q 'typo3-ci-workflows' "$f" && grep -qE '^exec[[:blank:]]+"[$][{]?RUNNER[}]?"' "$f"; then
        vd=$(jq -r '.config["vendor-dir"] // "vendor"' composer.json 2>/dev/null) || vd=vendor
        f="${vd:-vendor}/netresearch/typo3-ci-workflows/assets/Build/Scripts/runTests.sh"
        if [ ! -f "$f" ]; then
          jq -e '((.require // {}) + (."require-dev" // {})) | has("netresearch/typo3-ci-workflows")' composer.json >/dev/null 2>&1
          exit $?
        fi
      fi
      grep -qE 'TYPO3_VERSION|typo3/core' "$f"
    severity: warning
    desc: "runTests.sh should support TYPO3 version selection"

  - id: TT-52
    type: script
    command: |
      f=Build/Scripts/runTests.sh
      [ -f "$f" ] || exit 1
      if grep -q 'typo3-ci-workflows' "$f" && grep -qE '^exec[[:blank:]]+"[$][{]?RUNNER[}]?"' "$f"; then
        vd=$(jq -r '.config["vendor-dir"] // "vendor"' composer.json 2>/dev/null) || vd=vendor
        f="${vd:-vendor}/netresearch/typo3-ci-workflows/assets/Build/Scripts/runTests.sh"
        if [ ! -f "$f" ]; then
          jq -e '((.require // {}) + (."require-dev" // {})) | has("netresearch/typo3-ci-workflows")' composer.json >/dev/null 2>&1
          exit $?
        fi
      fi
      grep -qE 'PHP_VERSION|php.*version' "$f"
    severity: warning
    desc: "runTests.sh should support PHP version selection (-p flag)"

  - id: TT-53
    type: script
    command: |
      f=Build/Scripts/runTests.sh
      [ -f "$f" ] || exit 1
      if grep -q 'typo3-ci-workflows' "$f" && grep -qE '^exec[[:blank:]]+"[$][{]?RUNNER[}]?"' "$f"; then
        vd=$(jq -r '.config["vendor-dir"] // "vendor"' composer.json 2>/dev/null) || vd=vendor
        f="${vd:-vendor}/netresearch/typo3-ci-workflows/assets/Build/Scripts/runTests.sh"
        if [ ! -f "$f" ]; then
          jq -e '((.require // {}) + (."require-dev" // {})) | has("netresearch/typo3-ci-workflows")' composer.json >/dev/null 2>&1
          exit $?
        fi
      fi
      grep -qE 'suite|SUITE' "$f"
    severity: warning
    desc: "runTests.sh should support test suite selection (-s flag)"

  - id: TT-54
    type: command
    command: 'test -x Build/Scripts/runTests.sh'
    severity: error
    desc: "Build/Scripts/runTests.sh must be executable"

  # Two valid entry-point layouts, and this check accepts both:
  #   1. composer scripts call Build/Scripts/runTests.sh directly;
  #   2. composer scripts (ci:test:php:unit, ci:test:php:functional, ...) call
  #      phpunit directly, and runTests.sh runs the suites in TYPO3
  #      core-testing containers for multi-version runs. The TYPO3 reference
  #      extension TYPO3BestPractices/tea does exactly this (check:tests:unit
  #      calls phpunit; runTests.sh starts .Build/bin/phpunit in a container),
  #      and 17 of 25 netresearch/t3x-* extensions did on 2026-10-07. tea names
  #      its scripts check:tests:unit / check:tests:functional, so those count too.
  # runTests.sh itself stays mandatory (TT-50) either way.
  # Requiring shape 1 alone reported an error-free repo as non-conformant.
  # A composer.json with neither shape still fails: it defines no test entry
  # point at all.
  #
  # Read structurally out of `.scripts`, not as a regex over the whole file:
  # `scripts-descriptions` is a real composer field and NR extensions document
  # exactly these script names in it, so the text match passed a composer.json
  # whose `scripts` object defines no test entry point.
  - id: TT-55
    type: json_path
    target: composer.json
    pattern: '(.scripts // {}) | (has("ci:test:php:unit") or has("ci:test:php:functional") or has("check:tests:unit") or has("check:tests:functional") or (tostring | contains("Build/Scripts/runTests.sh")))'
    severity: warning
    desc: "composer.json should define a test entry point — either a script invoking Build/Scripts/runTests.sh, or ci:test:php:unit / ci:test:php:functional scripts, or check:tests:unit / check:tests:functional scripts (TYPO3BestPractices/tea)"

  - id: TT-56
    type: regex
    target: Makefile
    pattern: 'typo3-ci-workflows/Makefile.include'
    severity: info
    desc: "Makefile should include shared targets from typo3-ci-workflows"

  # The TYPO3 standard build layout is .Build/ (TYPO3BestPractices/tea:
  # vendor-dir .Build/vendor, bin-dir .Build/bin, web-dir .Build/public), and
  # runTests.sh, the CI templates and the PHPUnit configs in assets/ assume it.
  # Lowercase .build/vendor and an unset vendor-dir are legacy: a warning, not
  # an error, so existing extensions keep passing until their build setup is
  # touched. typo3-conformance checks the same value as TC-194.
  - id: TT-137
    type: json_path
    target: composer.json
    pattern: '.config["vendor-dir"] == ".Build/vendor"'
    severity: warning
    desc: "composer.json config.vendor-dir should be .Build/vendor (TYPO3 standard, see TYPO3BestPractices/tea); .build/vendor or an unset vendor-dir is a legacy layout - migrate when touching the build setup"

  # === RUNTESTS.SH QUALITY ===
  - id: TT-57
    type: script
    command: |
      f=Build/Scripts/runTests.sh
      [ -f "$f" ] || exit 1
      if grep -q 'typo3-ci-workflows' "$f" && grep -qE '^exec[[:blank:]]+"[$][{]?RUNNER[}]?"' "$f"; then
        vd=$(jq -r '.config["vendor-dir"] // "vendor"' composer.json 2>/dev/null) || vd=vendor
        f="${vd:-vendor}/netresearch/typo3-ci-workflows/assets/Build/Scripts/runTests.sh"
        if [ ! -f "$f" ]; then
          jq -e '((.require // {}) + (."require-dev" // {})) | has("netresearch/typo3-ci-workflows")' composer.json >/dev/null 2>&1
          exit $?
        fi
      fi
      grep -qE 'ghcr\.io/typo3/core-testing|CONTAINER_BIN|docker\s+run' "$f"
    severity: warning
    desc: "runTests.sh should use Docker containers (TYPO3 core-testing images) for consistent test environments"

  # Fires only on the real bug: a make RECIPE that invokes runTests.sh without
  # -s. It no longer demands that the Makefile drive runTests.sh at all, which
  # reported an error against every repo where the Makefile is a thin wrapper
  # over composer scripts (ci:test:php:unit, ...) and runTests.sh is reserved
  # for Docker-based multi-version runs.
  #
  # Anchored on a literal TAB: make recipes must be tab-indented, assignments
  # never are, so `RUNTESTS := Build/Scripts/runTests.sh` — with or without the
  # leading spaces it gets inside an ifeq block — is not mistaken for a call.
  # The runner must also be the recipe's COMMAND WORD, after make's optional
  # `@`/`-`/`+` prefixes: a `help:` target whose `@echo` merely names
  # runTests.sh is a mention, not a call, and matching it anywhere on the line
  # reported an error against a correct Makefile. `[)}]` accepts both
  # $(RUNTESTS) and ${RUNTESTS}; `\S*runTests\.sh` accepts the direct path.
  # The lookahead keeps this a single grep: a `grep … | grep -q …` pipeline
  # reports the opposite verdict under `set -o pipefail`, because grep -q
  # closes the pipe early and the SIGPIPE on the left-hand grep becomes the
  # pipeline's status.
  # Known limitations: a call split across a backslash line-continuation (also
  # true of the regex this replaced), and a call that is not in command
  # position (`cd Build && $(RUNTESTS) unit`), are not seen. Needs grep -P:
  # where PCRE is missing, `grep -qP` exits 2, the leading `!` inverts that and
  # the check passes — it fails open instead of reporting every Makefile as
  # broken. TT-35 and TT-105 do not rely on that; they were switched to a
  # PCRE-free extraction, because their PCRE spelling failed CLOSED.
  - id: TT-58
    type: command
    pattern: "! grep -qP '^\\t[@+-]*([$][({]RUNTESTS[)}]|\\S*runTests\\.sh)(?!.* -s[ =])' Makefile 2>/dev/null"
    severity: error
    desc: "Makefile recipes that invoke runTests.sh must pass the -s flag (e.g. $(RUNTESTS) -s unit, not $(RUNTESTS) unit). A Makefile that wraps composer scripts instead, or no Makefile at all, satisfies this check."

  - id: TT-59
    type: script
    command: |
      f=Build/Scripts/runTests.sh
      [ -f "$f" ] || exit 1
      if grep -q 'typo3-ci-workflows' "$f" && grep -qE '^exec[[:blank:]]+"[$][{]?RUNNER[}]?"' "$f"; then
        vd=$(jq -r '.config["vendor-dir"] // "vendor"' composer.json 2>/dev/null) || vd=vendor
        f="${vd:-vendor}/netresearch/typo3-ci-workflows/assets/Build/Scripts/runTests.sh"
        if [ ! -f "$f" ]; then
          jq -e '((.require // {}) + (."require-dev" // {})) | has("netresearch/typo3-ci-workflows")' composer.json >/dev/null 2>&1
          exit $?
        fi
      fi
      grep -qE 'NETWORK=.*\$\{?SUFFIX' "$f"
    severity: warning
    desc: "Docker network name should include per-run suffix to avoid conflicts with concurrent runs"

  # === COMPOSER.JSON TEST DEPENDENCIES ===
  # TT-60 removed: phpunit/phpunit is pulled transitively via typo3/testing-framework.
  # Requiring it directly breaks the PHP-8.2 matrix cell because phpunit 12.5.8+
  # requires PHP ≥ 8.3. Do NOT add a direct phpunit/phpunit require-dev entry.

  # `||`-chained on one line, so the runner rejected it unread and it failed on
  # every project, including ones that do have PHPStan.
  #
  # The composer.json branch reads declared dependency KEYS. `grep -q phpstan
  # composer.json` also passed on a `description` or `scripts-descriptions`
  # entry that merely names PHPStan, so the warning was suppressed for projects
  # that never install it. netresearch/typo3-ci-workflows counts as a key,
  # because it is the shared CI package this check means by "transitively".
  - id: TT-61
    type: script
    command: |
      [ -f .Build/bin/phpstan ] && exit 0
      [ -f vendor/bin/phpstan ] && exit 0
      jq -e '((.require // {}) + (."require-dev" // {})) | keys | any(test("phpstan") or . == "netresearch/typo3-ci-workflows")' composer.json >/dev/null 2>&1 && exit 0
      grep -rq phpstan .Build/vendor/composer/installed.json 2>/dev/null && exit 0
      grep -rq phpstan vendor/composer/installed.json 2>/dev/null && exit 0
      echo "FAIL: PHPStan is not available directly or transitively"
      exit 1
    severity: warning
    desc: "PHPStan should be available (directly or transitively via a shared CI package)"

  - id: TT-62
    type: json_path
    target: composer.json
    pattern: '.scripts.test // .scripts["ci:test"] // .scripts["test:unit"] // .scripts["ci:test:php:unit"] // .scripts["check:tests:unit"]'
    severity: warning
    desc: "composer.json should define test script (test, ci:test, test:unit, ci:test:php:unit, or check:tests:unit as in TYPO3BestPractices/tea)"

# === ARCHITECTURE TESTING ===
  - id: TT-14
    type: file_exists
    target: Tests/Architecture/
    severity: info
    desc: "Tests/Architecture/ directory should exist for phpat architecture tests"

  - id: TT-15
    type: json_path
    target: composer.json
    pattern: '.["require-dev"]["phpat/phpat"] // .["require-dev"]["netresearch/typo3-ci-workflows"]'
    severity: warning
    desc: "composer.json should require phpat/phpat for architecture testing (package was renamed from carlosas/phpat in 2023; also satisfied when netresearch/typo3-ci-workflows meta-package is present, as phpat is transitive)"

  # === E2E TESTING (Playwright) ===
  - id: TT-16
    type: file_exists
    target: "{playwright.config.ts,playwright.config.js,Build/playwright.config.ts,Tests/E2E/playwright.config.ts}"
    severity: info
    desc: "Playwright config should exist for E2E testing (root, Build/, or Tests/E2E/)"

  - id: TT-17
    type: file_exists
    target: "{package.json,Build/package.json,Tests/E2E/package.json,Tests/E2E/Playwright/package.json}"
    severity: info
    desc: "package.json should exist for Playwright/JS test dependencies (root or Build/ or Tests/E2E/ subpath)"

  - id: TT-17a
    type: command
    command: |
      cfg=""
      for f in Build/playwright.config.ts playwright.config.ts; do
        [ -f "$f" ] && cfg="$f" && break
      done
      [ -z "$cfg" ] && exit 0
      grep -q 'chromium' "$cfg" && grep -q 'firefox' "$cfg" && exit 0
      echo "FAIL: $cfg should define at least Chromium and Firefox projects"
      exit 1
    severity: warning
    desc: "If playwright.config.ts exists, it should define at least Chromium and Firefox projects for cross-browser E2E coverage"

  # A suite that exists but is never invoked is not a gate, and it reads like one
  # in a release note. nr_passkeys_fe 1.0.0 shipped 19 specifications that every
  # file skipped and no workflow ran.
  - id: TT-17c
    type: command
    command: |
      specs=$(find Tests/E2E -name '*.spec.ts' 2>/dev/null)
      [ -z "$specs" ] && exit 0
      # An executable invocation, not a mention: a `run:` line that starts the
      # suite, or a caller of a shared e2e workflow. A cache key or a comment
      # naming playwright is not a run.
      grep -rhE '^\s*(-?\s*(run|uses):).*(npx playwright|playwright test|runTests\.sh -s e2e|ci-e2e|/e2e\.ya?ml@)' \
        .github/workflows/ 2>/dev/null | grep -q . && exit 0
      echo "FAIL: Tests/E2E holds specifications but no workflow invokes them"
      exit 1
    severity: error
    desc: "If Tests/E2E holds spec files, a workflow must invoke the suite — an uninvoked suite is not a gate"

  - id: TT-17d
    type: command
    command: |
      specs=$(find Tests/E2E -name '*.spec.ts' 2>/dev/null)
      [ -z "$specs" ] && exit 0
      # Column 0 is what separates a file-level skip from a conditional one
      # inside a test body, which is legitimate and always indented.
      skipped=$(printf '%s\n' "$specs" | xargs grep -lE '^(test\.skip|test\.describe\.skip|describe\.skip)\(' 2>/dev/null | wc -l)
      total=$(printf '%s\n' "$specs" | wc -l)
      [ "$skipped" -eq 0 ] && exit 0
      echo "FAIL: $skipped of $total E2E spec files are disabled by a file-level skip"
      exit 1
    severity: error
    desc: "No E2E spec file may be disabled by a file-level test.skip()/test.describe.skip() — a skipped suite passes CI while testing nothing"

  # === MUTATION TESTING ===
  - id: TT-18
    type: file_exists
    target: "{infection.json5,infection.json.dist,infection.json,Build/infection.json5,Build/infection.json.dist}"
    severity: info
    desc: "Infection config should exist for mutation testing (accepts .json5, .json.dist, .json)"

  - id: TT-18a
    type: command
    command: |
      test -f infection.json5 || exit 0
      grep -rql 'infection\|mutation\|ci:test:php:mutation' .github/workflows/*.yml 2>/dev/null && exit 0
      echo "FAIL: infection.json5 exists but mutation testing is not configured in CI"
      exit 1
    severity: warning
    desc: "If infection.json5 exists, mutation testing should be configured in CI (at least as a scheduled job)"

  # === ADDITIONAL DEV DEPENDENCIES ===
  # Accept either a direct require-dev declaration OR transitive availability
  # via the netresearch/typo3-ci-workflows meta-package, which bundles
  # php-cs-fixer, typo3/coding-standards, rector, phpstan, infection, etc.
  # Same pattern as TT-15 (phpat) — see references/ci-workflows-meta-package.md
  - id: TT-63
    type: json_path
    target: composer.json
    pattern: '.["require-dev"]["friendsofphp/php-cs-fixer"] // .["require-dev"]["typo3/coding-standards"] // .["require-dev"]["netresearch/typo3-ci-workflows"]'
    severity: warning
    desc: "composer.json should require php-cs-fixer or typo3/coding-standards in require-dev (also satisfied transitively via netresearch/typo3-ci-workflows)"

  - id: TT-64
    type: json_path
    target: composer.json
    pattern: '.["require-dev"]["rector/rector"] // .["require-dev"]["netresearch/typo3-ci-workflows"]'
    severity: info
    desc: "composer.json should require rector/rector in require-dev (also satisfied transitively via netresearch/typo3-ci-workflows)"

  # === CI MATRIX BEST PRACTICES ===
  - id: TT-25
    type: regex
    target: .github/workflows/*.yml
    pattern: 'fail-fast:\s*false'
    severity: info
    desc: "CI matrix should use fail-fast: false to run all combinations"

  # === TEST ENVIRONMENT GUARDS ===
  - id: TT-75
    type: regex
    target: Tests/**/*Test.php
    pattern: 'tearDown\s*\(\s*\)\s*:\s*void'
    severity: warning
    desc: "Test classes creating filesystem artifacts (tempDir, tempFile) must implement tearDown() for cleanup"
    note: "Check manually: only flag when setUp() creates files/dirs but tearDown() is missing"

  # Was `type: regex` — it FAILED every test suite that does not use the
  # attribute and passed the ones that do. The attribute exists from PHPUnit
  # 12.5.2 on; PHPUnit 11.5 skips attributes whose class does not exist
  # (Metadata/Parser/AttributeParser.php, `if (!class_exists(...)) continue`),
  # so on 11.5 it has no effect, it is not fatal. Kept as an info-level hint
  # towards createStub(), the approach references/unit-testing.md prefers.
  - id: TT-76
    type: regex_not
    target: Tests/**/*Test.php
    pattern: 'AllowMockObjectsWithoutExpectations'
    severity: info
    provenance: upstream
    source: https://github.com/sebastianbergmann/phpunit/blob/12.5.2/src/Framework/Attributes/AllowMockObjectsWithoutExpectations.php
    verified: 2026-09-27
    desc: "Prefer createStub() for test doubles without expectations: #[AllowMockObjectsWithoutExpectations] exists only from PHPUnit 12.5.2 and has no effect on PHPUnit 11.5"

  # === NEGATIVE CHECKS (should NOT exist) ===

  - id: TT-90
    type: regex_not
    target: .github/workflows/*.yml
    pattern: 'ddev start|ddev exec|ddev-github-action|setup-ddev'
    severity: error
    desc: "DDEV must not be used in CI/CD. Use PHP built-in server or Docker containers."
    tags: [ci, ddev, e2e]

  # === GIT HOOKS ===
  # Netresearch convention: keep testing/CI config under Build/ so the repo
  # root stays focused on end-user files (README, LICENSE, composer.json,
  # ext_emconf.php). captainhook/hook-installer reads the config path from
  # composer.json "extra.captainhook.config", so Build/captainhook.json is
  # the enforced default.
  - id: TT-71
    type: file_exists
    target: "{Build/captainhook.json,captainhook.json,.captainhook/captainhook.json}"
    severity: error
    desc: >-
      captainhook config must exist for local git hook automation (pre-commit,
      commit-msg, pre-push) — mandatory for Netresearch extensions.
      Netresearch default: Build/captainhook.json (declared in composer.json
      "extra.captainhook.config"). Root captainhook.json or
      .captainhook/captainhook.json are accepted for backward compatibility
      but root location is flagged by TT-71a.

  - id: TT-71a
    type: file_not_exists
    target: captainhook.json
    severity: warning
    desc: >-
      captainhook.json should live under Build/captainhook.json (Netresearch
      convention — keeps repo root clean). Move to Build/ and declare
      "extra": {"captainhook": {"config": "Build/captainhook.json"}} in
      composer.json so captainhook/hook-installer picks it up.

  - id: TT-71b
    type: json_path
    target: composer.json
    pattern: '.extra.captainhook.config == "Build/captainhook.json"'
    severity: warning
    desc: >-
      composer.json should declare extra.captainhook.config:
      "Build/captainhook.json" so the captainhook/hook-installer plugin
      installs hooks using the Build/ subdir config on every composer install.

  # === GITIGNORE CHECKS ===
  - id: TT-70
    type: contains
    target: .gitignore
    pattern: "composer.lock"
    severity: error
    desc: "composer.lock should be in .gitignore (TYPO3 extensions should not commit it)"

  # === ADDITIONAL TEST INFRASTRUCTURE ===
  - id: TT-103
    type: json_path
    target: composer.json
    pattern: '.scripts["ci:test:php:functional"]'
    severity: warning
    desc: "composer.json should define ci:test:php:functional script for running functional tests via composer"

  - id: TT-104
    type: json_path
    target: composer.json
    pattern: '.scripts["ci:test:php:mutation"]'
    severity: info
    desc: "composer.json should define ci:test:php:mutation script for running mutation tests via composer"

  # === MUTATION TESTING QUALITY ===
  # `||`-chained and relying on an `expected:` field the runner never reads, so it
  # was rejected unread and failed even where infection.json5 sets a high minMsi.
  # Not applicable when the project does not do mutation testing at all — TT-104
  # already reports the missing configuration.
  #
  # PCRE-free extraction for the same reason as TT-35: with `grep -oP …\K` on a
  # grep without PCRE the capture is empty and the check reported "declares no
  # minMsi" against a config that declares one.
  #
  # Reads minCoveredMsi as well as minMsi and judges the higher of the two. Since
  # Infection 0.31 uncovered code is not mutated without --with-uncovered, so MSI
  # equals Covered MSI and the higher threshold is the one that gates. The skill
  # recommends minCoveredMsi alone (references/mutation-testing.md), which this
  # check used to reject as "declares no minMsi".
  #
  # json5_num reads a key the way Infection does: `//` and `/* */` comments are
  # dropped first (outside strings, so the `https://` of `$schema` survives), so
  # a commented-out `// "minCoveredMsi": 95` is not taken for the active value.
  # Keys may be double-quoted, single-quoted or bare, values may be decimal
  # (both JSON5). A string value such as `"90"` is not a number and reads as
  # unset. Plain POSIX awk; TT-106 carries the same function.
  - id: TT-105
    type: script
    command: |
      [ -f infection.json5 ] || exit 0
      json5_num() {
        awk -v key="$1" '
          { text = text $0 "\n" }
          END {
            n = length(text); i = 1; q = ""; out = ""
            while (i <= n) {
              c = substr(text, i, 1)
              if (q != "") {
                out = out c
                if (c == "\\") { out = out substr(text, i + 1, 1); i += 2; continue }
                if (c == q) q = ""
                i++; continue
              }
              d = substr(text, i, 2)
              if (d == "//") { j = index(substr(text, i), "\n"); if (j == 0) break; i += j - 1; continue }
              if (d == "/*") { j = index(substr(text, i + 2), "*/"); if (j == 0) break; out = out " "; i += j + 3; continue }
              if (c == "\"" || c == "\047") q = c
              out = out c; i++
            }
            re = "(^|[^A-Za-z0-9_$])[\"\047]?" key "[\"\047]?[ \t\r\n]*:[ \t\r\n]*[0-9]+([.][0-9]+)?"
            if (match(out, re)) {
              v = substr(out, RSTART, RLENGTH)
              sub(/^.*:[ \t\r\n]*/, "", v)
              print v
            }
          }' "$2" 2>/dev/null
      }
      msi=$(json5_num minMsi infection.json5)
      cmsi=$(json5_num minCoveredMsi infection.json5)
      if [ -z "$msi" ] && [ -z "$cmsi" ]; then
        echo "FAIL: infection.json5 declares neither minCoveredMsi nor minMsi"
        exit 1
      fi
      gate=$(awk -v a="$msi" -v b="$cmsi" 'BEGIN { if (a == "" || (b != "" && b + 0 >= a + 0)) print b; else print a }')
      awk -v g="$gate" 'BEGIN { exit !(g + 0 >= 90) }' && exit 0
      echo "FAIL: infection MSI threshold is $gate (minCoveredMsi=${cmsi:-unset}, minMsi=${msi:-unset}); expected >= 90"
      exit 1
    severity: warning
    desc: "Infection minCoveredMsi (or minMsi) should be >= 90% for mature test suites"

  # Printed "consistent" or "mismatch: …" and relied on an `expected:` field the
  # runner never reads; the runner judges a script by its exit status, and the
  # final `echo` made that 0 either way, so a mismatch passed. It now exits 1 on
  # a mismatch. Only a threshold declared in BOTH files can disagree; a missing
  # file or a file without the key is TT-104's and TT-105's concern. Compares
  # minMsi and minCoveredMsi separately, because a config that follows the
  # skill declares only minCoveredMsi and would otherwise never be compared.
  #
  # PCRE-free extraction for the same reason as TT-35 and TT-105, with the same
  # json5_num as TT-105: comments are ignored, whitespace before the colon
  # (`"minMsi" : 90`), single-quoted and bare keys are read. Values compare as
  # numbers, so 80 and 80.0 agree and 80 and 80.5 do not.
  - id: TT-106
    type: script
    command: |
      [ -f infection.json5 ] && [ -f infection-full.json5 ] || exit 0
      json5_num() {
        awk -v key="$1" '
          { text = text $0 "\n" }
          END {
            n = length(text); i = 1; q = ""; out = ""
            while (i <= n) {
              c = substr(text, i, 1)
              if (q != "") {
                out = out c
                if (c == "\\") { out = out substr(text, i + 1, 1); i += 2; continue }
                if (c == q) q = ""
                i++; continue
              }
              d = substr(text, i, 2)
              if (d == "//") { j = index(substr(text, i), "\n"); if (j == 0) break; i += j - 1; continue }
              if (d == "/*") { j = index(substr(text, i + 2), "*/"); if (j == 0) break; out = out " "; i += j + 3; continue }
              if (c == "\"" || c == "\047") q = c
              out = out c; i++
            }
            re = "(^|[^A-Za-z0-9_$])[\"\047]?" key "[\"\047]?[ \t\r\n]*:[ \t\r\n]*[0-9]+([.][0-9]+)?"
            if (match(out, re)) {
              v = substr(out, RSTART, RLENGTH)
              sub(/^.*:[ \t\r\n]*/, "", v)
              print v
            }
          }' "$2" 2>/dev/null
      }
      rc=0
      for key in minMsi minCoveredMsi; do
        v1=$(json5_num "$key" infection.json5)
        v2=$(json5_num "$key" infection-full.json5)
        [ -z "$v1" ] || [ -z "$v2" ] && continue
        awk -v a="$v1" -v b="$v2" 'BEGIN { exit !(a + 0 == b + 0) }' && continue
        echo "FAIL: $key is $v1 in infection.json5 and $v2 in infection-full.json5"
        rc=1
      done
      exit "$rc"
    severity: warning
    desc: "All infection config files must use consistent MSI thresholds"

  # === TEST QUALITY PATTERNS ===
  - id: TT-110
    type: regex_not
    target: Tests/**/*Test.php
    pattern: 'assertEquals\s*\(\s*null\s*,'
    severity: warning
    desc: "Use assertNull() instead of assertEquals(null, ...) for clearer intent and better failure messages"

  - id: TT-111
    type: file_exists
    target: Tests/Functional/Fixtures/
    severity: info
    desc: "Tests/Functional/Fixtures/ directory should exist for CSV fixture data used by functional tests"

  # === TEST ANTI-PATTERNS ===
  - id: TT-130
    type: regex_not
    target: Tests/**/*Test.php
    pattern: "markTestSkipped\\(['\"].*(?:GD|gd|Imagick|imagick).*extension"
    severity: error
    desc: "Tests must not skip for missing image extensions — run tests in Docker via runTests.sh where GD/Imagick are available"

  - id: TT-131
    type: regex_not
    target: Tests/**/*Test.php
    pattern: 'createMock\(.*ImageManager::class\)'
    severity: warning
    desc: "Do not mock final classes (e.g., Intervention ImageManager) — use real instances or interface mocks"

  # === TEST COVERAGE COMPLETENESS ===
  # `-not -path` excludes a dependency tree vendored underneath the search
  # root. The exclusions are spelled `*/vendor/*` rather than `./vendor/*`:
  # find starts at Classes/Service, so it emits paths beginning `Classes/`,
  # and a `./`-anchored pattern could never match one.
  #
  # The counter used to be incremented inside `find … | while …`, which bash
  # runs in a subshell: `missing` came back 0 no matter how many classes were
  # reported, so the check printed its findings and still exited 0 — it could
  # not fail. Process substitution keeps the loop in the current shell.
  #
  # Spelled `type: script` + `command: |` — the only multi-line shape both the
  # runner and validate-checkpoints.sh accept (see TD-05 in typo3-docs).
  #
  # Only files that declare behaviour need a test of their own: interfaces,
  # abstract classes (tested through their concrete subclasses) and classes or
  # enums without a single method (constants holders, backed enums) are
  # skipped. The PHP tokenizer decides, so a docblock mentioning `interface`
  # does not count, and neither does a `use function` import.
  - id: TT-120
    type: script
    command: |
      missing=0
      while IFS= read -r f; do
        base=$(basename "$f" .php)
        rel=${f#Classes/}
        dir=$(dirname "$rel")
        if [ ! -f "Tests/Unit/${dir}/${base}Test.php" ] && [ ! -f "Tests/Functional/${dir}/${base}Test.php" ]; then
          echo "Missing test for $f"
          missing=$((missing+1))
        fi
      done < <(find Classes/Service -name '*.php' -not -path '*/.Build/*' -not -path '*/vendor/*' -not -path '*/node_modules/*' 2>/dev/null | php -r '
      foreach (array_filter(explode("\n", stream_get_contents(STDIN))) as $f) {
          $t = array_values(array_filter(token_get_all(file_get_contents($f)), fn($x) => !(is_array($x) && in_array($x[0], [T_WHITESPACE, T_COMMENT, T_DOC_COMMENT], true))));
          $kind = null; $abstract = false; $methods = 0;
          foreach ($t as $i => $x) {
              if (!is_array($x)) continue;
              if ($kind === null && in_array($x[0], [T_CLASS, T_INTERFACE, T_TRAIT, T_ENUM], true) && !(is_array($t[$i - 1] ?? null) && $t[$i - 1][0] === T_DOUBLE_COLON)) {
                  $kind = $x[0];
                  for ($j = $i - 1; $j >= 0 && is_array($t[$j]) && in_array($t[$j][0], [T_ABSTRACT, T_FINAL, T_READONLY], true); $j--) {
                      if ($t[$j][0] === T_ABSTRACT) $abstract = true;
                  }
              }
              if ($x[0] === T_FUNCTION && !(is_array($t[$i - 1] ?? null) && $t[$i - 1][0] === T_USE)) $methods++; // `use function foo;` is an import, not a method
          }
          if ($kind === null || $kind === T_INTERFACE || $abstract || $methods === 0) continue;
          echo $f, "\n";
      }')
      [ "$missing" -eq 0 ]
    severity: warning
    desc: "Every class or enum with methods in Classes/Service/ (including subdirectories) should have a corresponding test; interfaces and abstract classes are exempt"

  # Abstract ViewHelper bases are exempt: they are tested through their
  # concrete subclasses. Interfaces, traits and enums are skipped too, since
  # none of them is a ViewHelper. Unlike TT-120 a concrete class without own
  # methods is NOT
  # exempt: `final class FooViewHelper extends AbstractFooViewHelper {}`
  # inherits render() and is a ViewHelper that templates call, so it needs a test.
  - id: TT-121
    type: script
    command: |
      missing=0
      while IFS= read -r f; do
        base=$(basename "$f" .php)
        rel=${f#Classes/}
        dir=$(dirname "$rel")
        if [ ! -f "Tests/Unit/${dir}/${base}Test.php" ] && [ ! -f "Tests/Functional/${dir}/${base}Test.php" ]; then
          echo "Missing test for $f"
          missing=$((missing+1))
        fi
      done < <(find Classes/ViewHelpers -name '*ViewHelper.php' -not -path '*/.Build/*' -not -path '*/vendor/*' -not -path '*/node_modules/*' 2>/dev/null | php -r '
      foreach (array_filter(explode("\n", stream_get_contents(STDIN))) as $f) {
          $t = array_values(array_filter(token_get_all(file_get_contents($f)), fn($x) => !(is_array($x) && in_array($x[0], [T_WHITESPACE, T_COMMENT, T_DOC_COMMENT], true))));
          $kind = null; $abstract = false;
          foreach ($t as $i => $x) {
              if (!is_array($x)) continue;
              if ($kind === null && in_array($x[0], [T_CLASS, T_INTERFACE, T_TRAIT, T_ENUM], true) && !(is_array($t[$i - 1] ?? null) && $t[$i - 1][0] === T_DOUBLE_COLON)) {
                  $kind = $x[0];
                  for ($j = $i - 1; $j >= 0 && is_array($t[$j]) && in_array($t[$j][0], [T_ABSTRACT, T_FINAL, T_READONLY], true); $j--) {
                      if ($t[$j][0] === T_ABSTRACT) $abstract = true;
                  }
              }
          }
          if ($kind !== T_CLASS || $abstract) continue;
          echo $f, "\n";
      }')
      [ "$missing" -eq 0 ]
    severity: warning
    desc: "Every concrete ViewHelper in Classes/ViewHelpers/ (including subdirectories) should have a corresponding test; abstract bases are exempt"

  # === ADDITIONAL CHECKPOINTS (t3x-nr-vault v0.5.0 session) ===

  - id: TT-132
    type: regex_not
    target: Tests/E2E/**/*.spec.ts
    pattern: "(['\"])\\s*/(?:home|Users)/[^'\"/]+/[^'\"]*\\1|(['\"])C:\\\\\\\\"
    severity: error
    desc: "E2E specs must not contain hardcoded absolute local paths (/home/username/..., /Users/username/..., C:\\...) — breaks CI and every other developer's machine. Use process.cwd() or paths relative to the test file instead."
    note: "Real bug caught in t3x-nr-vault review: hardcoded /home/cybot/projects/.../main in an execFileSync cwd."

  - id: TT-135
    type: regex_not
    target: Tests/Functional/**/*Test.php
    pattern: 'function setUp\(\)[^}]*(?:(?:self|static)::markTestSkipped|\$this->markTestSkipped)[^}]*parent::setUp\(\)'
    multiline: true
    severity: error
    desc: >-
      markTestSkipped() must not be called before parent::setUp() in a FunctionalTestCase::setUp().
      Typed properties like $this->instancePath are uninitialised until parent::setUp() runs;
      tearDown() then crashes with "accessed before initialization".
      Correct order: parent::setUp() first, then check the skip condition.

llm_reviews:
  # === MOVED FROM mechanical: — type llm_review is not a mechanical type,
  # so the runner skipped these three as 'Unknown checkpoint type' and they
  # never ran anywhere. ===
  - id: TT-133
    type: llm_review
    domain: test-quality
    target: "Tests/Unit/**/*Test.php"
    severity: warning
    desc: "#[CoversClass] target must not appear in phpunit.xml <source><exclude>. PHPUnit 12 emits 'not a valid target for code coverage' when a class is both covered and excluded; failOnWarning=true promotes this to a failure. Use #[CoversNothing] for tests of excluded classes, or remove the class from the exclude list."
  - id: TT-134
    type: llm_review
    domain: test-quality
    target: "Tests/**/*.php"
    severity: error
    desc: "Tests that call vfsStream::url() must call vfsStream::setup() for the same root in setUp(). On a fresh CI run the virtual-filesystem root does not exist; file_put_contents('vfs://keys/...') fails with 'No such file or directory'. Works locally only when a previous test in the same process already registered the root."
  # Same candidate paths as TT-30/TT-32/TT-35: a project whose PHPStan config
  # lives in Build/ was skipped by this review entirely.
  - id: TT-136
    type: llm_review
    domain: static-analysis
    target: "{phpstan.neon,Build/phpstan.neon,Build/phpstan/phpstan.neon}"
    severity: error
    desc: "phpstan.neon must not combine phpstan/extension-installer auto-registration with explicit includes: of the same extension .neon files. When extension-installer is active (via netresearch/typo3-ci-workflows), phpstan-phpunit, phpstan-strict-rules, phpstan-deprecation-rules, phpat, and saschaegerer/phpstan-typo3 are already registered — adding explicit includes: causes 'These files are included multiple times' and PHPStan exits 1. Remove duplicate includes or use Build/phpstan.no-plugins.neon for local runs without extension-installer."

  # === DEPENDENCY FRESHNESS ===
  - id: TT-19
    type: llm_review
    severity: info
    domain: dependencies
    prompt: |
      If package.json exists, check that test framework dependencies
      (vitest, playwright, jest, jsdom) are not more than 2 major
      versions behind the latest release. Outdated test tools can
      cause false positives/negatives and miss new testing features.
    desc: "npm test dependencies not excessively outdated"
    tags: [npm, dependencies, testing]

  # === TEST QUALITY REVIEWS ===
  - id: TT-85
    domain: code-quality
    prompt: |
      Review test files for incorrect #[CoversClass] attribute usage:
      - Each #[CoversClass(Foo::class)] must reference a class that the test
        actually exercises (creates, calls methods on, or asserts behavior of).
      - Flag any test that declares #[CoversClass] on a class it does not
        instantiate, mock, or invoke methods on within any test method.
      - Common mistake: copy-pasting CoversClass from another test file.

      Examine Tests/Unit/**/*Test.php and Tests/Functional/**/*Test.php.
    severity: warning
    desc: "Verify #[CoversClass] attributes reference classes the test actually exercises"

  - id: TT-80
    domain: code-quality
    prompt: |
      Review the test directory structure and naming conventions:
      - Tests should be in Tests/Unit/ and Tests/Functional/
      - Test class names should end with "Test"
      - Test methods should start with "test" or use #[Test] attribute
      - Namespace should mirror Classes/ structure

      Check Tests/ directory and report compliance.
    severity: warning
    desc: "Verify test structure follows TYPO3 conventions"

  - id: TT-81
    domain: code-quality
    prompt: |
      Review PHPUnit configuration for best practices:
      - Bootstrap file should be configured
      - Test suites should be properly defined
      - Code coverage filter should exclude Tests/ and vendor/
      - Fail on warnings/risky should be enabled

      Examine phpunit.xml, Build/UnitTests.xml, Build/phpunit.xml or
      Build/phpunit/UnitTests.xml and report findings.
    severity: info
    desc: "Verify PHPUnit configuration follows best practices"

  - id: TT-82
    domain: code-quality
    prompt: |
      Review CI test matrix configuration:
      - Should test against supported PHP versions (8.2, 8.3, 8.4, 8.5)
      - Should test against supported TYPO3 versions (v13, v14)
      - If using netresearch/typo3-ci-workflows, check that php-versions
        and typo3-versions inputs are properly configured
      - Should have reasonable timeout limits

      Examine .github/workflows/*.yml and report matrix coverage.
    severity: warning
    desc: "Verify CI matrix covers required PHP and TYPO3 versions"

  - id: TT-83
    domain: code-quality
    prompt: |
      Review PHPStan configuration for TYPO3 extension development:
      - Should include saschaegerer/phpstan-typo3 extension
      - Level should be 6 or higher (ideally 9/max, or 10 with PHPStan 2.x)
      - Should scan Classes/ directory
      - Baseline file is acceptable for legacy code

      Examine phpstan.neon (root or Build/ directory) and report quality.
    severity: info
    desc: "Verify PHPStan is properly configured for TYPO3"

  - id: TT-84
    domain: code-quality
    prompt: |
      Review test coverage and quality indicators:
      - Are there tests for core functionality?
      - Do tests cover edge cases and error conditions?
      - Are mocks used appropriately?
      - Is test isolation maintained?

      Sample Tests/Unit/ and Tests/Functional/ directories and assess quality.
    severity: info
    desc: "Assess overall test coverage and quality"

  # === TEST INFRASTRUCTURE ANTI-PATTERNS ===
  - id: TT-91
    domain: test-quality
    prompt: |
      Review test code for markTestSkipped guards that hide missing Docker infrastructure:
      1. Find all uses of markTestSkipped() or $this->markTestSkipped() in test files
      2. Check if any skip guards test for PHP extensions (GD, Imagick, etc.) or
         external tools that should be available in the Docker test environment
      3. Tests for image/file/network operations should always run in containerized
         environments via runTests.sh, not be skipped on bare-metal
      4. Flag any markTestSkipped() that tests for extensions or binaries that the
         Docker container should provide
    severity: warning
    desc: "Review test code for markTestSkipped guards that hide missing Docker infrastructure — tests for image/file/network operations should always run in containerized environments"

  - id: TT-92
    domain: test-quality
    prompt: |
      Review test code for mocking of final or readonly classes:
      1. Find all createMock() and getMockBuilder() calls in test files
      2. Resolve the target class and check if it is declared final or readonly
         (e.g., Intervention\Image\ImageManager is final in v3+)
      3. Mocking a final class causes a runtime error in PHPUnit
      4. Suggest alternatives: use real instances, mock interfaces instead,
         or use adapter patterns that wrap the final class
      5. Flag any createMock(FinalClass::class) usage
    severity: warning
    desc: "Review for mocking of final or readonly classes — use real instances, interface mocks, or adapter patterns instead of createMock() on final classes"

  # === TEST COVERAGE COMPLETENESS ===
  - id: TT-87
    domain: code-quality
    prompt: |
      Check every class in Classes/Service/ has a corresponding test in Tests/Unit/Service/ or Tests/Functional/Service/:
      1. List all PHP classes in Classes/Service/
      2. For each class, check if a matching test file exists in Tests/Unit/Service/ or Tests/Functional/Service/
      3. Flag any service class without a corresponding test
    severity: warning
    desc: "Every service class in Classes/Service/ should have a test in Tests/Unit/Service/ or Tests/Functional/Service/"

  - id: TT-88
    domain: code-quality
    prompt: |
      Check every ViewHelper in Classes/ViewHelpers/ has a corresponding test:
      1. List all ViewHelper classes in Classes/ViewHelpers/ (recursively)
      2. For each ViewHelper, check if a matching test file exists in Tests/Unit/ViewHelpers/ or Tests/Functional/ViewHelpers/
      3. Flag any ViewHelper without a corresponding test
    severity: warning
    desc: "Every ViewHelper in Classes/ViewHelpers/ should have a corresponding test"

  # === EVENT DISPATCH AND ERROR HANDLING ===
  - id: TT-77
    domain: testing
    prompt: |
      Review test coverage for event dispatch error handling:
      1. Find all try/catch blocks that wrap EventDispatcherInterface::dispatch() calls
      2. Verify each catch block has a dedicated test that triggers the exception path
      3. Check that tests mock the EventDispatcher to throw exceptions and verify the catch behavior
      4. Flag any event dispatch catch block without corresponding test coverage
    severity: warning
    desc: "Event dispatch catch blocks must have dedicated tests that verify exception handling behavior"

  - id: TT-78
    domain: testing
    prompt: |
      Review test coverage for PHP error suppression and trigger_error:
      1. Find all uses of @ error suppression operator in production code
      2. Verify each @-suppressed call has tests covering both success and failure paths
      3. Find functions that are known to trigger PHP warnings on failure (e.g., getimagesize, file_get_contents, mkdir, unlink, fopen)
      4. Verify tests exist that trigger the warning path (e.g., invalid file input)
      5. Check that tests don't just suppress warnings but explicitly test error handling
      6. Find all trigger_error() calls in production code and the code paths that rely on them
      7. Verify tests assert how each triggered error is handled or propagated (e.g., custom error handlers, exceptions, logging)
    severity: warning
    desc: "Functions with @ suppression, trigger_error(), or that trigger PHP warnings must have tests covering error paths"

  # === CONTROLLER FUNCTIONAL TEST COVERAGE ===
  - id: TT-86
    domain: testing
    prompt: |
      Review backend controller test coverage:
      1. Find all classes in Classes/Controller/ (especially AJAX controllers returning JsonResponse)
      2. For each controller, check that functional tests exist in Tests/Functional/Controller/
      3. Verify that functional tests check:
         a. HTTP status codes (200 for success, 403 for unauthorized, etc.)
         b. Access control (calling without proper BE user session returns 403/401)
         c. CSRF token validation (requests without valid token are rejected)
      4. Flag any backend controller without functional tests covering these three aspects
    severity: warning
    desc: "Every backend controller (especially AJAX controllers returning JSON) should have functional tests verifying HTTP status codes, access control, and CSRF"

  # === CI TEST MATRIX COMPLETENESS ===
  - id: TT-27
    domain: ci
    prompt: |
      Verify that the CI test matrix covers all supported version combinations:
      1. Read the PHP version constraint from composer.json require.php (e.g., "^8.2")
      2. Read the TYPO3 core constraint from composer.json require["typo3/cms-core"] (e.g., "^13.4 || ^14.0")
      3. Determine which PHP minor versions fall in the constraint range (e.g., 8.2, 8.3, 8.4)
      4. Determine which TYPO3 major versions fall in the constraint range (e.g., 13, 14)
      5. Check the CI workflow (.github/workflows/*.yml) matrix or reusable workflow inputs
      6. Flag any PHP version in the supported range that is missing from the CI matrix
      7. Flag any TYPO3 version in the supported range that is missing from the CI matrix
      8. If using netresearch/typo3-ci-workflows, check php-versions and typo3-versions inputs
    severity: warning
    desc: "CI should test all PHP versions in composer.json require.php range x all TYPO3 versions in require[\"typo3/cms-core\"] range"

  # === E2E MULTI-BROWSER COVERAGE ===
  - id: TT-17b
    domain: testing
    prompt: |
      If a Playwright config exists (Build/playwright.config.ts or playwright.config.ts):
      1. Read the projects array in the Playwright config
      2. Verify it defines at least two browser projects: Chromium and Firefox
      3. Optionally check for WebKit as a third browser
      4. Flag if only a single browser is configured, as cross-browser testing catches
         rendering and API differences that single-browser testing misses
    severity: warning
    desc: "If playwright.config.ts exists, it should define at least Chromium and Firefox projects for cross-browser E2E coverage"

  # === PATCH COVERAGE ===
  - id: TT-89
    domain: testing
    prompt: |
      Review whether new code paths introduced in recent changes have corresponding tests:
      1. Identify new public methods in Classes/ that were added or significantly changed
      2. For each new public method, check whether a test exists that calls or exercises it
      3. Focus on Classes/Service/, Classes/Controller/, and Classes/ViewHelpers/ directories
      4. Flag any new public method that has no corresponding test coverage
      5. Ignore getters/setters, constructors, and simple delegation methods
    severity: warning
    desc: "New code paths (especially new public methods) should have corresponding unit or functional tests"

  # === TEST UID SETUP ===
  - id: TT-112
    domain: testing
    prompt: |
      Review test helpers and factories that create entity or model mocks:
      1. Find helper methods that create domain model instances for testing
         (e.g., createSecretEntity, createMockRecord, buildFixture)
      2. For each helper, check whether the production code that consumes
         these entities guards on UID being set (e.g., `if ($uid !== null)`,
         `if ($entity->getUid() > 0)`, `$uid ?? throw`)
      3. If the production code has a UID null-check, verify the test helper
         sets a UID via `_setProperty('uid', ...)` or `setUid(...)` or reflection
      4. Flag any test helper that creates entities without setting UID when
         the consuming production code guards on UID presence
    severity: warning
    desc: "Test helpers creating entity mocks must set UID when production code guards with null/zero UID checks"

  # === MOCK CONSTRUCTOR SYNC ===
  - id: TT-113
    domain: testing
    prompt: |
      Review whether test mocks stay in sync with class constructors:
      1. Find classes in Classes/ whose constructors have changed recently
         or have 3+ parameters (likely to drift)
      2. For each class, find all tests that instantiate it directly
         (new ClassName(...)) or via createMock/getMockBuilder
      3. Check that direct instantiations pass the correct number and types
         of constructor arguments
      4. For createMock/getMockBuilder, check if disableOriginalConstructor()
         is used — if not, the mock will call the real constructor and must
         match its signature
      5. Flag any test where constructor argument count or types are mismatched
         with the current class constructor
    severity: warning
    desc: "When a class constructor gains new parameters, all tests that instantiate it must be updated to match"

  # === MOCK VALIDITY FOR MULTI-VERSION DEPENDENCIES ===
  - id: TT-114
    domain: testing
    prompt: |
      Review test mocks for multi-version dependency compatibility:
      1. Read composer.json require and require-dev for dependencies with
         multi-major-version constraints (e.g., "^3 || ^4", "^2 || ^3")
      2. For each such dependency, find all test files that mock its interfaces
         (grep for createMock/createStub with the dependency's namespace)
      3. For each mocked method, verify the method exists on the **interface**
         (not just the concrete class) in ALL supported major versions
      4. Flag any mock that calls ->method('foo') where foo only exists on
         a concrete implementation or only in one major version
      5. Suggest using an adapter interface pattern when version-specific
         mocking is detected
    severity: warning
    desc: "Test mocks on dependency interfaces must reference methods that exist in ALL supported versions of the dependency"

  # === MOCK CALLBACK SIGNATURE VERIFICATION ===
  - id: TT-115
    domain: testing
    prompt: |
      Review willReturnCallback() usages for signature mismatches:
      1. Find all willReturnCallback() calls in test files
      2. For each callback, identify the method being mocked
      3. Compare the callback's parameter list with the actual method signature
         in the production class or interface
      4. Flag callbacks whose parameter count or types do not match the
         actual method signature
      5. Flag callbacks that are missing parameters that the production code
         now passes (e.g., production added a $quality parameter but the
         callback only accepts $path)
      6. Suggest using variadic signatures (mixed ...$options) when the
         method signature may evolve across dependency versions
    severity: warning
    desc: "willReturnCallback() signatures must match the actual method signature; stale callbacks silently drop arguments"

  # === TEST ASSERTION SPECIFICITY AFTER REFACTORING ===
  - id: TT-116
    domain: testing
    prompt: |
      Review whether test assertions maintain equivalent specificity after
      production code refactoring:
      1. Find test methods that assert on method calls (expects/method/with)
      2. Compare these assertions with the current production code paths
      3. Flag tests where a refactoring changed the production API surface
         but the test assertion lost specificity. Examples:
         - Old: expects(never())->method('toWebp') → New: expects(once())->method('save')
           (lost the "not WebP" assertion)
         - Old: expects(once())->method('resize')->with(800, 600) → New: just expects(once())->method('process')
           (lost dimension verification)
      4. Verify that expects(exactly(N)) with callback assertions include
         argument-level checks (e.g., asserting path extensions, dimensions)
      5. Flag generic willReturnCallback() that only provides return values
         without asserting anything about the arguments
    severity: warning
    desc: "After refactoring, test assertions must maintain equivalent specificity — don't trade specific method assertions for generic ones"

  # === ADAPTER PATTERN TESTING ===
  - id: TT-117
    domain: testing
    prompt: |
      Review whether tests mock third-party library internals directly
      when an adapter pattern would be more appropriate:
      1. Find test files that create mocks of third-party library interfaces
         or classes (namespaces outside the extension's own namespace)
      2. Check if the mocked third-party class has version-specific APIs
         (compare composer.json constraints for multi-major-version support)
      3. Flag tests that create complex mock setups for third-party internals
         (e.g., mocking DriverInterface + constructing real ImageManager)
      4. Suggest introducing an adapter interface owned by the extension
         and mocking that instead, which eliminates version-specific mock setup
      5. Exception: simple, stable interfaces (like PSR interfaces) are fine
         to mock directly
    severity: warning
    desc: "Prefer mocking your own adapter interfaces over third-party library internals to avoid version-specific mock breakage"
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall