Skip to content

netresearch/typo3-testing-skill / skills/typo3-testing

typo3-testing

Flagged

Use when reproducing defects as failing tests, testing template or rendered output changes, confirming changes on a running TYPO3 instance, or setting up TYPO3 extension tests: unit/functional/E2E, PHPUnit 11-13, mutation testing, final-class mocks (v14), CI matrix across TYPO3 12/13/14.3 LTS, typo3-ci-workflows meta-package, CI failures. Also: transaction rollback, worker crashes, competing requests, testing-framework setup, test matrix, integration/e2e testing, coverage, test generation.

FileKindSize
SKILL.mdskill md6.8 KB
assets/.php-cs-fixer.dist.phpscript7.0 KB
assets/AGENTS.mdmarkdown5.0 KB
assets/Build/Scripts/runTests.shexecutablescript27.5 KB
assets/Build/playwright/.nvmrcother6 B
assets/Build/playwright/package.jsonmanifest535 B
assets/Build/playwright/playwright.config.tsscript1.3 KB
assets/Build/playwright/tests/playwright/accessibility/modules.spec.tsscript2.0 KB
assets/Build/playwright/tests/playwright/config.tsscript712 B
assets/Build/playwright/tests/playwright/e2e/backend-module.spec.tsscript1.9 KB
assets/Build/playwright/tests/playwright/fixtures/setup-fixtures.tsscript2.7 KB
assets/Build/playwright/tests/playwright/helper/login.setup.tsscript975 B
assets/FunctionalTests.xmlconfig1.4 KB
assets/FunctionalTestsBootstrap.phpscript1.6 KB
assets/Makefileother2.3 KB
assets/UnitTests.xmlconfig890 B
assets/UnitTestsBootstrap.phpscript2.1 KB
assets/bootstrap.phpscript1.4 KB
assets/codecov.ymlconfig1.6 KB
assets/docker/codeception.ymlconfig979 B
assets/docker/docker-compose.ymlconfig1.4 KB
assets/example-tests/ExampleAcceptanceCest.phpscript2.2 KB
assets/example-tests/ExampleFunctionalTest.phpscript2.4 KB
assets/example-tests/ExampleUnitTest.phpscript1.6 KB
assets/fixtures/README.mdmarkdown2.0 KB
assets/fixtures/be_users.csvother361 B
assets/fixtures/pages.csvother257 B
assets/fixtures/sys_category.csvother228 B
assets/fixtures/tt_content.csvother338 B
assets/github-actions-e2e.ymlconfig8.8 KB
assets/github-actions-tests.ymlconfig6.3 KB
assets/infection.json5other2.8 KB
assets/phpat.neonother330 B
assets/phpat.phpscript8.4 KB
assets/phpstan-baseline.neonother365 B
assets/phpstan.neonother1.4 KB
assets/rector.phpscript2.9 KB
checkpoints.yamlconfig61.7 KB
references/accessibility-testing.mdmarkdown7.6 KB
references/architecture-testing.mdmarkdown4.4 KB
references/asset-templates-guide.mdmarkdown3.4 KB
references/backend-module-render-verification.mdmarkdown9.7 KB
references/backend-user-access-testing.mdmarkdown9.5 KB
references/captainhook-setup.mdmarkdown2.4 KB
references/ci-cd.mdmarkdown39.5 KB
references/ci-debugging.mdmarkdown20.0 KB
references/ci-workflows-meta-package.mdmarkdown11.6 KB
references/crash-and-concurrency-testing.mdmarkdown4.8 KB
references/crypto-testing.mdmarkdown16.6 KB
references/datahandler-silent-rewrites.mdmarkdown7.2 KB
references/ddev-testing.mdmarkdown1.3 KB
references/e2e-testing.mdmarkdown53.0 KB
references/enforcement-rules.mdmarkdown5.7 KB
references/event-dispatch-testing.mdmarkdown2.8 KB
references/framework-compat-gate.mdmarkdown4.9 KB
references/functional-test-patterns.mdmarkdown11.0 KB
references/functional-testing.mdmarkdown46.3 KB
references/fuzz-testing.mdmarkdown8.6 KB
references/integration-testing.mdmarkdown14.1 KB
references/javascript-testing.mdmarkdown29.3 KB
references/live-instance-verification.mdmarkdown8.6 KB
references/mock-validity.mdmarkdown15.7 KB
references/mutation-testing.mdmarkdown36.0 KB
references/performance-testing.mdmarkdown11.2 KB
references/quality-tools.mdmarkdown25.5 KB
references/release-workflow-validation.mdmarkdown3.6 KB
references/sonarcloud.mdmarkdown18.8 KB
references/synthetic-secret-fixtures.mdmarkdown3.8 KB
references/tdd-discipline.mdmarkdown9.1 KB
references/test-environment-guards.mdmarkdown18.5 KB
references/test-runners.mdmarkdown24.6 KB
references/typo3-ci-config-patterns.mdmarkdown9.0 KB
references/typo3-v14-final-classes.mdmarkdown16.5 KB
references/unit-testing.mdmarkdown59.1 KB
scripts/generate-test.shexecutablescript6.0 KB
scripts/setup-testing.shexecutablescript6.3 KB
scripts/validate-setup.shexecutablescript4.0 KB

references/mock-validity.md

View on GitHub
<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
<!-- SPDX-FileCopyrightText: Netresearch DTT GmbH -->

# Mock Validity for Multi-Version Dependencies

When extensions support multiple major versions of a dependency (e.g., `"intervention/image": "^3 || ^4"`), test mocks must remain compatible across all supported versions. This reference covers common pitfalls and patterns.

## The Problem

Mocking a method that exists on an interface in one version but not another causes silent test failures or runtime errors when CI runs against the other version.

```php
// WRONG - The mock targets Intervention\Image\Interfaces\ImageInterface,
// but it configures toWebp(), which is only implemented on the concrete
// Image class in intervention/image v3 and is not declared on ImageInterface
// in any supported version.
$imageMock = $this->createMock(ImageInterface::class);
$imageMock->expects(self::once())
    ->method('toWebp')  // Does not exist on ImageInterface in v4!
    ->willReturn($encodedMock);
```

## Rule: Verify Mocked Methods Exist on the Interface

Before mocking `->method('foo')` on an interface, verify that `foo` is declared on that interface in **all** supported versions of the dependency.

### Verification Checklist

1. Open the interface definition in `vendor/` for the current version
2. Check the method exists on the **interface**, not just concrete implementations
3. Repeat for each major version in the composer constraint
4. If the method only exists on a concrete class, either:
   - Mock the concrete class instead (if not final)
   - Use an adapter pattern (preferred -- see below)
   - Use `method_exists()` guards in the mock setup

### Example: Intervention Image v3 vs v4

```php
// intervention/image v3: Image class has ->toWebp(), ->toJpeg(), etc.
// intervention/image v4: Image class has ->encodeByPath(), ->encodeByExtension()
// Both versions: ImageInterface has ->save(), ->encode()

// WRONG - version-specific method on interface mock
$imageMock = $this->createMock(ImageInterface::class);
$imageMock->method('toWebp')->willReturn($encodedMock);

// CORRECT - use only methods defined on the interface in ALL versions
$imageMock = $this->createMock(ImageInterface::class);
$imageMock->method('save')->willReturn($imageMock);
```

## Scanning Mocks After Dependency Constraint Changes

When widening a dependency constraint (e.g., `"^3"` to `"^3 || ^4"`), perform a mock audit:

```bash
# Find all mocked methods on the dependency's interfaces
grep -rn "->method('" Tests/ | grep -i "intervention\|dependency-name"

# Cross-reference with the interface in each supported version
# Check vendor/intervention/image/src/Interfaces/ImageInterface.php
```

### Automated Check Pattern

```php
/**
 * Verify that all mocked methods exist on the interface.
 * Run this test against each supported dependency version.
 */
#[Test]
public function mockedMethodsExistOnInterface(): void
{
    $interface = new \ReflectionClass(ImageInterface::class);
    $methods = array_map(
        static fn(\ReflectionMethod $m) => $m->getName(),
        $interface->getMethods(),
    );

    // List every method your tests mock on this interface
    $mockedMethods = ['save', 'encode', 'width', 'height'];

    foreach ($mockedMethods as $method) {
        self::assertTrue(
            in_array($method, $methods, true),
            sprintf(
                'Test mocks %s::%s() but this method does not exist on the interface '
                . 'in the installed version. Check all supported versions.',
                ImageInterface::class,
                $method,
            ),
        );
    }
}
```

## Mock Callback Signature Verification

When using `willReturnCallback()`, the callback's parameter signature must match the actual method signature. If production code changes to pass additional arguments, callbacks silently ignore them or fail.

### The Problem

```php
// Production code changed from:
//   $processor->process($path)
// to:
//   $processor->process($path, $quality)

// WRONG - callback signature is stale, ignores $quality
$processorMock->method('process')
    ->willReturnCallback(function (string $path) {
        return '/processed/' . basename($path);
    });

// Test passes but $quality is silently dropped -- assertions on quality are missing
```

### Rule: Match Callback Signatures to Method Signatures

```php
// CORRECT - callback accepts all parameters the real method declares
$processorMock->method('process')
    ->willReturnCallback(function (string $path, int $quality = 80) {
        self::assertSame(75, $quality); // Verify the quality parameter
        return '/processed/' . basename($path);
    });
```

### Use Variadic Signatures for Forward Compatibility

When a method signature may evolve across versions, use a variadic callback:

```php
// FORWARD-COMPATIBLE - accepts any arguments the method might pass
$processorMock->method('process')
    ->willReturnCallback(function (string $path, mixed ...$options): string {
        // $options[0] would be quality if passed
        return '/processed/' . basename($path);
    });
```

### Callback Signature Audit

After any production code change that adds parameters to a method call, search for all test callbacks mocking that method:

```bash
# Find all willReturnCallback usages for a specific method
grep -rn "method('process')" Tests/ | grep -A5 "willReturnCallback"
```

### A Missing `use` in a Callback Signature Passes the Suite

The sibling failure of a stale signature: a callback whose parameter *type* is not imported. PHP resolves a parameter's class type only when a value is actually checked against it, and `null` never is — so a nullable parameter that only ever receives `null` in the test never touches the unresolvable name.

```php
// The test file has no `use ...\ValueObject\AgentRunReference;`
$mgr->method('chatWithTools')->willReturnCallback(
    function (
        array $messages,
        ?AgentRunReference $run = null,   // resolves to the CURRENT namespace
        ?InjectedContext $injected = null,
    ) use (&$seen): CompletionResponse {
        $seen = $injected;

        return $this->response('done');
    },
);
```

Green. Every call in the test passes `null` for `$run`, so PHP never loads
`Netresearch\NrLlm\Tests\Unit\Service\Tool\AgentRunReference` — a class that
does not exist. The day production starts passing a real object, the test dies
with `Argument #2 must be of type ?AgentRunReference` naming a class nobody
recognises.

**PHPStan finds it and the suite cannot:**

```
Parameter $run of anonymous function has invalid type
Netresearch\NrLlm\Tests\Unit\Service\Tool\AgentRunReference.
```

The recommended PHPStan configuration in `quality-tools.md` lists `Tests` in
`paths`, so the analyser does see this — but only if you run it **after**
writing the tests, over the final tree. Ordering matters here in a way it does
not for production code: a green `runTests.sh -s unit` is not evidence that a
new test file is even type-correct, so analysing before the tests exist checks
the half that was already fine.

Two habits close it without waiting for the analyser:

- Copy the parameter list from the interface, then copy its `use` statements.
  A signature pasted without its imports is the whole failure mode.
- Treat every `?Type $x = null` parameter in a test double as unverified until
  something passes a non-null value — either a second test case that does, or
  PHPStan.

## Test Assertion Specificity After Refactoring

When refactoring production code, test assertions must maintain equivalent specificity. A refactoring that changes the API surface (e.g., replacing `toWebp()->save()` with `save('output.webp')`) requires updated assertions that verify the same behavior.

### The Problem

```php
// OLD production code:
//   $image->toWebp()->save($path)
// OLD test assertion:
//   $imageMock->expects(self::never())->method('toWebp')
//   This asserts "WebP conversion never happens"

// NEW production code:
//   $image->save($path)  -- format determined by extension
// NEW test (WRONG - lost specificity):
//   $imageMock->expects(self::once())->method('save')
//   This only asserts "save was called" but not "save was NOT called with .webp"
```

### Rule: Maintain Equivalent Assertion Specificity

```php
// NEW test (CORRECT - equivalent specificity to the old assertion)
$imageMock->expects(self::exactly(2))
    ->method('save')
    ->willReturnCallback(function (string $path) use ($imageMock): ImageInterface {
        // Assert that no .webp save happens (equivalent to old "never toWebp()")
        self::assertStringNotContainsString(
            '.webp',
            $path,
            'Image should not be saved as WebP when WebP is disabled',
        );
        return $imageMock;
    });
```

### Specificity Mapping Guide

When refactoring, map old assertions to new ones:

| Old Assertion | New Equivalent |
|--------------|----------------|
| `expects(never())->method('toWebp')` | `save()` callback asserts path has no `.webp` extension |
| `expects(once())->method('toJpeg')` | `save()` callback asserts path ends with `.jpg` |
| `expects(once())->method('resize')->with(800, 600)` | `save()` callback verifies dimensions if applicable |
| Method-level `expects(exactly(N))` | `expects(exactly(N))` with path-based assertions in callback |

### Anti-Pattern: Generic Callbacks Without Assertions

```php
// WRONG - callback provides return value but asserts nothing
$mock->method('save')
    ->willReturnCallback(fn(string $path) => $mock);

// CORRECT - callback asserts expectations about the arguments
$mock->expects(self::exactly(2))
    ->method('save')
    ->willReturnCallback(function (string $path) use ($mock): ImageInterface {
        self::assertStringEndsWith('.jpg', $path);
        return $mock;
    });
```

### Gotcha: `->with()` Only Constrains the Arguments You Pass

`->with($a)` checks **only** the first argument; any further arguments in the actual
call are unconstrained. So when you add a parameter to a method, existing
single-argument expectations keep passing silently — they neither fail nor verify
the new argument:

```php
// Production change:
//   getCacheLifetime(Context $c)  ->  getCacheLifetime(Context $c, ?int $pageId)

// This existing expectation still PASSES after the change, even though the call
// is now getCacheLifetime($context, 42) — the extra arg is not checked:
$mock->expects(self::once())->method('getCacheLifetime')->with($context);
```

Consequences:

- Widening a signature (N -> N+1 args) will **not** break `->with()` assertions that
  only pin the original args — convenient, but a green suite does not prove the new
  argument is exercised.
- To verify the new argument, pin it explicitly: `->with($context, 42)` (or
  `self::anything()` for positions you intentionally leave open).

### Gotcha: a String in `->with()` Accepts Any `Stringable`

`->with('gpt-image-2')` wraps the literal in `IsEqual`, which compares loosely:
an object whose `__toString()` returns `'gpt-image-2'` satisfies it — measured on
PHPUnit 12.5.6 / PHP 8.5.10, including an object of an unrelated class that
renders the same text. The value-object form rejects it: the failure reads
`Other Object … is not instance of expected class "Name"`. When a
parameter changes from `string` to a value object, the old expectation keeps
passing. That is convenient during the migration, but afterwards it pins only
the string form: any `Stringable` rendering the same text passes, including an
object of the wrong type. Pin the value object once the parameter is typed:

```php
// Passes for new ProviderModelName('gpt-image-2') AND for any other
// Stringable whose __toString() returns 'gpt-image-2':
$repository->expects(self::once())->method('findOneByModelId')
    ->with('gpt-image-2');

// Pins type and value:
$repository->expects(self::once())->method('findOneByModelId')
    ->with(new ProviderModelName('gpt-image-2'));
```

### Gotcha: an Expectation Copied From the Implementation Pins Its Defect

A mock that expects the call the code under test *currently makes* passes by
construction, whether or not that call is right. In one extension a cost
calculator passed an API model name to `findOneByIdentifier()`, which matches a
different column; its unit test stubbed exactly `findOneByIdentifier()`, and
the suite stayed green over a lookup that could never match a real row. Derive
the expected call from the contract — which column, what the argument actually
identifies — rather than from reading the implementation, and back it with one
functional test against real rows.

## Adapter Pattern Testing

When your extension wraps a version-specific third-party API behind an adapter interface, test through the adapter interface rather than creating complex version-specific mock setups.

### The Problem

```php
// WRONG - creates version-specific mocks that break across versions
$driverMock = $this->createMock(DriverInterface::class); // v3-only interface
$driverMock->method('init')->willReturn($driverMock);
$manager = new ImageManager($driverMock);
// ... complex setup that differs between v3 and v4
```

### The Solution: Mock the Adapter Interface

```php
// Define your adapter interface
interface ImageProcessorInterface
{
    public function resize(string $sourcePath, int $width, int $height): string;
    public function convert(string $sourcePath, string $targetFormat): string;
    public function optimize(string $sourcePath, int $quality): string;
}

// Your adapter wraps the version-specific API
final class InterventionImageProcessor implements ImageProcessorInterface
{
    public function __construct(private readonly ImageManager $manager) {}

    public function resize(string $sourcePath, int $width, int $height): string
    {
        $image = $this->manager->read($sourcePath);
        $image->resize($width, $height);
        return $image->save($sourcePath)->basePath(); // Note: This example overwrites the source file.
    }
}
```

```php
// TEST - mock the adapter interface, not the third-party library
final class ImageServiceTest extends UnitTestCase
{
    /** @var ImageProcessorInterface&MockObject */
    private ImageProcessorInterface $processorMock;

    protected function setUp(): void
    {
        parent::setUp();

        /** @var ImageProcessorInterface&MockObject $processorMock */
        $processorMock = $this->createMock(ImageProcessorInterface::class);
        $this->processorMock = $processorMock;

        $this->subject = new ImageService($this->processorMock);
    }

    #[Test]
    public function optimizeImageCallsProcessorWithCorrectQuality(): void
    {
        $this->processorMock
            ->expects(self::once())
            ->method('optimize')
            ->with('/path/to/image.jpg', 75)
            ->willReturn('/path/to/image.jpg');

        $this->subject->optimizeImage('/path/to/image.jpg', 75);
    }
}
```

### Benefits of Adapter Pattern Testing

| Concern | Without Adapter | With Adapter |
|---------|----------------|--------------|
| Version-specific mocks | Required for each version | None needed |
| Test complexity | High (mock internal APIs) | Low (mock your own interface) |
| Breakage on upgrade | Tests break when dependency updates | Only adapter implementation changes |
| Mock validity | Must verify methods on third-party interfaces | Mock your own stable interface |
| Test isolation | Coupled to dependency internals | Fully decoupled |

### When to Use Adapter Pattern

- The dependency has significantly different APIs across supported major versions
- Multiple classes in your extension interact with the dependency
- The dependency's interfaces are not stable across versions
- You need to support `^major1 || ^major2` in `composer.json`

### When NOT to Use Adapter Pattern

- The dependency has a stable, well-defined interface that does not change
- Only one class in your extension uses the dependency
- The overhead of the adapter exceeds the testing benefit
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall