Skip to content

netresearch/typo3-testing-skill / skills/typo3-testing

typo3-testing

Flagged

Use when reproducing defects as failing tests, testing template or rendered output changes, confirming changes on a running TYPO3 instance, or setting up TYPO3 extension tests: unit/functional/E2E, PHPUnit 11-13, mutation testing, final-class mocks (v14), CI matrix across TYPO3 12/13/14.3 LTS, typo3-ci-workflows meta-package, CI failures. Also: transaction rollback, worker crashes, competing requests, testing-framework setup, test matrix, integration/e2e testing, coverage, test generation.

FileKindSize
SKILL.mdskill md6.8 KB
assets/.php-cs-fixer.dist.phpscript7.0 KB
assets/AGENTS.mdmarkdown5.0 KB
assets/Build/Scripts/runTests.shexecutablescript27.5 KB
assets/Build/playwright/.nvmrcother6 B
assets/Build/playwright/package.jsonmanifest535 B
assets/Build/playwright/playwright.config.tsscript1.3 KB
assets/Build/playwright/tests/playwright/accessibility/modules.spec.tsscript2.0 KB
assets/Build/playwright/tests/playwright/config.tsscript712 B
assets/Build/playwright/tests/playwright/e2e/backend-module.spec.tsscript1.9 KB
assets/Build/playwright/tests/playwright/fixtures/setup-fixtures.tsscript2.7 KB
assets/Build/playwright/tests/playwright/helper/login.setup.tsscript975 B
assets/FunctionalTests.xmlconfig1.4 KB
assets/FunctionalTestsBootstrap.phpscript1.6 KB
assets/Makefileother2.3 KB
assets/UnitTests.xmlconfig890 B
assets/UnitTestsBootstrap.phpscript2.1 KB
assets/bootstrap.phpscript1.4 KB
assets/codecov.ymlconfig1.6 KB
assets/docker/codeception.ymlconfig979 B
assets/docker/docker-compose.ymlconfig1.4 KB
assets/example-tests/ExampleAcceptanceCest.phpscript2.2 KB
assets/example-tests/ExampleFunctionalTest.phpscript2.4 KB
assets/example-tests/ExampleUnitTest.phpscript1.6 KB
assets/fixtures/README.mdmarkdown2.0 KB
assets/fixtures/be_users.csvother361 B
assets/fixtures/pages.csvother257 B
assets/fixtures/sys_category.csvother228 B
assets/fixtures/tt_content.csvother338 B
assets/github-actions-e2e.ymlconfig8.8 KB
assets/github-actions-tests.ymlconfig6.3 KB
assets/infection.json5other2.8 KB
assets/phpat.neonother330 B
assets/phpat.phpscript8.4 KB
assets/phpstan-baseline.neonother365 B
assets/phpstan.neonother1.4 KB
assets/rector.phpscript2.9 KB
checkpoints.yamlconfig61.7 KB
references/accessibility-testing.mdmarkdown7.6 KB
references/architecture-testing.mdmarkdown4.4 KB
references/asset-templates-guide.mdmarkdown3.4 KB
references/backend-module-render-verification.mdmarkdown9.7 KB
references/backend-user-access-testing.mdmarkdown9.5 KB
references/captainhook-setup.mdmarkdown2.4 KB
references/ci-cd.mdmarkdown39.5 KB
references/ci-debugging.mdmarkdown20.0 KB
references/ci-workflows-meta-package.mdmarkdown11.6 KB
references/crash-and-concurrency-testing.mdmarkdown4.8 KB
references/crypto-testing.mdmarkdown16.6 KB
references/datahandler-silent-rewrites.mdmarkdown7.2 KB
references/ddev-testing.mdmarkdown1.3 KB
references/e2e-testing.mdmarkdown53.0 KB
references/enforcement-rules.mdmarkdown5.7 KB
references/event-dispatch-testing.mdmarkdown2.8 KB
references/framework-compat-gate.mdmarkdown4.9 KB
references/functional-test-patterns.mdmarkdown11.0 KB
references/functional-testing.mdmarkdown46.3 KB
references/fuzz-testing.mdmarkdown8.6 KB
references/integration-testing.mdmarkdown14.1 KB
references/javascript-testing.mdmarkdown29.3 KB
references/live-instance-verification.mdmarkdown8.6 KB
references/mock-validity.mdmarkdown15.7 KB
references/mutation-testing.mdmarkdown36.0 KB
references/performance-testing.mdmarkdown11.2 KB
references/quality-tools.mdmarkdown25.5 KB
references/release-workflow-validation.mdmarkdown3.6 KB
references/sonarcloud.mdmarkdown18.8 KB
references/synthetic-secret-fixtures.mdmarkdown3.8 KB
references/tdd-discipline.mdmarkdown9.1 KB
references/test-environment-guards.mdmarkdown18.5 KB
references/test-runners.mdmarkdown24.6 KB
references/typo3-ci-config-patterns.mdmarkdown9.0 KB
references/typo3-v14-final-classes.mdmarkdown16.5 KB
references/unit-testing.mdmarkdown59.1 KB
scripts/generate-test.shexecutablescript6.0 KB
scripts/setup-testing.shexecutablescript6.3 KB
scripts/validate-setup.shexecutablescript4.0 KB

references/fuzz-testing.md

View on GitHub
<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
<!-- SPDX-FileCopyrightText: Netresearch DTT GmbH -->

# Fuzz Testing for TYPO3 Extensions

## Overview

Fuzz testing (fuzzing) automatically generates random/mutated inputs to find crashes, memory exhaustion, or unexpected exceptions. This is critical for code that parses untrusted input like HTML, XML, or user data.

> **Key Distinction**: Fuzz testing mutates **inputs** to find bugs. For testing that mutates **code** to verify test quality, see [Mutation Testing](mutation-testing.md).

## When to Use Fuzz Testing

- HTML/XML parsers (e.g., DOMDocument-based code)
- User input processors
- Data transformation services
- File format parsers
- Any code handling untrusted external data

## Tools

### nikic/php-fuzzer (Recommended)

Coverage-guided fuzzer for PHP library/parser testing.

**Installation:**
```bash
composer require --dev nikic/php-fuzzer:^0.0.11
```

**Key features:**
- Coverage-guided mutation (finds new code paths)
- Corpus management (saves interesting inputs)
- Crash detection and reproduction
- Memory limit enforcement

## Creating Fuzz Targets

### Basic Structure

Create fuzz targets in `Tests/Fuzz/` directory:

```php
<?php

declare(strict_types=1);

use MyVendor\MyExtension\Service\MyParser;

require_once dirname(__DIR__, 2) . '/.Build/vendor/autoload.php';

/** @var PhpFuzzer\Config $config */
$parser = new MyParser();

$config->setTarget(function (string $input) use ($parser): void {
    // Call the method being fuzzed
    $parser->parse($input);
});

// Limit input length to prevent memory exhaustion
$config->setMaxLen(65536);
```

### TYPO3 Extension Example

For a TYPO3 extension with HTML parsing (like an RTE image handler):

```php
<?php

declare(strict_types=1);

/**
 * Fuzzing target for ImageAttributeParser.
 *
 * Tests parseImageAttributes() with random/mutated HTML inputs
 * to find crashes, memory exhaustion, or unexpected exceptions.
 */

use MyVendor\MyExtension\Service\ImageAttributeParser;

require_once dirname(__DIR__, 2) . '/.Build/vendor/autoload.php';

/** @var PhpFuzzer\Config $config */
$parser = new ImageAttributeParser();

$config->setTarget(function (string $input) use ($parser): void {
    // Test primary parsing method
    $parser->parseImageAttributes($input);

    // Test related methods with same input
    $parser->parseLinkWithImages($input);
});

$config->setMaxLen(65536);
```

### Testing Classes with Dependencies

For classes requiring TYPO3 dependencies:

```php
<?php

declare(strict_types=1);

use MyVendor\MyExtension\DataHandling\SoftReference\MySoftReferenceParser;
use TYPO3\CMS\Core\Html\HtmlParser;

require_once dirname(__DIR__, 2) . '/.Build/vendor/autoload.php';

/** @var PhpFuzzer\Config $config */

// Create dependencies
$htmlParser = new HtmlParser();
$parser = new MySoftReferenceParser($htmlParser);

// Set required properties via reflection if needed
$reflection    = new ReflectionClass($parser);
$parserKeyProp = $reflection->getProperty('parserKey');
$parserKeyProp->setValue($parser, 'my_parser_key');

$config->setTarget(function (string $input) use ($parser): void {
    $parser->parse(
        'tt_content',
        'bodytext',
        1,
        $input,
    );
});

$config->setMaxLen(65536);
```

## Seed Corpus

Create seed inputs that the fuzzer uses as starting points in `Tests/Fuzz/corpus/`:

```
Tests/Fuzz/
├── ImageAttributeParserTarget.php
├── SoftReferenceParserTarget.php
├── corpus/
│   ├── image-parser/
│   │   ├── basic-img.txt           # <img src="test.jpg" alt="Test" />
│   │   ├── fal-reference.txt       # <img data-htmlarea-file-uid="123" />
│   │   ├── nested-structure.txt    # <a href="#"><img src="x.jpg" /></a>
│   │   └── malformed.txt           # <img src="test
│   └── softref-parser/
│       ├── basic-content.txt
│       └── multiple-images.txt
└── README.md
```

### Good Seed Inputs

Include variety in seeds:
- Valid minimal inputs
- Valid complex inputs
- Edge cases (empty, very long)
- Malformed inputs
- Special characters and encoding edge cases

## Running Fuzz Tests

### Via Composer Scripts

```json
{
    "scripts": {
        "ci:fuzz:image-parser": [
            ".Build/bin/php-fuzzer fuzz Tests/Fuzz/ImageAttributeParserTarget.php Tests/Fuzz/corpus/image-parser --max-runs 10000"
        ],
        "ci:fuzz:softref-parser": [
            ".Build/bin/php-fuzzer fuzz Tests/Fuzz/SoftReferenceParserTarget.php Tests/Fuzz/corpus/softref-parser --max-runs 10000"
        ],
        "ci:fuzz": [
            "@ci:fuzz:image-parser",
            "@ci:fuzz:softref-parser"
        ]
    }
}
```

### Via runTests.sh

```bash
# Add to Build/Scripts/runTests.sh
fuzz)
    FUZZ_TARGET="${1:-Tests/Fuzz/ImageAttributeParserTarget.php}"
    FUZZ_CORPUS="Tests/Fuzz/corpus/image-parser"
    FUZZ_MAX_RUNS="${2:-10000}"

    if [[ "${FUZZ_TARGET}" == *"SoftReference"* ]]; then
        FUZZ_CORPUS="Tests/Fuzz/corpus/softref-parser"
    fi

    COMMAND=(.Build/bin/php-fuzzer fuzz "${FUZZ_TARGET}" "${FUZZ_CORPUS}" --max-runs "${FUZZ_MAX_RUNS}")
    ${CONTAINER_BIN} run ${CONTAINER_COMMON_PARAMS} --name fuzz-${SUFFIX} ${IMAGE_PHP} "${COMMAND[@]}"
    SUITE_EXIT_CODE=$?
    ;;
```

Usage:
```bash
# Default target
Build/Scripts/runTests.sh -s fuzz

# Specific target
Build/Scripts/runTests.sh -s fuzz Tests/Fuzz/ImageAttributeParserTarget.php

# Custom max-runs
Build/Scripts/runTests.sh -s fuzz Tests/Fuzz/ImageAttributeParserTarget.php 50000
```

### Directly

```bash
# Run fuzzer with corpus directory (positional argument, not --corpus option!)
.Build/bin/php-fuzzer fuzz Tests/Fuzz/ImageAttributeParserTarget.php \
    Tests/Fuzz/corpus/image-parser \
    --max-runs 10000
```

## Interpreting Results

### Normal Output

```
Running fuzz test...
  NEW: 0xabc123 - Found new coverage path
  REDUCE: 0xdef456 - Simplified input while maintaining coverage
  ...
Fuzzing complete. 10000 runs, 0 crashes.
```

### Crash Found

```
CRASH: Tests/Fuzz/crashes/crash-abc123.txt
  Error: Call to undefined method...

To reproduce:
  php Tests/Fuzz/ImageAttributeParserTarget.php < Tests/Fuzz/crashes/crash-abc123.txt
```

### What to Look For

| Result | Meaning | Action |
|--------|---------|--------|
| NEW | Found input triggering new code path | Good - corpus expanding |
| REDUCE | Simplified input while keeping coverage | Good - efficient corpus |
| CRASH | Input caused exception/error | **Fix the bug** |
| TIMEOUT | Input caused infinite loop/hang | **Fix the performance issue** |
| OOM | Input caused memory exhaustion | **Fix memory handling** |

## CI Integration

Fuzz testing is typically **not run in CI** due to time requirements. Instead:

1. Run locally before releases
2. Run on schedule (weekly) for security-critical code
3. Run in dedicated security testing pipelines

### Optional CI Integration (Short Runs)

```yaml
# .github/workflows/fuzz.yml
name: Fuzz Testing

on:
  schedule:
    - cron: '0 0 * * 0'  # Weekly on Sunday
  workflow_dispatch:      # Manual trigger

jobs:
  fuzz:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: shivammathur/setup-php@v2
        with:
          php-version: '8.2'
      - run: composer install
      - name: Fuzz ImageAttributeParser
        run: composer ci:fuzz:image-parser
        continue-on-error: true
      - name: Upload crash artifacts
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: fuzz-crashes
          path: Tests/Fuzz/crashes/
```

## Best Practices

1. **Target security-critical code** - Prioritize parsers handling untrusted input
2. **Use meaningful seeds** - Good starting corpus improves coverage
3. **Set memory limits** - Prevent runaway memory usage with `setMaxLen()`
4. **Fix crashes immediately** - Fuzzer-found bugs are often exploitable
5. **Don't ignore OOM** - Memory exhaustion can be a DoS vector
6. **Document findings** - Track what was fuzzed and any issues found

## Directory Structure

```
Tests/
├── Unit/
├── Functional/
├── E2E/
└── Fuzz/
    ├── README.md
    ├── ImageAttributeParserTarget.php
    ├── SoftReferenceParserTarget.php
    ├── corpus/
    │   ├── image-parser/
    │   │   ├── seed1.txt
    │   │   └── seed2.txt
    │   └── softref-parser/
    │       └── seed1.txt
    └── crashes/          # Auto-generated when crashes found
        └── crash-xxx.txt
```

## Resources

- [nikic/php-fuzzer](https://github.com/nikic/PHP-Fuzzer) - PHP coverage-guided fuzzer
- [Google OSS-Fuzz](https://google.github.io/oss-fuzz/) - Continuous fuzzing infrastructure
- [OWASP Fuzzing](https://owasp.org/www-community/Fuzzing) - Security fuzzing concepts
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall