Skip to content

netresearch/typo3-testing-skill / skills/typo3-testing

typo3-testing

Flagged

Use when reproducing defects as failing tests, testing template or rendered output changes, confirming changes on a running TYPO3 instance, or setting up TYPO3 extension tests: unit/functional/E2E, PHPUnit 11-13, mutation testing, final-class mocks (v14), CI matrix across TYPO3 12/13/14.3 LTS, typo3-ci-workflows meta-package, CI failures. Also: transaction rollback, worker crashes, competing requests, testing-framework setup, test matrix, integration/e2e testing, coverage, test generation.

FileKindSize
SKILL.mdskill md6.8 KB
assets/.php-cs-fixer.dist.phpscript7.0 KB
assets/AGENTS.mdmarkdown5.0 KB
assets/Build/Scripts/runTests.shexecutablescript27.5 KB
assets/Build/playwright/.nvmrcother6 B
assets/Build/playwright/package.jsonmanifest535 B
assets/Build/playwright/playwright.config.tsscript1.3 KB
assets/Build/playwright/tests/playwright/accessibility/modules.spec.tsscript2.0 KB
assets/Build/playwright/tests/playwright/config.tsscript712 B
assets/Build/playwright/tests/playwright/e2e/backend-module.spec.tsscript1.9 KB
assets/Build/playwright/tests/playwright/fixtures/setup-fixtures.tsscript2.7 KB
assets/Build/playwright/tests/playwright/helper/login.setup.tsscript975 B
assets/FunctionalTests.xmlconfig1.4 KB
assets/FunctionalTestsBootstrap.phpscript1.6 KB
assets/Makefileother2.3 KB
assets/UnitTests.xmlconfig890 B
assets/UnitTestsBootstrap.phpscript2.1 KB
assets/bootstrap.phpscript1.4 KB
assets/codecov.ymlconfig1.6 KB
assets/docker/codeception.ymlconfig979 B
assets/docker/docker-compose.ymlconfig1.4 KB
assets/example-tests/ExampleAcceptanceCest.phpscript2.2 KB
assets/example-tests/ExampleFunctionalTest.phpscript2.4 KB
assets/example-tests/ExampleUnitTest.phpscript1.6 KB
assets/fixtures/README.mdmarkdown2.0 KB
assets/fixtures/be_users.csvother361 B
assets/fixtures/pages.csvother257 B
assets/fixtures/sys_category.csvother228 B
assets/fixtures/tt_content.csvother338 B
assets/github-actions-e2e.ymlconfig8.8 KB
assets/github-actions-tests.ymlconfig6.3 KB
assets/infection.json5other2.8 KB
assets/phpat.neonother330 B
assets/phpat.phpscript8.4 KB
assets/phpstan-baseline.neonother365 B
assets/phpstan.neonother1.4 KB
assets/rector.phpscript2.9 KB
checkpoints.yamlconfig61.7 KB
references/accessibility-testing.mdmarkdown7.6 KB
references/architecture-testing.mdmarkdown4.4 KB
references/asset-templates-guide.mdmarkdown3.4 KB
references/backend-module-render-verification.mdmarkdown9.7 KB
references/backend-user-access-testing.mdmarkdown9.5 KB
references/captainhook-setup.mdmarkdown2.4 KB
references/ci-cd.mdmarkdown39.5 KB
references/ci-debugging.mdmarkdown20.0 KB
references/ci-workflows-meta-package.mdmarkdown11.6 KB
references/crash-and-concurrency-testing.mdmarkdown4.8 KB
references/crypto-testing.mdmarkdown16.6 KB
references/datahandler-silent-rewrites.mdmarkdown7.2 KB
references/ddev-testing.mdmarkdown1.3 KB
references/e2e-testing.mdmarkdown53.0 KB
references/enforcement-rules.mdmarkdown5.7 KB
references/event-dispatch-testing.mdmarkdown2.8 KB
references/framework-compat-gate.mdmarkdown4.9 KB
references/functional-test-patterns.mdmarkdown11.0 KB
references/functional-testing.mdmarkdown46.3 KB
references/fuzz-testing.mdmarkdown8.6 KB
references/integration-testing.mdmarkdown14.1 KB
references/javascript-testing.mdmarkdown29.3 KB
references/live-instance-verification.mdmarkdown8.6 KB
references/mock-validity.mdmarkdown15.7 KB
references/mutation-testing.mdmarkdown36.0 KB
references/performance-testing.mdmarkdown11.2 KB
references/quality-tools.mdmarkdown25.5 KB
references/release-workflow-validation.mdmarkdown3.6 KB
references/sonarcloud.mdmarkdown18.8 KB
references/synthetic-secret-fixtures.mdmarkdown3.8 KB
references/tdd-discipline.mdmarkdown9.1 KB
references/test-environment-guards.mdmarkdown18.5 KB
references/test-runners.mdmarkdown24.6 KB
references/typo3-ci-config-patterns.mdmarkdown9.0 KB
references/typo3-v14-final-classes.mdmarkdown16.5 KB
references/unit-testing.mdmarkdown59.1 KB
scripts/generate-test.shexecutablescript6.0 KB
scripts/setup-testing.shexecutablescript6.3 KB
scripts/validate-setup.shexecutablescript4.0 KB

references/crypto-testing.md

View on GitHub
<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
<!-- SPDX-FileCopyrightText: Netresearch DTT GmbH -->

# Cryptographic Testing Patterns

Testing cryptographic code requires specific patterns to ensure security while maintaining testability.

## When to Apply

- Secrets management extensions
- Envelope encryption implementations
- Key derivation functions
- Token/credential storage
- Memory-safe secret handling

## Unit Testing Cryptographic Services

### Testing Encryption Services

```php
<?php

declare(strict_types=1);

namespace Vendor\Extension\Tests\Unit\Service;

use PHPUnit\Framework\Attributes\Test;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Service\EncryptionService;

final class EncryptionServiceTest extends UnitTestCase
{
    private EncryptionService $subject;
    private string $testKey;

    protected function setUp(): void
    {
        parent::setUp();
        // Use deterministic test key - NEVER use production keys
        $this->testKey = sodium_crypto_secretbox_keygen();
        $this->subject = new EncryptionService($this->testKey);
    }

    protected function tearDown(): void
    {
        // Clear sensitive test data from memory
        sodium_memzero($this->testKey);
        parent::tearDown();
    }

    #[Test]
    public function encryptAndDecryptRoundTrip(): void
    {
        $plaintext = 'sensitive-api-key-12345';

        $encrypted = $this->subject->encrypt($plaintext);
        $decrypted = $this->subject->decrypt($encrypted);

        self::assertSame($plaintext, $decrypted);
        self::assertNotSame($plaintext, $encrypted);
    }

    #[Test]
    public function encryptProducesDifferentCiphertextForSamePlaintext(): void
    {
        $plaintext = 'secret-value';

        $encrypted1 = $this->subject->encrypt($plaintext);
        $encrypted2 = $this->subject->encrypt($plaintext);

        // Random nonce ensures different ciphertext each time
        self::assertNotSame($encrypted1, $encrypted2);
    }

    #[Test]
    public function decryptWithWrongKeyThrowsException(): void
    {
        $encrypted = $this->subject->encrypt('secret');
        $wrongKey = sodium_crypto_secretbox_keygen();
        $wrongService = new EncryptionService($wrongKey);

        $this->expectException(\RuntimeException::class);
        $this->expectExceptionMessage('Decryption failed');

        $wrongService->decrypt($encrypted);

        sodium_memzero($wrongKey);
    }
}
```

### Testing Envelope Encryption (DEK + KEK Pattern)

Envelope encryption uses a Data Encryption Key (DEK) encrypted by a Key Encryption Key (KEK):

```php
<?php

declare(strict_types=1);

namespace Vendor\Extension\Tests\Unit\Service;

use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\MockObject\MockObject;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Service\EnvelopeEncryptionService;
use Vendor\Extension\Service\KeyManagementServiceInterface;

final class EnvelopeEncryptionServiceTest extends UnitTestCase
{
    private EnvelopeEncryptionService $subject;
    private KeyManagementServiceInterface&MockObject $keyManagementService;
    private string $testKek;

    protected function setUp(): void
    {
        parent::setUp();
        $this->testKek = sodium_crypto_secretbox_keygen();

        $this->keyManagementService = $this->createMock(KeyManagementServiceInterface::class);
        $this->keyManagementService
            ->method('getKeyEncryptionKey')
            ->willReturn($this->testKek);

        $this->subject = new EnvelopeEncryptionService($this->keyManagementService);
    }

    protected function tearDown(): void
    {
        sodium_memzero($this->testKek);
        parent::tearDown();
    }

    #[Test]
    public function storeGeneratesUniqueDekPerSecret(): void
    {
        $result1 = $this->subject->store('secret1');
        $result2 = $this->subject->store('secret2');

        // Each secret gets its own DEK
        self::assertNotSame($result1['encrypted_dek'], $result2['encrypted_dek']);
    }

    #[Test]
    public function retrieveDecryptsWithCorrectDek(): void
    {
        $original = 'my-api-secret';
        $stored = $this->subject->store($original);

        $retrieved = $this->subject->retrieve(
            $stored['encrypted_value'],
            $stored['encrypted_dek'],
            $stored['nonce']
        );

        self::assertSame($original, $retrieved);
    }

    #[Test]
    public function keyRotationReEncryptsWithNewKek(): void
    {
        $original = 'secret-to-rotate';
        $stored = $this->subject->store($original);

        $newKek = sodium_crypto_secretbox_keygen();
        $rotated = $this->subject->rotateKey($stored, $this->testKek, $newKek);

        // Encrypted DEK changes, but value remains accessible
        self::assertNotSame($stored['encrypted_dek'], $rotated['encrypted_dek']);

        sodium_memzero($newKek);
    }
}
```

## Testing Memory-Safe Secret Handling

### Verifying sodium_memzero() Usage

For security-critical code, verify that secrets are cleared from memory:

```php
<?php

declare(strict_types=1);

namespace Vendor\Extension\Tests\Unit\Http;

use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\MockObject\MockObject;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Http\VaultHttpClient;
use Vendor\Extension\Service\VaultServiceInterface;
use Psr\Http\Client\ClientInterface;

final class VaultHttpClientTest extends UnitTestCase
{
    private VaultHttpClient $subject;
    private VaultServiceInterface&MockObject $vaultService;
    private ClientInterface&MockObject $httpClient;

    protected function setUp(): void
    {
        parent::setUp();
        $this->vaultService = $this->createMock(VaultServiceInterface::class);
        $this->httpClient = $this->createMock(ClientInterface::class);
        $this->subject = new VaultHttpClient($this->vaultService, $this->httpClient);
    }

    #[Test]
    public function secretIsRetrievedJustInTime(): void
    {
        // Verify secret is retrieved only when needed
        $this->vaultService
            ->expects(self::once())
            ->method('retrieve')
            ->with('api-key-identifier')
            ->willReturn('secret-value');

        $this->httpClient
            ->expects(self::once())
            ->method('sendRequest');

        $this->subject->request('GET', 'https://api.example.com', [
            'auth_secret' => 'api-key-identifier',
        ]);
    }

    #[Test]
    public function secretNotRetrievedWhenNotNeeded(): void
    {
        // Verify no vault access for requests without auth
        $this->vaultService
            ->expects(self::never())
            ->method('retrieve');

        $this->subject->request('GET', 'https://api.example.com');
    }
}
```

### Testing the Secret Clearing Pattern

While directly testing `sodium_memzero()` is difficult (the memory is zeroed), test the pattern:

```php
#[Test]
public function requestClearsSecretEvenOnException(): void
{
    $this->vaultService
        ->method('retrieve')
        ->willReturn('secret-value');

    $this->httpClient
        ->method('sendRequest')
        ->willThrowException(new \RuntimeException('Network error'));

    // The implementation should use try/finally to ensure cleanup
    try {
        $this->subject->request('GET', 'https://api.example.com', [
            'auth_secret' => 'test-key',
        ]);
    } catch (\RuntimeException) {
        // Expected - secret should still be cleared in finally block
    }

    // If we got here without memory issues, the pattern is correct
    self::assertTrue(true);
}
```

## Test Data Patterns

### Deterministic Test Keys

```php
final class CryptoTestHelper
{
    /**
     * Generate a deterministic test key for reproducible tests.
     * NEVER use in production - only for testing.
     */
    public static function createTestKey(string $seed = 'test'): string
    {
        return sodium_crypto_generichash($seed, '', SODIUM_CRYPTO_SECRETBOX_KEYBYTES);
    }

    /**
     * Create a test secret with cleanup callback.
     * @return array{secret: string, cleanup: callable}
     */
    public static function createTestSecret(string $value): array
    {
        $secret = $value;
        return [
            'secret' => $secret,
            'cleanup' => static function () use (&$secret): void {
                if ($secret !== '') {
                    sodium_memzero($secret);
                }
            },
        ];
    }
}
```

### Using Test Helpers

```php
protected function setUp(): void
{
    parent::setUp();
    $this->testData = CryptoTestHelper::createTestSecret('api-key-123');
}

protected function tearDown(): void
{
    ($this->testData['cleanup'])();
    parent::tearDown();
}
```

## Functional Testing Encrypted Storage

For database-backed secret storage:

```php
<?php

declare(strict_types=1);

namespace Vendor\Extension\Tests\Functional\Repository;

use PHPUnit\Framework\Attributes\Test;
use TYPO3\TestingFramework\Core\Functional\FunctionalTestCase;
use Vendor\Extension\Repository\SecretRepository;

final class SecretRepositoryTest extends FunctionalTestCase
{
    protected array $testExtensionsToLoad = ['vendor/extension'];

    private SecretRepository $subject;
    private string $testKey;

    protected function setUp(): void
    {
        parent::setUp();
        $this->testKey = sodium_crypto_secretbox_keygen();
        $this->subject = $this->get(SecretRepository::class);
    }

    protected function tearDown(): void
    {
        sodium_memzero($this->testKey);
        parent::tearDown();
    }

    #[Test]
    public function storedSecretIsEncryptedInDatabase(): void
    {
        $identifier = 'test-api-key';
        $plaintext = 'super-secret-value';

        $this->subject->store($identifier, $plaintext, $this->testKey);

        // Direct database query to verify encryption
        $row = $this->getConnectionPool()
            ->getConnectionForTable('tx_extension_secret')
            ->select(['*'], 'tx_extension_secret', ['identifier' => $identifier])
            ->fetchAssociative();

        // Value in database should NOT match plaintext
        self::assertNotSame($plaintext, $row['encrypted_value']);
        self::assertNotEmpty($row['encrypted_dek']);
        self::assertNotEmpty($row['nonce']);
    }

    #[Test]
    public function retrieveReturnsDecryptedValue(): void
    {
        $identifier = 'test-secret';
        $plaintext = 'my-secret-123';

        $this->subject->store($identifier, $plaintext, $this->testKey);
        $retrieved = $this->subject->retrieve($identifier, $this->testKey);

        self::assertSame($plaintext, $retrieved);
    }
}
```

## Algorithm-Specific Nonce Lengths

Different algorithms require different nonce lengths. Using wrong nonce length reduces security.

### The Problem

```php
// ❌ WRONG - Same nonce length for all algorithms reduces entropy
private const NONCE_LENGTH = 12;  // AES-GCM length

public function encrypt(string $plaintext): string
{
    $nonce = random_bytes(self::NONCE_LENGTH);  // Always 12 bytes!

    // For XChaCha20-Poly1305, this wastes 12 bytes of nonce entropy
    // (should be 24 bytes)
    if ($this->algorithm === 'xchacha20') {
        $nonce = str_pad($nonce, 24, "\0");  // Padding with zeros = BAD!
    }
}
```

### The Fix - Dynamic Nonce Length

```php
private function getNonceLength(): int
{
    return match ($this->algorithm) {
        'aes-256-gcm' => SODIUM_CRYPTO_AEAD_AES256GCM_NPUBBYTES,        // 12 bytes
        'xchacha20-poly1305' => SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES,  // 24 bytes
        default => throw new \InvalidArgumentException('Unknown algorithm'),
    };
}

public function encrypt(string $plaintext): string
{
    // ✅ CORRECT - Full entropy for each algorithm
    $nonce = random_bytes($this->getNonceLength());
}
```

### Testing Nonce Length

```php
#[Test]
public function aesGcmUsesCorrectNonceLength(): void
{
    $service = new EncryptionService('aes-256-gcm', $this->key);
    $encrypted = $service->encrypt('test');

    // Extract nonce from ciphertext
    $nonce = substr($encrypted, 0, SODIUM_CRYPTO_AEAD_AES256GCM_NPUBBYTES);

    self::assertSame(12, strlen($nonce));
}

#[Test]
public function xchachaUsesCorrectNonceLength(): void
{
    $service = new EncryptionService('xchacha20-poly1305', $this->key);
    $encrypted = $service->encrypt('test');

    // Extract nonce from ciphertext
    $nonce = substr($encrypted, 0, SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES);

    self::assertSame(24, strlen($nonce));
}

#[Test]
public function noncesAreFullyRandom(): void
{
    // Verify nonce doesn't contain padding zeros
    $service = new EncryptionService('xchacha20-poly1305', $this->key);

    $nonces = [];
    for ($i = 0; $i < 10; $i++) {
        $encrypted = $service->encrypt('test');
        $nonce = substr($encrypted, 0, 24);
        $nonces[] = $nonce;

        // No nonce should end with 12 zero bytes (padding pattern)
        $lastBytes = substr($nonce, 12);
        self::assertNotSame(str_repeat("\0", 12), $lastBytes);
    }

    // All nonces should be unique
    self::assertSame(10, count(array_unique($nonces)));
}
```

## Security Test Checklist

| Test Case | Purpose |
|-----------|---------|
| Round-trip encrypt/decrypt | Basic correctness |
| Different ciphertext for same input | Nonce randomness |
| Correct nonce length per algorithm | Algorithm compliance |
| Wrong key fails decryption | Key isolation |
| Tampered ciphertext fails | Integrity protection |
| Empty input handling | Edge case security |
| Key rotation preserves access | Migration safety |
| Secret cleared after use | Memory safety |
| No plaintext in logs | Audit safety |

## Anti-Patterns to Avoid

### Never Log Secrets

```php
// WRONG - logs actual secret
$this->logger->debug('Retrieved secret: ' . $secret);

// CORRECT - log only identifier
$this->logger->debug('Retrieved secret', ['identifier' => $identifier]);
```

### Never Use Weak Keys in Tests

```php
// WRONG - predictable key
$key = str_repeat('0', 32);

// CORRECT - proper key generation
$key = sodium_crypto_secretbox_keygen();
```

### Never Skip Cleanup in Tests

```php
// WRONG - secret remains in memory
protected function tearDown(): void
{
    parent::tearDown();
}

// CORRECT - explicit cleanup
protected function tearDown(): void
{
    if (isset($this->testKey)) {
        sodium_memzero($this->testKey);
    }
    parent::tearDown();
}
```

## CI Integration

For security-critical extensions, run crypto tests in isolation:

```yaml
# .github/workflows/test.yml
jobs:
  crypto-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: shivammathur/setup-php@v2
        with:
          php-version: '8.2'
          extensions: sodium
      - run: composer install
      - name: Run crypto-specific tests
        run: |
          vendor/bin/phpunit --testsuite Unit \
            --filter 'Encryption|Crypto|Secret|Vault'
```

## Randomized Property Tests — Deterministic Seeding

Redaction, sanitizer, and streaming-window code is best proven with a randomized
property test (generate many raw inputs, assert an invariant such as
`concat(redactedChunks) === redact(fullRaw)`). Such a test must be **reproducible**
on failure — but do not reach for `srand()` + `mt_rand()`:

- **`mt_rand()` gets rewritten by CGL.** The coding-standards fixer (php-cs-fixer
  `random_api_migration`) rewrites `mt_rand()` → `random_int()`. `random_int()` is
  a CSPRNG and **ignores `srand()`**, so a seed you set has no effect after the
  fixer runs. The test still passes locally (before the fixer) and becomes silently
  non-deterministic in CI (after the fixer) — a flake you cannot reproduce.

Use a hand-rolled deterministic generator in the test instead — it survives the
fixer and gives a fixed sequence per seed. Prefer a hash-based generator: it is
100% portable (no integer-overflow / float-cast platform dependence a raw LCG has
on 32-bit PHP) and just as simple:

```php
$seed = 'fixed-seed';
$counter = 0;
$roll = static function () use (&$counter, $seed): int {
    // 7 hex chars = 28 bits, always fits a signed int on 32- and 64-bit PHP
    return (int) hexdec(substr(hash('sha256', $seed . $counter++), 0, 7));
};

for ($i = 0; $i < 1000; $i++) {
    $raw = $this->randomPayload($roll);        // build input from $roll(), not mt_rand()
    self::assertSame(redact($raw), implode('', $this->streamThrough($raw)));
}
```

**Rule:** never depend on `srand()`/`mt_rand()` determinism in a test when CGL runs
in the gate. A deterministic hash-based generator (or a fixed data provider) is the
portable choice.

## Resources

- [libsodium Documentation](https://doc.libsodium.org/)
- [PHP Sodium Functions](https://www.php.net/manual/en/book.sodium.php)
- [OWASP Cryptographic Storage](https://cheatsheetseries.owasp.org/cheatsheets/Cryptographic_Storage_Cheat_Sheet.html)
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall