Skip to content

po4yka/RIPDPI / .agents/skills/golden-test-management

golden-test-management

Verified

Golden-fixture contract discipline: locations, bless workflow, env vars, volatile-field scrubbing, failure-artifact triage. Use when a golden/snapshot test fails or a new golden fixture is added.

Golden Test Management

Golden contracts enforce compatibility for telemetry, diagnostics events, strategy-probe progress/report payloads, and exported data. Fixtures are read-only by default -- tests fail on unexpected diffs. Bless with RIPDPI_BLESS_GOLDENS=1 to update.

Full documentation: docs/testing.md (Golden contracts section).

Fixture Locations

LayerLocationExamples
Rustnative/rust/crates/{crate}/tests/golden/android-support, ripdpi-android, ripdpi-tunnel-android, ripdpi-monitor-engine
JVMcore/{module}/src/test/resources/golden/engine, service, diagnostics
Android instrumentationapp/src/androidTest/assets/golden/Copies of JVM fixtures for on-device smoke tests

Environment Variables

VariablePurposeDefault
RIPDPI_BLESS_GOLDENSSet to any value to write fixtures instead of comparingNot set (read-only)
RIPDPI_GOLDEN_ARTIFACT_DIROverride diff artifact output directorytarget/golden-diffs (Rust), build/golden-diffs/ (JVM)

Blessing Workflow

This section is the only bless procedure in the repository; other skills, agents, and docs link here.

Authorization. Run a bless or record command only after the user explicitly authorizes blessing in the current conversation and names the fixture family or path (.claude/rules/golden-bless-discipline.md). A failing golden is not authorization. Use the golden-blesser agent in an isolated worktree, run only the narrow owning command, and inspect the full diff before committing.

Fixture familyLocationNarrow bless command
Roborazzi screenshotsapp/src/test/screenshots/./gradlew :app:recordRoborazziGithubDebug -Pripdpi.skipNativeBuild=true -Pripdpi.includeRoborazziUnitTests=true --tests '<FQN>'. Pixels match CI only when recorded on ubuntu-latest; commit only the PNGs of tests that fail verify.
Rust telemetry/loggingnative/rust/crates/{crate}/tests/golden/RIPDPI_BLESS_GOLDENS=1 cargo test --locked -p {crate} --manifest-path native/rust/Cargo.toml
JVM telemetry/loggingcore/{module}/src/test/resources/golden/RIPDPI_BLESS_GOLDENS=1 ./gradlew :{module}:testDebugUnitTest --tests "*.{TestClass}"
Rust contract_fixturesnative/rust/crates/{crate}/tests/contract_fixtures/RIPDPI_BLESS_GOLDENS=1 cargo test --locked --manifest-path native/rust/Cargo.toml -p {crate}
Wire contract (shared)Read through GoldenContractSupport.readSharedFixture()RIPDPI_BLESS_GOLDENS=1 ./gradlew :core:engine:testDebugUnitTest --tests "*.{TestClass}"
Android instrumentationapp/src/androidTest/assets/golden/Copied from JVM fixtures by scripts/tests/bless-telemetry-goldens.sh

Bless All Telemetry/Logging Goldens

bash scripts/tests/bless-telemetry-goldens.sh

This script:

  1. Blesses Rust goldens (android-support, ripdpi-android, ripdpi-tunnel-android, ripdpi-monitor-engine)
  2. Blesses JVM goldens (NativeTelemetryGoldenTest, ServiceTelemetryGoldenTest)
  3. Syncs instrumentation fixtures (copies JVM fixtures to app/src/androidTest/assets/golden/)

Bless Specific Crate/Module

# Rust
RIPDPI_BLESS_GOLDENS=1 cargo test --locked -p ripdpi-android --manifest-path native/rust/Cargo.toml

# Kotlin
RIPDPI_BLESS_GOLDENS=1 ./gradlew :core:engine:testDebugUnitTest --tests "*.NativeTelemetryGoldenTest"

After Blessing

  1. Review diffs: git diff -- verify changes are intentional
  2. Commit with explanation of why the golden changed
  3. If instrumentation fixtures were affected, verify the sync step ran

Failure Artifacts

On mismatch, both Rust and JVM support libraries write three files:

FileContent
{name}.expectedThe golden fixture content
{name}.actualWhat the test produced
{name}.diffUnified diff between expected and actual

Rust: Written to target/golden-diffs/ (or RIPDPI_GOLDEN_ARTIFACT_DIR). JVM: Written to {module}/build/golden-diffs/.

Support Libraries

Rust: golden-test-support Crate

use golden_test_support::{assert_text_golden, canonicalize_json, canonicalize_json_with};

// Simple text comparison
assert_text_golden(env!("CARGO_MANIFEST_DIR"), "tests/golden/output.json", &actual);

// JSON with key sorting
let canonical = canonicalize_json(&json_string)?;
assert_text_golden(env!("CARGO_MANIFEST_DIR"), "tests/golden/data.json", &canonical);

// JSON with custom scrubbing
let canonical = canonicalize_json_with(&json_string, |value| {
    // Remove volatile fields like timestamps
    scrub_timestamps(value);
})?;
assert_text_golden(env!("CARGO_MANIFEST_DIR"), "tests/golden/data.json", &canonical);

Kotlin: GoldenContractSupport Object

Located in core/engine/src/test/kotlin/com/poyka/ripdpi/core/GoldenContractSupport.kt:

// JSON comparison with canonical key ordering
GoldenContractSupport.assertJsonGolden(
    "snapshot.json",
    json.encodeToString(serializer, data),
)

// JSON with custom scrubbing
GoldenContractSupport.assertJsonGolden(
    "snapshot.json",
    json.encodeToString(serializer, data),
    ::scrubVolatileFields,  // (JsonElement) -> JsonElement
)

// Plain text comparison
GoldenContractSupport.assertTextGolden("output.txt", actualText)

Scrubbing Volatile Fields

Fields that change between runs must be scrubbed for deterministic comparison:

Scrubbed (non-deterministic):

  • Timestamps (serviceStartedAt, lastFailureAt, updatedAt, capturedAt, createdAt)
  • Generated session IDs
  • Loopback ports
  • Absolute temp paths
  • Archive-time dynamic file names

Strict (must match exactly):

  • State and health values
  • Counters
  • Event order
  • Log level and message text
  • Route group and target metadata
  • Strategy signatures and recommendations
  • Per-lane TCP/QUIC/DNS metadata
  • Strategy-probe progress lane/candidate metadata
  • Audit assessment and target-selection metadata
  • Resolver metadata and fallback state

Kotlin Scrubbing Pattern

private fun scrubVolatileFields(value: JsonElement): JsonElement =
    when (value) {
        is JsonObject -> JsonObject(
            value.mapValues { (key, element) ->
                when (key) {
                    "serviceStartedAt", "lastFailureAt", "updatedAt",
                    "capturedAt", "createdAt" -> Json.parseToJsonElement("0")
                    else -> scrubVolatileFields(element)
                }
            },
        )
        else -> value
    }

Adding a New Golden Test

Rust

  1. Add golden-test-support to [dev-dependencies] in the crate's Cargo.toml:

    [dev-dependencies]
    golden-test-support = { path = "../golden-test-support" }
    
  2. Write the test:

    #[test]
    fn my_output_matches_golden() {
        let actual = produce_output();
        let canonical = canonicalize_json(&serde_json::to_string(&actual).unwrap()).unwrap();
        assert_text_golden(env!("CARGO_MANIFEST_DIR"), "tests/golden/my_output.json", &canonical);
    }
    
  3. Create initial fixture: RIPDPI_BLESS_GOLDENS=1 cargo test --locked -p my-crate my_output_matches_golden

  4. Review and commit the new fixture file.

Kotlin

  1. Write the test using GoldenContractSupport:

    @Test
    fun myOutputMatchesGolden() {
        val actual = produceOutput()
        GoldenContractSupport.assertJsonGolden("my_output.json", jsonEncode(actual))
    }
    
  2. Create initial fixture: RIPDPI_BLESS_GOLDENS=1 ./gradlew :module:testDebugUnitTest --tests "*.MyGoldenTest"

  3. Review and commit the new fixture file.

Instrumentation Sync

If the new golden is needed for Android instrumentation tests:

  1. Add a cp line to scripts/tests/bless-telemetry-goldens.sh
  2. Copy the JVM fixture to app/src/androidTest/assets/golden/

Common Mistakes

MistakeFix
Blessing without reviewing diffsAlways git diff after blessing. Unexpected changes may indicate bugs.
Forgetting to scrub volatile fieldsNew timestamp/ID fields cause non-deterministic failures. Add to scrub function.
Not explaining golden changes in commitGolden updates should always explain why the expected output changed.
Missing instrumentation syncJVM fixture updated but app/src/androidTest/assets/golden/ not. Run bless script or add cp manually.
Adding golden without canonicalize_jsonJSON key order is non-deterministic. Always canonicalize before comparing.
Hardcoding fixture pathRust: use env!("CARGO_MANIFEST_DIR"). Kotlin: GoldenContractSupport resolves repo root automatically.

See Also

  • docs/testing.md -- Full test stack documentation including golden contracts section
  • native/rust/crates/golden-test-support/src/lib.rs -- Rust support library source
  • core/engine/src/test/kotlin/.../GoldenContractSupport.kt -- Kotlin support library source
  • scripts/tests/bless-telemetry-goldens.sh -- Blessing script
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall