po4yka/RIPDPI / .agents/skills/golden-test-management
golden-test-management
VerifiedGolden-fixture contract discipline: locations, bless workflow, env vars, volatile-field scrubbing, failure-artifact triage. Use when a golden/snapshot test fails or a new golden fixture is added.
Golden Test Management
Golden contracts enforce compatibility for telemetry, diagnostics events, strategy-probe progress/report payloads, and exported data. Fixtures are read-only by default -- tests fail on unexpected diffs. Bless with RIPDPI_BLESS_GOLDENS=1 to update.
Full documentation: docs/testing.md (Golden contracts section).
Fixture Locations
| Layer | Location | Examples |
|---|---|---|
| Rust | native/rust/crates/{crate}/tests/golden/ | android-support, ripdpi-android, ripdpi-tunnel-android, ripdpi-monitor-engine |
| JVM | core/{module}/src/test/resources/golden/ | engine, service, diagnostics |
| Android instrumentation | app/src/androidTest/assets/golden/ | Copies of JVM fixtures for on-device smoke tests |
Environment Variables
| Variable | Purpose | Default |
|---|---|---|
RIPDPI_BLESS_GOLDENS | Set to any value to write fixtures instead of comparing | Not set (read-only) |
RIPDPI_GOLDEN_ARTIFACT_DIR | Override diff artifact output directory | target/golden-diffs (Rust), build/golden-diffs/ (JVM) |
Blessing Workflow
This section is the only bless procedure in the repository; other skills, agents, and docs link here.
Authorization. Run a bless or record command only after the user explicitly authorizes blessing in the current conversation and names the fixture family or path (.claude/rules/golden-bless-discipline.md). A failing golden is not authorization. Use the golden-blesser agent in an isolated worktree, run only the narrow owning command, and inspect the full diff before committing.
| Fixture family | Location | Narrow bless command |
|---|---|---|
| Roborazzi screenshots | app/src/test/screenshots/ | ./gradlew :app:recordRoborazziGithubDebug -Pripdpi.skipNativeBuild=true -Pripdpi.includeRoborazziUnitTests=true --tests '<FQN>'. Pixels match CI only when recorded on ubuntu-latest; commit only the PNGs of tests that fail verify. |
| Rust telemetry/logging | native/rust/crates/{crate}/tests/golden/ | RIPDPI_BLESS_GOLDENS=1 cargo test --locked -p {crate} --manifest-path native/rust/Cargo.toml |
| JVM telemetry/logging | core/{module}/src/test/resources/golden/ | RIPDPI_BLESS_GOLDENS=1 ./gradlew :{module}:testDebugUnitTest --tests "*.{TestClass}" |
| Rust contract_fixtures | native/rust/crates/{crate}/tests/contract_fixtures/ | RIPDPI_BLESS_GOLDENS=1 cargo test --locked --manifest-path native/rust/Cargo.toml -p {crate} |
| Wire contract (shared) | Read through GoldenContractSupport.readSharedFixture() | RIPDPI_BLESS_GOLDENS=1 ./gradlew :core:engine:testDebugUnitTest --tests "*.{TestClass}" |
| Android instrumentation | app/src/androidTest/assets/golden/ | Copied from JVM fixtures by scripts/tests/bless-telemetry-goldens.sh |
Bless All Telemetry/Logging Goldens
bash scripts/tests/bless-telemetry-goldens.sh
This script:
- Blesses Rust goldens (
android-support,ripdpi-android,ripdpi-tunnel-android,ripdpi-monitor-engine) - Blesses JVM goldens (
NativeTelemetryGoldenTest,ServiceTelemetryGoldenTest) - Syncs instrumentation fixtures (copies JVM fixtures to
app/src/androidTest/assets/golden/)
Bless Specific Crate/Module
# Rust
RIPDPI_BLESS_GOLDENS=1 cargo test --locked -p ripdpi-android --manifest-path native/rust/Cargo.toml
# Kotlin
RIPDPI_BLESS_GOLDENS=1 ./gradlew :core:engine:testDebugUnitTest --tests "*.NativeTelemetryGoldenTest"
After Blessing
- Review diffs:
git diff-- verify changes are intentional - Commit with explanation of why the golden changed
- If instrumentation fixtures were affected, verify the sync step ran
Failure Artifacts
On mismatch, both Rust and JVM support libraries write three files:
| File | Content |
|---|---|
{name}.expected | The golden fixture content |
{name}.actual | What the test produced |
{name}.diff | Unified diff between expected and actual |
Rust: Written to target/golden-diffs/ (or RIPDPI_GOLDEN_ARTIFACT_DIR).
JVM: Written to {module}/build/golden-diffs/.
Support Libraries
Rust: golden-test-support Crate
use golden_test_support::{assert_text_golden, canonicalize_json, canonicalize_json_with};
// Simple text comparison
assert_text_golden(env!("CARGO_MANIFEST_DIR"), "tests/golden/output.json", &actual);
// JSON with key sorting
let canonical = canonicalize_json(&json_string)?;
assert_text_golden(env!("CARGO_MANIFEST_DIR"), "tests/golden/data.json", &canonical);
// JSON with custom scrubbing
let canonical = canonicalize_json_with(&json_string, |value| {
// Remove volatile fields like timestamps
scrub_timestamps(value);
})?;
assert_text_golden(env!("CARGO_MANIFEST_DIR"), "tests/golden/data.json", &canonical);
Kotlin: GoldenContractSupport Object
Located in core/engine/src/test/kotlin/com/poyka/ripdpi/core/GoldenContractSupport.kt:
// JSON comparison with canonical key ordering
GoldenContractSupport.assertJsonGolden(
"snapshot.json",
json.encodeToString(serializer, data),
)
// JSON with custom scrubbing
GoldenContractSupport.assertJsonGolden(
"snapshot.json",
json.encodeToString(serializer, data),
::scrubVolatileFields, // (JsonElement) -> JsonElement
)
// Plain text comparison
GoldenContractSupport.assertTextGolden("output.txt", actualText)
Scrubbing Volatile Fields
Fields that change between runs must be scrubbed for deterministic comparison:
Scrubbed (non-deterministic):
- Timestamps (
serviceStartedAt,lastFailureAt,updatedAt,capturedAt,createdAt) - Generated session IDs
- Loopback ports
- Absolute temp paths
- Archive-time dynamic file names
Strict (must match exactly):
- State and health values
- Counters
- Event order
- Log level and message text
- Route group and target metadata
- Strategy signatures and recommendations
- Per-lane TCP/QUIC/DNS metadata
- Strategy-probe progress lane/candidate metadata
- Audit assessment and target-selection metadata
- Resolver metadata and fallback state
Kotlin Scrubbing Pattern
private fun scrubVolatileFields(value: JsonElement): JsonElement =
when (value) {
is JsonObject -> JsonObject(
value.mapValues { (key, element) ->
when (key) {
"serviceStartedAt", "lastFailureAt", "updatedAt",
"capturedAt", "createdAt" -> Json.parseToJsonElement("0")
else -> scrubVolatileFields(element)
}
},
)
else -> value
}
Adding a New Golden Test
Rust
-
Add
golden-test-supportto[dev-dependencies]in the crate'sCargo.toml:[dev-dependencies] golden-test-support = { path = "../golden-test-support" } -
Write the test:
#[test] fn my_output_matches_golden() { let actual = produce_output(); let canonical = canonicalize_json(&serde_json::to_string(&actual).unwrap()).unwrap(); assert_text_golden(env!("CARGO_MANIFEST_DIR"), "tests/golden/my_output.json", &canonical); } -
Create initial fixture:
RIPDPI_BLESS_GOLDENS=1 cargo test --locked -p my-crate my_output_matches_golden -
Review and commit the new fixture file.
Kotlin
-
Write the test using
GoldenContractSupport:@Test fun myOutputMatchesGolden() { val actual = produceOutput() GoldenContractSupport.assertJsonGolden("my_output.json", jsonEncode(actual)) } -
Create initial fixture:
RIPDPI_BLESS_GOLDENS=1 ./gradlew :module:testDebugUnitTest --tests "*.MyGoldenTest" -
Review and commit the new fixture file.
Instrumentation Sync
If the new golden is needed for Android instrumentation tests:
- Add a
cpline toscripts/tests/bless-telemetry-goldens.sh - Copy the JVM fixture to
app/src/androidTest/assets/golden/
Common Mistakes
| Mistake | Fix |
|---|---|
| Blessing without reviewing diffs | Always git diff after blessing. Unexpected changes may indicate bugs. |
| Forgetting to scrub volatile fields | New timestamp/ID fields cause non-deterministic failures. Add to scrub function. |
| Not explaining golden changes in commit | Golden updates should always explain why the expected output changed. |
| Missing instrumentation sync | JVM fixture updated but app/src/androidTest/assets/golden/ not. Run bless script or add cp manually. |
Adding golden without canonicalize_json | JSON key order is non-deterministic. Always canonicalize before comparing. |
| Hardcoding fixture path | Rust: use env!("CARGO_MANIFEST_DIR"). Kotlin: GoldenContractSupport resolves repo root automatically. |
See Also
docs/testing.md-- Full test stack documentation including golden contracts sectionnative/rust/crates/golden-test-support/src/lib.rs-- Rust support library sourcecore/engine/src/test/kotlin/.../GoldenContractSupport.kt-- Kotlin support library sourcescripts/tests/bless-telemetry-goldens.sh-- Blessing script
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |