meirborowski/Lodestone / .claude/skills/update-reference-images
update-reference-images
VerifiedAdd a reference-image test to Lodestone, or deliberately update reference images after an intended rendering change. Use when a rendering feature needs image coverage, or a reference-image test fails because rendering was meant to change.
Add or Update Reference Images
Reference images are the expected output of rendering tests, compared on lavapipe at the Mesa version pinned in tools/lavapipe.env (see docs/Testing.md#reference-images and docs/Decisions/0009-lavapipe.md). They live in Tests/ReferenceImages and are stored with Git LFS.
Never regenerate a reference image to make a failing test pass. Update one only when rendering is meant to change, and only after confirming the new image is right. If a test fails and the change wasn't meant to alter rendering, the code is wrong, not the image.
Prerequisites
lavapipe must be available, or reference-image tests are skipped: on Windows CMake downloads it; on Linux build it with tools/build-lavapipe.sh and reconfigure. macOS can't create or check reference images - use Windows or Linux.
Add a reference-image test
- Write the test in
Tests/Render(inLodestoneRenderTests), in theReferenceImagestest suite so it's left out where lavapipe isn't available:
Use small, fixed image sizes (256x256 is plenty for most features), and a fixed camera, time and random seed, so the image is deterministic.TEST_CASE("Shadows soften with distance" * doctest::test_suite("ReferenceImages")) { // Render offscreen into a texture, then: const auto image = ReadTexture(nvrhiDevice, target); REQUIRE_MESSAGE(image.has_value(), fmt::format("{}", image.error())); Testing::CheckReferenceImage("SoftShadows", *image); CHECK(device->GetValidationErrorCount() == 0); } - Keep the default tolerance (
ImageTolerance) unless the feature needs another, and say why in a comment if it does. - Generate the image (below), look at it, and commit it with the test.
Generate or update images
- Build, then run the rendering tests with
LS_UPDATE_REFERENCE_IMAGES=1. Limit the run to the tests whose images should change:
In PowerShell:cmake --build --preset debug LS_UPDATE_REFERENCE_IMAGES=1 ctest --preset debug -L render -R "<test name pattern>"$env:LS_UPDATE_REFERENCE_IMAGES = '1', runctest, thenRemove-Item Env:LS_UPDATE_REFERENCE_IMAGES. - Look at every new or changed image in
Tests/ReferenceImages(git status). Check it shows exactly what the feature should render - this is the only review the image gets. - Run the rendering tests again without the variable; they must pass.
- Commit the images with the code change. The commit message says which images changed and why (for a Mesa update: the old and new versions).
When a reference-image test fails
The test writes <name>.actual.png and <name>.difference.png (mismatched pixels in red) to build/<preset>/RenderOutput; CI uploads the same files as an artifact (gh run download <run id>). Look at both before deciding whether the code or the image is wrong.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |