gpsaggese/helpers / .claude/skills/testing.update_test_outcomes
testing.update_test_outcomes
VerifiedUpdate the expected outcome of failing unit tests
Goal
- The user passes a list of tests, or a repro script, with failing tests
<FAILING_TESTS>whose actual and expected outputs mismatch
Important
- For all code, follow
.claude/skills/coding.rules.md - Follow the invariants and conventions for unit tests in
.claude/skills/testing.rules.md
Workflow
Run the Failing Tests
- Run the tests one at a time, the way the user suggested, to confirm they fail due to mismatching outputs
Classify the Assertion Method
-
For each failing test method, check whether the result is checked with
self.check_string()self.assert_equal()or a similar assertion (e.g.,hunteuti.assert_sys_calls(),self.assertEqual())
-
Print a table with the following info: path of test | fail / pass | check_string or assert_equal
| Path | Test | Status | Assertion Method | |------|------|--------|------------------| | dev_scripts_helpers/documentation/test/test_notes_to_pdf.py | Test_notes_to_pdf_latex_options::test1 | FAIL | self.assert_equal() with fuzzy_match=True | | dev_scripts_helpers/documentation/test/test_notes_to_pdf.py | Test_notes_to_pdf_latex_options::test2 | FAIL | self.assert_equal() with fuzzy_match=True | | dev_scripts_helpers/documentation/test/test_notes_to_pdf.py | Test_notes_to_pdf_output_types::test1 | FAIL | self.assert_equal() with fuzzy_match=True | -
Save this to
tmp.update_test_outcomes.failing_tests.md
Update the Expected Output
-
For each failing test method:
- If the result is checked with
self.check_string()- Run the test with the pytest option
--update_goldento refresh the output
- Run the test with the pytest option
- If the result is checked with
self.assert_equal()or a similar assertion (e.g.,hunteuti.assert_sys_calls()), read the file<TEST_DIR>/tmp.initial.actual.txt(e.g.,dev_scripts_helpers/documentation/test/outcomes/Test_run_pandoc_to_html.test1) that holds the actual variable, and update theexpectedvariable- E.g.,
expected = """ [{'args': (f'jupytext --to ipynb --update {py_file}',), 'function': 'hsystem.system', 'kwargs': {}}, {'args': (f'jupytext --sync {py_file}',), 'function': 'hsystem.system', 'kwargs': {}}] """
- E.g.,
- If the result is checked with
-
Follow
.claude/skills/testing.rules.md:## Code Formatting in Tests# Test Input and Output Handling
-
Do not change how results are checked
- E.g., do not replace
self.assert*()withself.check_string()or vice versa - E.g., do not replace
self.assert*()withself.assertIn()
- E.g., do not replace
Verification
- Run all the tests to make sure they are passing
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |