gpsaggese/helpers / .claude/skills/pytest.fix_ci_tests
pytest.fix_ci_tests
VerifiedAnalyze and fix failure of tests in GitHub CI
Goal
- Given a run of the GitHub CI, analyze the failures and fix the failing tests
Workflow
- Read
.claude/skills/pytest.rules.md
Parse Logs From CI
-
Parse the output of
invoke gh_workflow_list, which lists the failing tests -
The CI log of the failing tests is in a file
<CI_LOG>, e.g.:# Log is in 'tmp.failure.check_if_the_linter_was_run.gp_scratch_19.txt' -
The list of failing tests
<FAILING_TESTS>is in a file<REPRO_SCRIPT>, e.g.:... # To run the tests: ... > ./tmp.pytest_repro.fast_tests.sh
Check Whether the Failure Reproduces Locally
- Parse
<CI_LOG> - Run
<REPRO_SCRIPT>locally to check whether<FAILING_TESTS>fail there too- For tests that run Docker or Apple containers, run them with each container
build, per
## Analyze the Effect of Container Buildsin.claude/skills/pytest.rules.md
- For tests that run Docker or Apple containers, run them with each container
build, per
- Note any CI vs. local difference: env vars, secrets, container build, or
flakiness.
/pytest.fix_local_testscannot see this CI-specific context, so capture it here
Diagnose and Fix
- Run
/pytest.fix_local_testswith<REPRO_SCRIPT>as<FAILURE_INFO>to diagnose the root cause and fix the failing tests- If a failure reproduces only on CI, diagnose the CI-specific cause directly (env, secrets, container build, flakiness) and fix it
- In
plan-pytest.fix_ci_tests.mdin the current dir, report a table of which tests pass on CI and locally, and why they fail on CI and locally - Verify the fixes per
# Verification of the Fixin.claude/skills/pytest.rules.md
Ask the User to Commit
- Ask the user to review and commit the changes
- Wait for CI to pick up the changes and run the tests, then repeat the workflow
Verification
-
plan-pytest.fix_ci_tests.mdlists a fix for every failing test - Every fix follows
# Verification of the Fixin.claude/skills/pytest.rules.md - The failing tests pass on both CI and locally after the fix
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 318k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 318k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 318k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |