gpsaggese/helpers / .claude/skills/pytest.fix_ci_tests
pytest.fix_ci_tests
VerifiedAnalyze and fix failure of tests in GitHub CI
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 2.0 KB |
SKILL.md
View on GitHub---
description: Analyze and fix failure of tests in GitHub CI
model: haiku
---
# Goal
- Given a run of the GitHub CI, analyze the failures and fix the failing tests
# Workflow
- Read `.claude/skills/pytest.rules.md`
## Parse Logs From CI
- Parse the output of `invoke gh_workflow_list`, which lists the failing tests
- The CI log of the failing tests is in a file `<CI_LOG>`, e.g.:
```bash
# Log is in 'tmp.failure.check_if_the_linter_was_run.gp_scratch_19.txt'
```
- The list of failing tests `<FAILING_TESTS>` is in a file `<REPRO_SCRIPT>`, e.g.:
```bash
... # To run the tests:
... > ./tmp.pytest_repro.fast_tests.sh
```
## Check Whether the Failure Reproduces Locally
- Parse `<CI_LOG>`
- Run `<REPRO_SCRIPT>` locally to check whether `<FAILING_TESTS>` fail there too
- For tests that run Docker or Apple containers, run them with each container
build, per `## Analyze the Effect of Container Builds` in
`.claude/skills/pytest.rules.md`
- Note any CI vs. local difference: env vars, secrets, container build, or
flakiness. `/pytest.fix_local_tests` cannot see this CI-specific context, so
capture it here
## Diagnose and Fix
- Run `/pytest.fix_local_tests` with `<REPRO_SCRIPT>` as `<FAILURE_INFO>` to
diagnose the root cause and fix the failing tests
- If a failure reproduces only on CI, diagnose the CI-specific cause directly
(env, secrets, container build, flakiness) and fix it
- In `plan-pytest.fix_ci_tests.md` in the current dir, report a table of which tests
pass on CI and locally, and why they fail on CI and locally
- Verify the fixes per `# Verification of the Fix` in
`.claude/skills/pytest.rules.md`
## Ask the User to Commit
- Ask the user to review and commit the changes
- Wait for CI to pick up the changes and run the tests, then repeat the workflow
# Verification
- [ ] `plan-pytest.fix_ci_tests.md` lists a fix for every failing test
- [ ] Every fix follows `# Verification of the Fix` in
`.claude/skills/pytest.rules.md`
- [ ] The failing tests pass on both CI and locally after the fix
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |