Skip to content

gaia-research/gaia-skill-tree / .agents/skills/ci-churn

ci-churn

Verified

Measure CI churn cost for a pull request: how many commits and how many CI-compute seconds were burned on avoidable "fix the CI" push rounds. Use when you want to understand the efficiency cost of a PR's iteration cycle, get actionable pre-push check suggestions, or include a churn summary in a pipeline report. Triggers: "ci churn", "how much CI did we burn", "avoidable commits", "ci efficiency", "wasted CI time", "churn report", "/ci-churn". Called automatically by /fp-drift at pipeline close.

FileKindSize
SKILL.mdskill md6.1 KB
---
name: ci-churn
description: >-
  Measure CI churn cost for a pull request: how many commits and how many
  CI-compute seconds were burned on avoidable "fix the CI" push rounds.
  Use when you want to understand the efficiency cost of a PR's iteration
  cycle, get actionable pre-push check suggestions, or include a churn
  summary in a pipeline report. Triggers: "ci churn", "how much CI did we
  burn", "avoidable commits", "ci efficiency", "wasted CI time", "churn
  report", "/ci-churn". Called automatically by /fp-drift at pipeline close.
version: 1.0.0
---

# ci-churn — CI Efficiency Report

Measure how many commits and CI-compute seconds were spent on avoidable
"fix the CI" push rounds for a pull request, and surface the pre-push
checks that would have prevented them.

## When to use

- At the end of any feature pipeline run (`/fp-drift` calls this automatically)
- Standalone after any PR you want to audit for iteration efficiency
- As a retrospective tool: "we spent N minutes of CI compute on churn"

## How it works

### 1. Commit classification

Every commit in the PR is classified by its subject line:

| Label | Pattern examples | Meaning |
|---|---|---|
| `feature` | `feat(...)`, `add ...`, `refactor ...` | Intentional work |
| `review-fix` | `per review`, `address review findings` | Fixed after code review |
| `ci-fix` | `fix import`, `restore export`, `codeql`, `lazy-import`, `wheel smoke` | Fixed because CI caught something locally-detectable |

`feature` commits are the signal. `ci-fix` and `review-fix` are churn.

### 2. CI duration fetch

For each commit SHA, the GitHub Actions API is queried for every workflow
run triggered by that push. Duration is computed as
`updated_at − run_started_at` (no `run_duration_ms` field exists in the API).

### 3. Metrics computed

| Metric | Formula |
|---|---|
| **Churn ratio** | `avoidable_commits / total_commits` |
| **CI compute burned** | Sum of all run durations on avoidable commits |
| **Agent blocked-wait (min)** | Sum of *failed* run durations — the minimum time the agent was blocked waiting for bad results |
| **Agent blocked-wait (max)** | Same as CI compute burned — the agent also waited during the fix runs |
| **Session time** | Parsed from `--session-log` if provided (pi/claude-code JSONL) |

### 4. Root cause hints

Based on which `ci-fix` commit messages matched which patterns, the script
suggests concrete local pre-push checks that would have caught the same
failure before it hit CI.

## Run it

```bash
# Basic — auto-detects repo from git remote
python3 scripts/ci_churn.py <pr-number>

# With session log for agent time estimate (pi or claude-code JSONL)
python3 scripts/ci_churn.py <pr-number> --session-log ~/.pi/sessions/latest.jsonl

# Machine-readable JSON output
python3 scripts/ci_churn.py <pr-number> --json

# Explicit repo
python3 scripts/ci_churn.py <pr-number> --owner myorg --repo myrepo
```

## Reading the output

```
CI Churn Report — PR #1017  (gaia-research/gaia-skill-tree)
════════════════════════════════════════════════════════

Commits: 5 total  (1 feature · 1 review-fix · 3 ci-fix)
Churn ratio: 80.0%  (4 of 5 commits were avoidable)

CI compute burned on avoidable commits : 22m 49s
CI compute on failed runs (all commits): 4m 16s
Agent blocked-wait estimate (min / max): 4m 16s / 22m 49s

Commit breakdown
────────────────────────────────────────────────────────
      SHA  Label         CI (s)  Fails  Message
─────────  ────────────  ──────  ─────  ─────────────────────────────────
5f418b26b  feature            0      0  feat(intake): batch skill intake…
ff99b6165  △ review-fix     187      2  fix(intake): address review findings…
3f3f98776  ⚠ ci-fix         391      1  fix(prWriter): restore open_pr export…
e1b612175  ⚠ ci-fix         398      0  fix(pushFromFile): lazy-import yaml…
78e0028b5  ⚠ ci-fix         393      0  fix(pushFromFile): replace substring…

Suggested local pre-push checks
────────────────────────────────────────────────────────
  • python3 -c "from gaia_cli.main import main"   # import chain smoke test
  • python3 -m pytest tests/ -x -q --timeout=30   # fast local test gate
  • bandit -r src/ -ll                             # security lint (CodeQL class)
```

## Interpreting churn ratio

| Ratio | Signal |
|---|---|
| 0% | Perfect — every commit was intentional feature work |
| 1–20% | Healthy — minor CI surprises, typical for new code paths |
| 20–50% | Elevated — pre-push checks are missing or not being run |
| >50% | High churn — the local development loop is not catching what CI catches |

## Optional: session log integration

If you pass `--session-log PATH`, the script parses `durationMs` (pi) or
`duration_ms` (claude-code) fields from assistant turns in the JSONL log
and reports total agent session time alongside CI compute. This lets you
answer: "of the N minutes this agent spent on this PR, how many were
blocked waiting for CI results on avoidable failures?"

Session log paths:
- **pi:** `~/.pi/sessions/<project>/<session-id>.jsonl` (check `pi session ls`)
- **claude-code:** `~/.claude/projects/<project>/<session-id>.jsonl`

## Notes

- `run_started_at` → `updated_at` delta is used because the GitHub Actions
  API has no `run_duration_ms` field. This slightly over-counts (includes
  queue time), but is consistent across all runs.
- The first commit on a PR branch often has 0 runs because CI hadn't fired
  yet when the PR was opened — this is expected, not a bug.
- `gaia push --from-file`, `gaia dev add`, and pure tooling PRs will
  almost always show 0 feature commits and high churn ratios. That is
  accurate — the entire PR was iteration cost, not new capability.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall