Skip to content

ilandahan/AID / skills/pipeline-orchestrator

pipeline-orchestrator

Verified

Drives the Phase 4-5 dev pipeline state machine with sub-agent reviews and bounded autoresearch gates. Use when running /pipeline or executing an approved plan.

FileKindSize
SKILL.extended.mdmarkdown11.7 KB
SKILL.mdskill md70.3 KB
config.default.jsonconfig1.5 KB
gate.mjsscript38.3 KB
gate.test.mjsscript31.9 KB
references/escalation-protocol.mdmarkdown4.4 KB
references/load-bearing-assumptions.mdmarkdown3.2 KB
references/state-machine.mdmarkdown8.8 KB
test-escalation-loop.shexecutablescript13.8 KB

references/load-bearing-assumptions.md

View on GitHub
# Load-Bearing Assumptions

Every pipeline step encodes assumptions about model limitations. As models improve,
these assumptions may become outdated. Review quarterly.

Source: "Load-bearing assumptions deserve testing" — Anthropic, Harness Design (March 2026)

---

## Assumptions by Step

| # | Assumption | Step It Affects | Why We Believe It | How to Test | Last Verified |
|---|-----------|----------------|-------------------|-------------|---------------|
| 1 | Self-review is biased — models praise their own work | CODE_REVIEW, TEST_REVIEW | Anthropic article + observed score inflation | Run inline review vs sub-agent review on same code, compare scores | Not yet |
| 2 | Human fixes are more reliable than auto-fix | FIX_CODE, FIX_TESTS | Model can introduce regressions when auto-fixing | Enable auto-fix on 10 tasks, compare iteration count and final scores | Not yet |
| 3 | 3 code review iterations is sufficient | max_iterations.code_review | Empirical: issues usually resolve in 2 | Track actual iterations — if >50% reach max, increase | Not yet |
| 4 | Sequential steps are better than parallel | Step ordering | Fixing code before testing prevents wasted test-writing | Run parallel variant, compare total cost and final quality | Not yet |
| 5 | Code review must happen before tests | CODE_REVIEW → TDD order | Catching security/quality issues early saves test rework | Swap order on 5 tasks, measure rework rate | Not yet |
| 6 | Sub-agent isolation improves review quality | All review steps | Anthropic article: context causes confirmation bias | Run non-isolated review (inline), compare severity distribution | Not yet |
| 7 | Binary PASS/FAIL is the right gate mechanism | CODE_REVIEW, TEST_REVIEW | Simple, decisive, no ambiguity | Now upgraded to numeric scoring — monitor if threshold tuning is needed | Sprint 1 |
| 8 | Opus 4.6 needs the same step decomposition | All steps | Untested — article suggests Opus can sustain longer tasks | Create simplified pipeline (fewer steps), compare output quality | Not yet |
| 9 | Review criteria preview improves first-pass quality | DEVELOP step | Article: "criteria steer outputs" | Compare first-pass review scores before/after criteria preview | Sprint 1 |
| 10 | Skepticism prompting improves review quality | CODE_REVIEW, TEST_REVIEW | Article: "evaluators respond better to skepticism prompting" | Compare score distributions before/after skepticism prompts | Sprint 1 |

---

## How to Test an Assumption

1. **Define the experiment**: What exactly will you change?
2. **Define the metric**: What will you measure? (scores, iterations, cost, time)
3. **Run A/B**: Same task, different pipeline config
4. **Record results**: Add to `history.json` with experiment tag
5. **Update this doc**: Change "Last Verified" and add findings

---

## When to Re-Test

- After a major model upgrade (new Claude version)
- After changing pipeline config significantly
- If a step consistently escalates (assumption #3 may be wrong)
- If pipeline costs exceed expectations (assumption #8 may be wrong)
- Quarterly, regardless

---

## Retired Assumptions

| # | Assumption | Retired | Why |
|---|-----------|---------|-----|
| — | None yet | — | — |
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall