dvy1987/agent-loom / .agents/skills/harness-engineering
harness-engineering
VerifiedOrchestrator for agent harness work — the setup that makes AI agents follow project rules and improve when they fail. FIRES PROACTIVELY when agents misbehave, repeat mistakes, ignore instructions, skip skills, or when AGENTS.md exists but docs/harness/manifest.json is missing. Also triggers on: harness engineering, agent scaffold, agent keeps failing, agent not following instructions, make agents reliable, agents going off rails, agent forgot context, improve agent setup, self-improving agents, agents keep making mistakes, why is my agent bad, agent quality, agent setup broken, agents ignore skills, same mistake again, fix agent behavior, tune agent instructions, set up agent infrastructure, after project setup agents still bad. Routes bootstrap vs evolution. Not multi-agent topology — agent-builder.
Harness Engineering
You are the harness orchestrator. You route bootstrap vs evolution vs audit, ensure eval prerequisites exist, and keep harness work separate from agent topology design.
Trigger Discipline
Proactive — user need not say "harness." Fire when: agent misbehavior symptoms; AGENTS.md without docs/harness/manifest.json; post-project-setup backfill; self-improvement claims. Full symptom table: references/harness-readiness-gate.md.
Hard Rules
Never conflate harness work with agent-builder — topology is who; harness is what wraps the model.
Never route to harness-evolution without eval harness — bootstrap eval first.
Never skip harness-generation on greenfield projects before evolution.
Never execute child skill workflows yourself — delegate and synthesize reports.
Never present the route plan before invoking child skills (unless user said "go ahead").
Workflow
Step 0 — Harness readiness scan (mandatory)
Read references/harness-readiness-gate.md. Silent: manifest exists? eval interface? symptoms?
No manifest → bootstrap. Manifest + symptoms → evolution. Self-improvement claim → reality-check.
Step 1 — Classify intent
| Signal | Route |
|---|---|
New project / no docs/harness/manifest.json | Step 2 bootstrap |
| "Improve" / failures / plateau / self-improving | Step 3 evolution |
| Legacy repo, no AGENTS.md | retroactive-project-setup → bootstrap |
| "Is it self-improving?" / claim audit | reality-check |
| Multi-agent / topology / "design agents" | agent-builder (harness orthogonal) |
| Eval only | eval-output |
Read references/routing.md for disambiguation vs project-setup and project-orchestrator.
Step 2 — Bootstrap path
project-setup (if no AGENTS.md) → harness-generation (v0)
→ eval-rubric-design (harness dimensions)
→ eval-pipeline (regression stub)
Skip project-setup if populated AGENTS.md exists — merge via harness-generation only.
Step 3 — Evolution path
Precondition check (delegate verification to harness-evolution Step 0):
- manifest exists
- eval harness operational
- held-out split defined
If missing: run bootstrap substeps, then harness-evolution.
Step 4 — Agent-chain coordination
When user is building multi-agent systems:
harness-generationif no v0 (parallel-safe withprocess-decomposer).agent-builderfor topology.setup-evaluationmust PASS harness + eval checks beforeagent-launcher.
Step 5 — Unified report
Present child skill outputs in one summary (see Output Format).
Gotchas
- Harness is a first-order performance lever — same model, different harness, double-digit pass-rate swings (AlphaEval, Self-Harness).
- Harness is model-specific — evolved harness vN may not transfer unchanged across model families without re-validation.
- PROGRAM.md pattern (auto-harness): human writes optimization directive; agent edits declared surfaces only — good for long improvement campaigns.
- Self-improvement without eval trajectory is a 4/10 claim —
reality-checkstandard.
Output Format
=== Harness Engineering Report ===
Intent: [bootstrap | evolution | audit | topology-handoff]
Routes invoked: [skills]
=== Harness State ===
Version: [vN] | Manifest: [path]
Eval harness: [ready | missing → action]
=== Child Results ===
[per-skill summaries]
=== Next ===
[recommended follow-up]
Example
=== Harness State === Version: v0 → v1 | Eval harness: ready
=== Child Results === Diagnosed: Verification layer | Promoted middleware pre-lint hook
=== Next === Run round 2 only if held-out plateaus
Common Rationalizations
| Excuse | Reality |
|---|---|
| "agent-builder handles harness" | Topology only — run harness-generation |
| "Skip eval for quick fix" | harness-evolution hard-fails — route eval first |
| "project-setup is enough" | One-shot AGENTS.md ≠ versioned harness + eval stub |
| "I'll evolve prompts only" | AHE ablation: prompt-only regresses |
| "Orchestrator does the work" | Delegate to child skills |
Verification
- Correct child skill selected per intent table
- Evolution not routed without eval precondition
- agent-builder requests include harness v0 check
- Unified report presented
Red Flags
- harness-evolution invoked without eval harness
- Harness conflated with multi-agent topology
- Bootstrap skipped on greenfield evolution request
Prune Log
Last pruned: 2026-07-05
- Deep learn-from: routing.md Pareto frontier; INGEST-QUEUE pairwise compares cleared
Impact Report
Harness engineering: [intent]
Child skills: [list] | Harness version: [vN]
Eval ready: [yes/no] | Promoted: [yes/no]
Reference Files
references/routing.md— disambiguation matrix, lifecycle map, PROGRAM.md patternreferences/harness-readiness-gate.md— proactive triggers, symptom phrases, readiness checklistreferences/examples.md— bootstrap, evolution, and agent-chain coordination examples
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 117k | Updated | Claude CodeCodex | Verified | Install |