LeoYeAI/agent-master-skills / skills/agent-harness
agent-harness
VerifiedUse when an agent should drive a goal in one of this repo's domains to a verified close, before starting the work. Compiles the goal into a task plan with goal_compiler.py and runs it through loop_controller.py, which verifies each task, caps retries and escalates.
references/domain_harness_design.md
View on GitHub# Domain Harness Design
How a *domain folder full of skills* becomes an *agent harness*: a declarative manifest
mapping goals → skills → verifications, plus the repo primitives this skill deliberately
reuses instead of rebuilding.
## Sources
1. **AGENTS.md convention** — https://agents.md/ (launched Aug 2025; adopted by Codex,
Cursor, Devin, Gemini CLI, Copilot; stewarded by the Agentic AI Foundation under the Linux
Foundation since Dec 2025). The de-facto standard for declaring "how to build and verify
work here" in a file agents read first. The harness manifest is the same idea made
machine-readable per domain.
2. **Anthropic, "Effective harnesses for long-running agents", Nov 2025** —
https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents. Its
`feature-list.json` — each item carrying description + acceptance criteria + status, looped
until all verified — is the closest published goal→tasks→verification manifest and the
direct model for `plan.v1` / `state.v1`.
3. **Model Context Protocol** — https://modelcontextprotocol.io/ (Anthropic, Nov 2024;
multi-vendor stewardship 2025). JSON-schema'd tool registries as the declared action space
of a harness. The manifest's `tools[]` blocks follow the same declare-don't-discover
philosophy: an agent should read what a skill can do, not grep for it mid-loop.
4. **LangGraph checkpointers** — https://langchain-ai.github.io/langgraph/. Durable
execution: every super-step persisted, enabling pause/resume/replay and human-in-the-loop
interrupts. The stdlib equivalent here: an atomically-written JSON state file
(`os.replace`), append-only evidence entries, and git as the checkpoint layer.
5. **OpenAI Agents SDK** — https://openai.github.io/openai-agents-python/. `max_turns`
raising `MaxTurnsExceeded` and tripwire guardrails: caps are *runtime errors*, not
suggestions. Mirrored by controller exit codes 2 (escalate), 4 (close refused),
5 (iteration cap) — a caller script can branch on them mechanically.
6. **Forward Future, "Loop Library" (MIT, vendored at `loop-library/` in this repo)** — loop
anatomy (Observe/Choose/Act/Verify/Record/Repeat-or-stop) and the terminal-state taxonomy
(success · clean no-op · blocked · approval-required · exhausted · stagnated). The harness
adopts this vocabulary; "errors and exhausted budgets are never reported as success" is
implemented as the no-force `close` gate.
7. **Anthropic, "Effective context engineering for AI agents", Sep 2025** —
https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents.
Compaction, structured note-taking, and subagent summaries for long horizons. The state
file's `evidence` log + the close-time handoff block are the structured notes; a fresh
session needs only `plan.json` + `state.json` to continue the loop.
## The three-layer architecture
```
Layer 1 — INVENTORY (per domain, committed, regenerated by tooling)
assets/harnesses/<domain>.json what skills exist, what tools they ship,
what checks prove each tool works,
which agentic signals the skill already has
Layer 2 — PLAN (per goal, generated at run time)
plan.json ordered tasks: skill × objective ×
verification[] × max_attempts × done_when
Layer 3 — STATE (per run, the single source of truth)
.agent-harness/state.json task statuses, attempts, evidence log,
iteration counter, close/handoff record
```
Layer 1 is diff-stable (`--no-timestamp`) so CI can regenerate and `git diff --exit-code`
it — manifest drift against the tree becomes a machine-checkable gate, exactly like this
repo's `derive_counters.py --check` discipline.
## Reuse map (import the pattern, don't rebuild the mechanism)
| Need | Reuse from this repo | The harness adds |
|---|---|---|
| Task/state persistence + handoff schema | `engineering/skills/tc-tracker` (atomic writes, append-only history, session handoff block) | A goal-scoped, domain-agnostic state file (`.agent-harness/`), not per-code-change records |
| Anti-overfit locked evaluator | `engineering/autoresearch-agent` (never modify `evaluate.py`; KEEP/DISCARD/CRASH) | The same invariant generalized: `verify` re-runs gates itself |
| N-agent tournament on one task | `engineering/agenthub` (worktrees, DAG, result ranker) | Nothing — route there when the task wants competing attempts |
| Deterministic fan-out/pipeline topologies | `engineering/workflow-builder` (Workflow-tool .js + validator) | Nothing — route there when orchestrating Claude Code's Workflow tool |
| Loop design/audit vocabulary | `loop-library/` (six-step cycle, stop-state taxonomy) | An executable enforcement of that vocabulary |
| Severity-gated ship decision | `engineering/skills/ship-gate` (CRITICAL/HIGH/ADVISORY verdict) | Use as a close-time check inside a task's `verification[]` |
| Honest self-scoring | `engineering/skills/self-eval` (matrix-locked composite) | Optional close-out step before the handoff |
| Bounded-autonomy STOP triggers | `engineering/skills/spec-driven-workflow` + `focused-fix` 3-strike rule | `escalate_on` defaults in every manifest |
## Namespacing (collisions this skill deliberately avoids)
- State directory is **`.agent-harness/`** — never `.agenthub/`, `.autoresearch/`, or
`docs/TC/`, which belong to sibling skills.
- Command is **`/cs-harness`** — `/agenthub:*`, `/autoresearch-agent:*`, `/si:*`, `/tc` are taken.
- The runtime agent is **`harness-runner`** — "orchestrator" already denotes the
`context: fork` domain routers, and `hub-coordinator` / `experiment-runner` are taken.
- The bare name "loop" is overloaded (Claude Code's built-in `/loop` scheduler,
`loop-library`, workflow loop templates) — this skill never claims it.
## Extending a domain's harness
1. Author or improve skills so they carry the agentic signals (intake, refusal gates,
verification, loop discipline, close-out — see the July 2026 audit's AR1–AR6 rubric in
`audit/engineering-agentic-2026-07/RUBRIC.md`).
2. Regenerate the manifest:
`python3 scripts/harness_manifest_builder.py --domain <domain> --repo-root . --out-dir assets/harnesses --no-timestamp`
(run from this skill's directory).
3. The richer the skill's tools and `--sample` support, the more executable checks its tasks
get for free — `manual-evidence` tasks are the fallback, not the goal.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |