Skip to content

LeoYeAI/agent-master-skills / skills/agent-harness

agent-harness

Verified

Use when an agent should drive a goal in one of this repo's domains to a verified close, before starting the work. Compiles the goal into a task plan with goal_compiler.py and runs it through loop_controller.py, which verifies each task, caps retries and escalates.

references/domain_harness_design.md

View on GitHub
# Domain Harness Design

How a *domain folder full of skills* becomes an *agent harness*: a declarative manifest
mapping goals → skills → verifications, plus the repo primitives this skill deliberately
reuses instead of rebuilding.

## Sources

1. **AGENTS.md convention** — https://agents.md/ (launched Aug 2025; adopted by Codex,
   Cursor, Devin, Gemini CLI, Copilot; stewarded by the Agentic AI Foundation under the Linux
   Foundation since Dec 2025). The de-facto standard for declaring "how to build and verify
   work here" in a file agents read first. The harness manifest is the same idea made
   machine-readable per domain.
2. **Anthropic, "Effective harnesses for long-running agents", Nov 2025** —
   https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents. Its
   `feature-list.json` — each item carrying description + acceptance criteria + status, looped
   until all verified — is the closest published goal→tasks→verification manifest and the
   direct model for `plan.v1` / `state.v1`.
3. **Model Context Protocol** — https://modelcontextprotocol.io/ (Anthropic, Nov 2024;
   multi-vendor stewardship 2025). JSON-schema'd tool registries as the declared action space
   of a harness. The manifest's `tools[]` blocks follow the same declare-don't-discover
   philosophy: an agent should read what a skill can do, not grep for it mid-loop.
4. **LangGraph checkpointers** — https://langchain-ai.github.io/langgraph/. Durable
   execution: every super-step persisted, enabling pause/resume/replay and human-in-the-loop
   interrupts. The stdlib equivalent here: an atomically-written JSON state file
   (`os.replace`), append-only evidence entries, and git as the checkpoint layer.
5. **OpenAI Agents SDK** — https://openai.github.io/openai-agents-python/. `max_turns`
   raising `MaxTurnsExceeded` and tripwire guardrails: caps are *runtime errors*, not
   suggestions. Mirrored by controller exit codes 2 (escalate), 4 (close refused),
   5 (iteration cap) — a caller script can branch on them mechanically.
6. **Forward Future, "Loop Library" (MIT, vendored at `loop-library/` in this repo)** — loop
   anatomy (Observe/Choose/Act/Verify/Record/Repeat-or-stop) and the terminal-state taxonomy
   (success · clean no-op · blocked · approval-required · exhausted · stagnated). The harness
   adopts this vocabulary; "errors and exhausted budgets are never reported as success" is
   implemented as the no-force `close` gate.
7. **Anthropic, "Effective context engineering for AI agents", Sep 2025** —
   https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents.
   Compaction, structured note-taking, and subagent summaries for long horizons. The state
   file's `evidence` log + the close-time handoff block are the structured notes; a fresh
   session needs only `plan.json` + `state.json` to continue the loop.

## The three-layer architecture

```
Layer 1 — INVENTORY  (per domain, committed, regenerated by tooling)
  assets/harnesses/<domain>.json      what skills exist, what tools they ship,
                                      what checks prove each tool works,
                                      which agentic signals the skill already has
Layer 2 — PLAN       (per goal, generated at run time)
  plan.json                           ordered tasks: skill × objective ×
                                      verification[] × max_attempts × done_when
Layer 3 — STATE      (per run, the single source of truth)
  .agent-harness/state.json           task statuses, attempts, evidence log,
                                      iteration counter, close/handoff record
```

Layer 1 is diff-stable (`--no-timestamp`) so CI can regenerate and `git diff --exit-code`
it — manifest drift against the tree becomes a machine-checkable gate, exactly like this
repo's `derive_counters.py --check` discipline.

## Reuse map (import the pattern, don't rebuild the mechanism)

| Need | Reuse from this repo | The harness adds |
|---|---|---|
| Task/state persistence + handoff schema | `engineering/skills/tc-tracker` (atomic writes, append-only history, session handoff block) | A goal-scoped, domain-agnostic state file (`.agent-harness/`), not per-code-change records |
| Anti-overfit locked evaluator | `engineering/autoresearch-agent` (never modify `evaluate.py`; KEEP/DISCARD/CRASH) | The same invariant generalized: `verify` re-runs gates itself |
| N-agent tournament on one task | `engineering/agenthub` (worktrees, DAG, result ranker) | Nothing — route there when the task wants competing attempts |
| Deterministic fan-out/pipeline topologies | `engineering/workflow-builder` (Workflow-tool .js + validator) | Nothing — route there when orchestrating Claude Code's Workflow tool |
| Loop design/audit vocabulary | `loop-library/` (six-step cycle, stop-state taxonomy) | An executable enforcement of that vocabulary |
| Severity-gated ship decision | `engineering/skills/ship-gate` (CRITICAL/HIGH/ADVISORY verdict) | Use as a close-time check inside a task's `verification[]` |
| Honest self-scoring | `engineering/skills/self-eval` (matrix-locked composite) | Optional close-out step before the handoff |
| Bounded-autonomy STOP triggers | `engineering/skills/spec-driven-workflow` + `focused-fix` 3-strike rule | `escalate_on` defaults in every manifest |

## Namespacing (collisions this skill deliberately avoids)

- State directory is **`.agent-harness/`** — never `.agenthub/`, `.autoresearch/`, or
  `docs/TC/`, which belong to sibling skills.
- Command is **`/cs-harness`** — `/agenthub:*`, `/autoresearch-agent:*`, `/si:*`, `/tc` are taken.
- The runtime agent is **`harness-runner`** — "orchestrator" already denotes the
  `context: fork` domain routers, and `hub-coordinator` / `experiment-runner` are taken.
- The bare name "loop" is overloaded (Claude Code's built-in `/loop` scheduler,
  `loop-library`, workflow loop templates) — this skill never claims it.

## Extending a domain's harness

1. Author or improve skills so they carry the agentic signals (intake, refusal gates,
   verification, loop discipline, close-out — see the July 2026 audit's AR1–AR6 rubric in
   `audit/engineering-agentic-2026-07/RUBRIC.md`).
2. Regenerate the manifest:
   `python3 scripts/harness_manifest_builder.py --domain <domain> --repo-root . --out-dir assets/harnesses --no-timestamp`
   (run from this skill's directory).
3. The richer the skill's tools and `--sample` support, the more executable checks its tasks
   get for free — `manual-evidence` tasks are the fallback, not the goal.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall