Skip to content

jacob-balslev/skills / skills/knowledge-organization/skill-evolution

skill-evolution

Verified

Use when running or auditing Skill Graph's corpus-level `evolve` operation: the continuous skill-improvement loop that analyzes a skill library, triages a priority queue, executes bounded improve/scaffold/eval-generation actions, verifies the result, records checkpoints, and repeats. Covers `skill-graph evolve`, `lib/audit/skill-evolution-loop.js`, the Karpathy keep-or-revert spine, the priority signals based on Audit Status, standalone workspace flags, and the boundary between corpus walking and single-skill audit/improve/evaluate operations. Do NOT use for initial skill scaffolding alone (use skill-scaffold), single-skill schema/eval checks (use graph-audit or the audit operation), or generic evaluation rubric design (use evaluation / eval-driven-development).

Skill Evolution

Concept of the skill

Skill evolution is the corpus walker around individual skill maintenance. The primitives are: a skill corpus; each skill's SKILL.md teaching/routing contract; each skill's audit-state.json Audit Status sidecar; an analyzer that turns Audit Status, scores, and registry facts into a priority queue; a triage step that selects bounded work; an executor that runs improve/scaffold/eval-generation actions; a verification step that keeps or reverts; and checkpoint logs that let the run resume.

Coverage

Skill evolution is the project-grounded discipline for running Skill Graph's corpus-level evolve operation. It covers:

  • The public command: skill-graph evolve
  • The bundled implementation: skill-graph/lib/audit/skill-evolution-loop.js
  • The action loop: analyze -> triage -> execute -> verify -> checkpoint
  • The priority model: Audit Status, application verdicts, scores, registry facts, staleness, and action types
  • The keep-or-revert spine for improvement actions
  • Standalone workspace flags and the removal of cross-repo path assumptions
  • The boundary between walking a corpus and auditing, improving, evaluating, or scaffolding one skill

This skill is project-grounded to Skill Graph. It does not teach generic habit formation, generic evaluation theory, or first-time skill authoring.

Philosophy of the skill

Single-skill audit keeps one artifact honest. Skill evolution keeps the corpus from rotting silently. The loop exists because a large skill library has more potential drift than a human will remember to revisit.

The current implementation is not a simple hard-coded audit -> improve -> evaluate script. It is a continuous queue driver: analyze the current corpus, triage bounded work, execute one allowed action, verify the result, record findings/checkpoints, and optionally re-analyze. That shape matters because the queue can include different action types, and because a failed change must leave the corpus no worse than before.

Audit Status state lives in audit-state.json, not in SKILL.md frontmatter. SKILL.md teaches the skill; the sidecar records what the audit loop has proven about it. The evolve loop must preserve that split.

Key Files

Workspace-relative filePurpose
skill-graph/bin/skill-graph.jsPublic CLI surface for skill-graph evolve, help text, bundled-script mapping, and standalone requirements.
skill-graph/lib/audit/skill-evolution-loop.jsCurrent corpus walker: analyze, triage, execute, verify, checkpoint, continuous mode, pilot lanes, failure budget.
skill-graph/lib/audit/run-skill-improvement-loop.jsImprovement executor used for bounded skill edits and keep-or-revert behavior.
skill-graph/lib/audit/evaluate-skill.jsEvaluation runner that writes eval and behavior verdict state to audit-state.json.
skill-graph/skill-audit-loop/SKILL_AUDIT_LOOP.mdBinding operation doctrine: audit, improve, evaluate, evolve, write surfaces, and Behavior Gate semantics.
skill-graph/README.mdUser-facing standalone install, smoke-test, evolve flags, and exit-code guidance.

1. The Corpus Walker

The current evolve operation is a corpus queue driver:

analyze current skill library
triage top items by priority and allowed actions
execute one bounded action per item
verify the result
checkpoint progress and findings
repeat when continuous mode is enabled

The important operational guarantees are:

  • Bounded work: --top, --max-cycles, --max-iterations, --min-priority, --actions, and --pilot constrain the run.
  • Failure containment: --failure-budget stops repeated failures instead of letting the run thrash.
  • Resumability: checkpoints let interrupted runs continue.
  • Standalone operation: --workspace-root, --skills-dir, and --output-dir make the loop work outside the original Development monorepo.
  • Meta-skill protection: the implementation filters or gates meta skills unless an explicit pilot lane permits them.

2. Relationship to the Four Operations

OperationUnit of workWrites
auditOne skill's Integrity Gate and optional graded checksaudit-state.json plus evidence artifacts
improveOne bounded edit to one skill or assetSKILL.md or eval artifact, then keep-or-revert
evaluateOne skill's eval suiteaudit-state.json verdicts, scores, and receipts
evolveA prioritized corpus queueThe same writes as the operations/actions it invokes

Use audit, improve, or evaluate when the target skill is already known. Use evolve when the work is to walk the corpus by priority, keep progress resumable, and decide which skill or asset should be improved next.

3. Priority Signals

The queue is not a telemetry crawler. It reads structured skill state and registry facts. Important signals include:

  • application_verdict: unverified or negative behavior signals raise priority.
  • structural_verdict and truth_verdict: failing Integrity Gate slices raise priority.
  • last_audited: older audit state raises priority.
  • eval_score: missing or lower scores raise priority when available.
  • Registry/pilot metadata: some meta skills are frozen unless a specific pilot lane is active.

The exact queue formula belongs to skill-graph/lib/audit/skill-evolution-loop.js; this skill teaches the operating model and the safety boundaries, not a duplicated formula.

4. Commands

# Process a bounded queue
skill-graph evolve --top 5 --max-cycles 3

# Re-analyze and repeat until the cycle cap, convergence, or failure budget
skill-graph evolve --continuous --max-cycles 20 --min-priority 5

# Run the full auto-improve spine
skill-graph evolve --auto-improve --max-cycles 3 --failure-budget 5

# Analyze without executing improvements
skill-graph evolve --analyze-only

# Resume from a checkpoint
skill-graph evolve --resume

# Standalone workspace
skill-graph evolve --workspace-root /path/to/my-skills --skills-dir /path/to/my-skills/skills --output-dir /path/to/my-skills/audits

Exit codes:

CodeMeaning
0Loop completed successfully, or --analyze-only finished.
1Fatal error: missing dependency, unresolvable skill root, invalid workspace, or equivalent setup failure.
2Failure budget exceeded.

Evals

This skill includes a sibling comprehension eval file for audit-loop grading. Keep eval_state: unverified until the eval is run by an independent grader and produces a receipt.

Verification

After applying this skill, verify:

  • skill-graph evolve --help names the same flags and exit codes this skill teaches.
  • The implementation path in skill-graph/bin/skill-graph.js points to skill-graph/lib/audit/skill-evolution-loop.js.
  • Audit/eval/provenance writes are described as audit-state.json writes, not SKILL.md frontmatter writes.
  • The skill distinguishes corpus walking from one-skill audit, improve, evaluate, and scaffold work.
  • Local truth-source drift is PASS when hashes are recorded, or UNVERIFIED when hashes are absent.

Do NOT Use When

Instead of this skillUseWhy
Auditing one specific skill for schema, relations, sidecar, or manifest consistencygraph-audit or the audit operationSkill-evolution owns the corpus queue, not one artifact's static consistency.
Creating a new skill from scratchskill-scaffoldScaffolding owns the initial skill shape; evolve may invoke scaffold-like actions inside a queue.
Designing or interpreting a scoring rubricevaluation or eval-driven-developmentSkill-evolution decides when to invoke evaluation, not how evaluation is designed.
Choosing which skill should route for a user requestskill-routerRouting picks an owner skill for a request; evolve picks maintenance work for a corpus.

Key Sources

  • skill-graph/bin/skill-graph.js — public command contract.
  • skill-graph/lib/audit/skill-evolution-loop.js — current implementation.
  • skill-graph/lib/audit/run-skill-improvement-loop.js — keep-or-revert executor.
  • skill-graph/lib/audit/evaluate-skill.js — evaluation and sidecar write behavior.
  • skill-graph/skill-audit-loop/SKILL_AUDIT_LOOP.md — operation doctrine.
  • skill-graph/README.md — user-facing standalone usage.
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 467kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 467kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 467kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 193kUpdated Claude CodeCodexVerifiedInstall