Skip to content

notque/vexjoy-agent / skills/meta/toolkit

toolkit

Verified

Toolkit management: create and evaluate skills and agents, manage routing tables, generate Claude.md.

FileKindSize
SKILL.mdskill md15.5 KB
agents/skill-creator/analyzer.mdmarkdown4.8 KB
agents/skill-creator/comparator.mdmarkdown4.7 KB
agents/skill-creator/grader.mdmarkdown3.9 KB
assets/skill-creator/eval_viewer.htmlother57.1 KB
references/agent-comparison.mdmarkdown11.5 KB
references/agent-comparison/benchmark-tasks.mdmarkdown6.8 KB
references/agent-comparison/do-creation-compliance-tasks.jsonconfig8.5 KB
references/agent-comparison/examples-and-errors.mdmarkdown1.0 KB
references/agent-comparison/grading-rubric.mdmarkdown4.3 KB
references/agent-comparison/methodology.mdmarkdown7.4 KB
references/agent-comparison/optimization-guide.mdmarkdown12.0 KB
references/agent-comparison/optimization-tasks.example.jsonconfig853 B
references/agent-comparison/optimize-phase.mdmarkdown9.3 KB
references/agent-comparison/read-only-ops-short-tasks.jsonconfig375 B
references/agent-comparison/report-template.mdmarkdown3.8 KB
references/agent-comparison/socratic-debugging-body-short-tasks.jsonconfig372 B
references/agent-comparison/socratic-debugging-trigger-tasks.jsonconfig2.8 KB
references/agent-creator.mdmarkdown10.9 KB
references/agent-creator/agent-design-patterns.mdmarkdown12.4 KB
references/agent-creator/agent-eval-design.mdmarkdown7.7 KB
references/agent-creator/agent-frontmatter-template.mdmarkdown9.3 KB
references/agent-evaluation.mdmarkdown10.6 KB
references/agent-evaluation/batch-evaluation.mdmarkdown1.6 KB
references/agent-evaluation/common-issues.mdmarkdown2.1 KB
references/agent-evaluation/report-templates.mdmarkdown2.2 KB
references/agent-evaluation/scoring-rubric.mdmarkdown2.4 KB
references/blind-ab-tests.mdmarkdown7.8 KB
references/generate-claudemd.mdmarkdown11.0 KB
references/generate-claudemd/CLAUDEMD_TEMPLATE.mdmarkdown4.1 KB
references/generate-claudemd/examples-and-errors.mdmarkdown7.4 KB
references/github-profile-rules.mdmarkdown7.5 KB
references/github-profile-rules/github-api-patterns.mdmarkdown8.2 KB
references/github-profile-rules/rule-categories.mdmarkdown4.8 KB
references/hook-development.mdmarkdown10.9 KB
references/hook-development/architecture.mdmarkdown3.7 KB
references/hook-development/code-examples.mdmarkdown9.4 KB
references/hook-development/preferred-patterns.mdmarkdown6.5 KB
references/hook-development/telemetry-database.mdmarkdown6.5 KB
references/routing-table-updater.mdmarkdown10.2 KB
references/routing-table-updater/batch-mode.mdmarkdown2.1 KB
references/routing-table-updater/conflict-resolution.mdmarkdown3.8 KB
references/routing-table-updater/error-handling.mdmarkdown1.9 KB
references/routing-table-updater/examples.mdmarkdown4.8 KB
references/routing-table-updater/extraction-patterns.mdmarkdown2.2 KB
references/routing-table-updater/routing-format.mdmarkdown3.6 KB
references/routing-table-updater/skill-examples.mdmarkdown2.7 KB
references/skill-composer.mdmarkdown9.1 KB
references/skill-composer/compatibility-matrix.mdmarkdown13.0 KB
references/skill-composer/composition-patterns.mdmarkdown4.9 KB
references/skill-composer/examples.mdmarkdown17.1 KB
references/skill-composer/skill-patterns.mdmarkdown12.5 KB
references/skill-creator.mdmarkdown28.9 KB
references/skill-creator/agent-template.mdmarkdown18.5 KB
references/skill-creator/artifact-schemas.mdmarkdown8.2 KB
references/skill-creator/bundled-components.mdmarkdown2.3 KB
references/skill-creator/complexity-tiers.mdmarkdown8.0 KB
references/skill-creator/domain-research-targets.mdmarkdown13.2 KB
references/skill-creator/enrichment-workflow.mdmarkdown9.9 KB
references/skill-creator/error-catalog.mdmarkdown9.9 KB
references/skill-creator/preferred-patterns.mdmarkdown11.7 KB
references/skill-creator/progressive-disclosure.mdmarkdown8.5 KB
references/skill-creator/skill-template.mdmarkdown14.6 KB
references/skill-creator/workflow-patterns.mdmarkdown7.6 KB
references/system-upgrade.mdmarkdown11.0 KB
references/system-upgrade/component-audit-checklists.mdmarkdown8.4 KB
references/system-upgrade/upgrade-failure-modes.mdmarkdown8.7 KB
references/system-upgrade/upgrade-signal-parsing.mdmarkdown7.3 KB
references/toolkit-evolution.mdmarkdown14.5 KB
references/toolkit-evolution/diagnose-scripts.mdmarkdown9.5 KB
references/toolkit-evolution/evolution-history.mdmarkdown5.5 KB
references/toolkit-evolution/evolution-report-template.mdmarkdown716 B
references/toolkit-evolution/evolve-preferred-patterns.mdmarkdown6.9 KB
references/toolkit-evolution/evolve-scripts.mdmarkdown3.7 KB
references/toolkit-governance.mdmarkdown16.0 KB
references/toolkit-governance/adr-lifecycle.mdmarkdown10.9 KB
references/toolkit-governance/frontmatter-compliance.mdmarkdown11.1 KB
references/toolkit-governance/hook-standardization.mdmarkdown21.5 KB
references/toolkit-governance/routing-table-patterns.mdmarkdown11.6 KB
references/weak-model-uplift.mdmarkdown10.3 KB
scripts/agent-comparison/compare.pyscript2.3 KB
scripts/agent-comparison/generate_variant.pyscript19.6 KB
scripts/agent-comparison/optimize_loop.pyscript89.3 KB
scripts/agent-comparison/trigger_eval.pyscript19.5 KB
scripts/routing-table-updater/extract_metadata.pyscript10.8 KB
scripts/routing-table-updater/generate_routes.pyscript10.5 KB
scripts/routing-table-updater/scan.pyscript4.7 KB
scripts/routing-table-updater/update_routing.pyscript12.4 KB
scripts/routing-table-updater/validate.pyscript7.3 KB
scripts/skill-composer/build_dag.pyscript12.5 KB
scripts/skill-composer/discover_skills.pyscript12.1 KB
scripts/skill-composer/validate.pyscript12.3 KB
scripts/skill-creator/aggregate_benchmark.pyscript10.0 KB
scripts/skill-creator/eval_compare.pyscript10.2 KB
scripts/skill-creator/optimize_description.pyscript12.5 KB
scripts/skill-creator/package_results.pyscript7.8 KB
scripts/skill-creator/run_eval.pyscript6.3 KB
---
name: toolkit
description: "Toolkit management: create and evaluate skills and agents, manage routing tables, generate Claude.md."
user-invocable: true
allowed-tools:
  - Read
  - Write
  - Edit
  - Bash
  - Glob
  - Grep
  - Agent
  - Task
  - Skill
routing:
  force_route: true
  not_for: "application code (use workflow), code review (use review)"
  triggers:
    - "create skill"
    - "create agent"
    - "scaffold skill"
    - "scaffold agent"
    - "new skill"
    - "new agent"
    - "skill template"
    - "agent template"
    - "eval skill"
    - "evaluate agent"
    - "benchmark skill"
    - "benchmark agents"
    - "compare agents"
    - "A/B test agents"
    - "optimize description"
    - "evolve toolkit"
    - "toolkit evolution"
    - "self-improve"
    - "update routing tables"
    - "routing maintenance"
    - "generate claude.md"
    - "create claude.md"
    - "compose skills"
    - "bake-off"
    - "skill quality"
    - "weaker model"
    - "weak model uplift"
    - "improve skill for weaker models"
    - "edit skill"
    - "routing table governance"
    - "ADR management"
    - "toolkit maintenance"
    - "check coverage"
    - "skill compliance"
    - "hook standardization"
    - "cross-component consistency"
    - "upgrade agents"
    - "system upgrade"
    - "claude update"
    - "upgrade skills"
    - "adapt workflow"
    - "apply claude update"
    - "apply update"
    - "system health"
    - "update system"
    - "new claude version"
    - "apply retro"
    - "create hook"
    - "hook development"
    - "event handler"
    - "PostToolUse"
    - "PreToolUse"
    - "SessionStart"
    - "routing telemetry hook"
    - "hook registration"
    - "github rules"
    - "profile analysis"
    - "coding style extraction"
    - "github conventions"
    - "programming rules"
  category: meta-tooling
  pairs_with:
    - review
    - workflow
    - assessment
    - docs-sync-checker
    - testing
    - pr-workflow
---

# Toolkit

Thirteen modes covering the full toolkit lifecycle: creating and improving skills
and agents; evaluating agents; maintaining routing tables; generating CLAUDE.md;
composing multi-skill DAGs; running the evolution loop; governance; system upgrades; hook development; and GitHub profile rule extraction. Classify the request
and follow the matching section.

## Mode Selection

| Mode | Signals | Section |
|------|---------|---------|
| **Skill Creator** | create skill, scaffold skill, new skill, build a skill | Create Skill |
| **Agent Creator** | create agent, scaffold agent, new agent | Create Agent |
| **Weak-Model Uplift** | weaker model, uplift skill, make a skill work for Opus 4.6, improve guidance from generated output | Uplift for Weaker Models |
| **Agent Comparison** | compare agents, A/B test agents, benchmark agents, benchmark skill, bake-off | Compare Agents |
| **Agent Evaluation** | evaluate agent quality, audit agent, grade agent, eval skill | Evaluate Agent |
| **Skill Composer** | compose skills, DAG orchestration, skill pipeline | Compose Skills |
| **Routing Tables** | update routing tables, sync routing, routing drift | Update Routing |
| **Toolkit Evolution** | evolve toolkit, self-improve, discover gaps | Evolve Toolkit |
| **Generate CLAUDE.md** | generate claude.md, create claude.md, init | Generate CLAUDE.md |
| **Governance** | edit skill, routing governance, ADR lifecycle, hook standardization, consistency check | Load `references/toolkit-governance.md` |
| **System Upgrade** | claude update, upgrade skills, apply update, apply retro | Load `references/system-upgrade.md` |
| **Hook Development** | create hook, PreToolUse, PostToolUse, SessionStart, hook registration | Load `references/hook-development.md` |
| **GitHub Profile Rules** | github rules, profile analysis, coding style extraction | Load `references/github-profile-rules.md` |

---

## Create Skill

Phases: **INTENT -> DRAFT -> TEST -> REGISTER**

1. **Capture intent.** What should the skill do? When should it trigger? What output? Are outputs objectively verifiable (code, data) or subjective (writing, design)?
2. **Duplicate check.** Run `grep -i "<domain>" skills/*/SKILL.md` to check existing coverage. If an umbrella skill covers the domain, add a reference file instead.
3. **Write SKILL.md.** Follow `references/skill-creator/skill-template.md` for frontmatter structure. Apply Dense-Complete Writing standard. Frontmatter must include: name, description, routing (triggers, not_for, category, pairs_with), allowed-tools.
4. **Test.** Try 3 should-trigger, 2 should-not-trigger, and 2 near-miss prompts with the skill loaded. Revise the SKILL.md until routing and output are right.
5. **Register.** Run `python3 scripts/generate-skill-index.py` to update routing.

Load `references/skill-creator.md` for the full workflow. Deep references in `references/skill-creator/` cover progressive disclosure, artifact schemas, complexity tiers, error catalog, enrichment workflow, and more.

Scripts: `scripts/skill-creator/`

---

## Create Agent

Repo agents are retired (plan 009). A request for a new agent in this repo produces a skill reference, or a new skill when no skill fits. Work runs in the main thread; when dispatch pays, a fresh `general-purpose` worker loads the skill and the reference.

Phases: **DISCOVER -> DESIGN -> SCAFFOLD -> REGISTER -> VALIDATE**

1. **Discover.** Find the owning skill: `grep -ril "<domain>" skills/*/*/SKILL.md skills/*/*/references/`. If a skill reference already covers the domain, extend it instead of adding a new one.
2. **Design.** Decide the owning skill (or a new skill), reference slug, the Reference Loading Table signals that load it (3-6 specific phrases), allowed tools for a worker that loads it, depth files under `references/<slug>/`, and activation cases.
3. **Scaffold.** Write `skills/<category>/<skill>/references/<slug>.md` (depth in `references/<slug>/`) and add its row to the skill's Reference Loading Table. For a new skill, use Create Skill. Follow `docs/PHILOSOPHY.md` for structure.
4. **Register.** Run `python3 scripts/generate-skill-index.py` when skill frontmatter changed.
5. **Validate.** Run `python3 scripts/validate-references.py --check-placeholders`, `--check-size`, and `--check-skill-calls`. Test activation with the 3+2+2 prompt set against the loading-table signals.

**Private overlay agent (owner request only).** When the owner asks for an agent file, write it in `private-agents/`, use `references/agent-creator/agent-frontmatter-template.md`, index it into `agents/INDEX.local.json` with `python3 scripts/generate-agent-index.py --include-private`, and validate it with `python3 scripts/validate-references.py --agent <name> --agents-dir private-agents`.

Load `references/agent-creator.md` for full phases. Deep references in `references/agent-creator/` cover design patterns, frontmatter template (overlay agents), eval design.

---

## Uplift for Weaker Models

Improve a skill, agent, or shared guide until a weaker model produces strong output with it. Load `references/weak-model-uplift.md` and follow its steps:

1. **Pick the target from data.** Query `~/.claude/learning/usage.db` and `learning.db` for heavily used or failing skills.
2. **Build tasks and checks first.** 4–8 tasks plus 1–2 held-out tasks; deterministic checks and a yes/no rubric written before any run.
3. **Run the arms.** No guidance and current guidance, two samples per task minimum, with `python3 scripts/weak_model_run.py`.
4. **Score and look.** Checks, rubric, your own review of every artifact, optional Jev questions on extracted facts.
5. **Turn failures into rules.** Concrete values, before/after examples, runnable checks; delete stale instructions; examples from unrelated products.
6. **Rerun** the guided arm and held-out tasks; stop when gains flatten or after three rounds.
7. **Report and ship** a per-round table with held-out results, cost, and caveats in the PR body.

---

## Compare Agents

Controlled benchmarks comparing agent variants on identical tasks.

1. **Select variants.** Identify the agents to compare (2-4 variants).
2. **Design benchmark.** Load `references/agent-comparison/benchmark-tasks.md`. Select 5-10 representative tasks covering the agent's domain. When the result decides a ship, also load `references/blind-ab-tests.md` and pass its difficulty pilot before the main run.
3. **Execute.** Run each task with each variant. Collect: output quality, token usage, tool calls, time.
4. **Grade.** Apply rubric from `references/agent-comparison/grading-rubric.md`. Score each dimension.
5. **Report.** Use `references/agent-comparison/report-template.md`. Include: methodology, per-task scores, aggregate rankings, cost analysis, recommendation.
6. **Optimize.** Load `references/agent-comparison/optimize-phase.md` to improve the winning variant further.

Load `references/agent-comparison.md` for the full methodology.

---

## Evaluate Agent

Static structural and standards-compliance grading with a 90-point deterministic scorer.

1. **Read the agent file.** Extract frontmatter, body sections, reference files.
2. **Score.** Apply rubric from `references/agent-evaluation/scoring-rubric.md`. Categories: identity (15 pts), expertise (20 pts), routing (15 pts), references (15 pts), workflow (15 pts), standards (10 pts).
3. **Report.** Use `references/agent-evaluation/report-templates.md`. Include: per-category scores, specific findings, improvement recommendations.
4. **Batch mode.** For multiple agents: `references/agent-evaluation/batch-evaluation.md`.

Load `references/agent-evaluation.md` for the full methodology.

---

## Compose Skills

DAG-based multi-skill orchestration with dependency resolution.

1. **Define the DAG.** List skills in execution order. Identify dependencies (skill B needs output from skill A).
2. **Check compatibility.** Load `references/skill-composer/compatibility-matrix.md`. Verify input/output contracts between skills.
3. **Build the pipeline.** Load `references/skill-composer/composition-patterns.md` for orchestration patterns (serial, parallel, fan-out, conditional).
4. **Execute.** Run skills in DAG order. Pass outputs between skills via the defined contracts.
5. **Validate.** Check all skills completed. Verify final output meets the composite goal.

Load `references/skill-composer.md` for the full methodology. See `references/skill-composer/examples.md` for worked examples.

Scripts: `scripts/skill-composer/`

---

## Update Routing

5-phase pipeline: SCAN -> EXTRACT -> GENERATE -> UPDATE -> VERIFY.

1. **SCAN.** Run `python3 scripts/generate-skill-index.py` to discover all skills and agents.
2. **EXTRACT.** Parse frontmatter from each SKILL.md and agent file. Extract triggers, description, category, complexity.
3. **GENERATE.** Build `skills/INDEX.json` and `agents/INDEX.json`.
4. **UPDATE.** Write index files. PostToolUse hooks auto-regenerate on individual edits; this covers bulk changes and drift.
5. **VERIFY.** Compare generated index against discovered files. Report missing entries, conflicts, or stale entries.

Load `references/routing-table-updater.md` for full phases. Deep references in `references/routing-table-updater/` cover routing format, extraction patterns, conflict resolution, batch mode.

---

## Evolve Toolkit

7-phase pipeline: DISCOVER -> DIAGNOSE -> PROPOSE -> CRITIQUE -> BUILD -> VALIDATE -> EVOLVE.

1. **DISCOVER.** Audit recent sessions for routing failures, skill gaps, agent weaknesses, user friction.
2. **DIAGNOSE.** Load `references/toolkit-evolution/diagnose-scripts.md`. Run gap analysis scripts. Identify patterns.
3. **PROPOSE.** Generate 3-5 improvement proposals with expected impact, effort, risk.
4. **CRITIQUE.** Apply multi-perspective review to proposals.
5. **BUILD.** Implement the approved proposals using the appropriate mode above (create skill, create agent, etc.).
6. **VALIDATE.** Run tests and validators on new/changed components.
7. **EVOLVE.** Update evolution history at `references/toolkit-evolution/evolution-history.md`.

Load `references/toolkit-evolution.md` for the full pipeline.

---

## Generate CLAUDE.md

4-phase pipeline: SCAN -> DETECT -> GENERATE -> VALIDATE.

1. **SCAN.** Check for existing CLAUDE.md. If present, write to `CLAUDE.md.generated` for comparison. Detect language, framework, build system from repo files.
2. **DETECT.** Identify domain enrichment opportunities. Load `references/generate-claudemd/examples-and-errors.md` for language-specific patterns.
3. **GENERATE.** Load template from `references/generate-claudemd/CLAUDEMD_TEMPLATE.md`. Fill sections: overview, commands, architecture, conventions, testing, deployment.
4. **VALIDATE.** Run all documented commands. Verify paths exist. Check for secrets in output.

Optional modes: subdirectory CLAUDE.md for monorepos; minimal mode (overview + commands + architecture only).

---

## Dispatching a Toolkit Worker

Repo agents are retired. When a toolkit task goes to a worker (a hook change in an upgrade, a parallel governance sweep), dispatch a fresh `general-purpose` agent whose prompt starts: "Read ~/.claude/skills/toolkit/references/<file> and follow it; load every file its Reference Loading Table matches."

## Deep References

Load when the task needs detailed schemas, templates, or methodology.

| Mode | Key References |
|------|---------------|
| Skill Creator | `references/skill-creator.md`, `references/skill-creator/{skill-template,progressive-disclosure,complexity-tiers,error-catalog,enrichment-workflow}.md` |
| Agent Creator | `references/agent-creator.md`, `references/agent-creator/{agent-design-patterns,agent-frontmatter-template,agent-eval-design}.md` |
| Weak-Model Uplift | `references/weak-model-uplift.md` |
| Agent Comparison | `references/agent-comparison.md`, `references/agent-comparison/{methodology,grading-rubric,benchmark-tasks,report-template,optimize-phase}.md` |
| Blind A/B Tests | `references/blind-ab-tests.md`: difficulty pilot, real-bug corpus, arm parity, grader validation, position bias, power, pre-registration, recompute audit |
| Agent Evaluation | `references/agent-evaluation.md`, `references/agent-evaluation/{scoring-rubric,report-templates,batch-evaluation}.md` |
| Skill Composer | `references/skill-composer.md`, `references/skill-composer/{compatibility-matrix,composition-patterns,skill-patterns,examples}.md` |
| Routing Tables | `references/routing-table-updater.md`, `references/routing-table-updater/{routing-format,extraction-patterns,conflict-resolution,examples}.md` |
| Toolkit Evolution | `references/toolkit-evolution.md`, `references/toolkit-evolution/{diagnose-scripts,evolution-history,evolve-preferred-patterns}.md` |
| Generate CLAUDE.md | `references/generate-claudemd.md`, `references/generate-claudemd/{CLAUDEMD_TEMPLATE,examples-and-errors}.md` |
| Governance | `references/toolkit-governance.md`, `references/toolkit-governance/{frontmatter-compliance,routing-table-patterns,adr-lifecycle,hook-standardization}.md` |
| System Upgrade | `references/system-upgrade.md`, `references/system-upgrade/{upgrade-signal-parsing,component-audit-checklists,upgrade-failure-modes}.md` |
| Hook Development | `references/hook-development.md`, `references/hook-development/{architecture,preferred-patterns,code-examples,telemetry-database}.md` |
| GitHub Profile Rules | `references/github-profile-rules.md`, `references/github-profile-rules/{rule-categories,github-api-patterns}.md` |

## Scripts and Agents

| Mode | Scripts | Agents |
|------|---------|--------|
| Skill Creator | `scripts/skill-creator/` | `agents/skill-creator/` |
| Skill Composer | `scripts/skill-composer/` | -- |
| Weak-Model Uplift | `scripts/weak_model_run.py` (repo root) | -- |
| Routing Tables | `scripts/routing-table-updater/` | -- |
| Agent Comparison | `scripts/agent-comparison/` | -- |
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall