notque/vexjoy-agent / skills/meta/toolkit
toolkit
VerifiedToolkit management: create and evaluate skills and agents, manage routing tables, generate Claude.md.
SKILL.md
View on GitHub---
name: toolkit
description: "Toolkit management: create and evaluate skills and agents, manage routing tables, generate Claude.md."
user-invocable: true
allowed-tools:
- Read
- Write
- Edit
- Bash
- Glob
- Grep
- Agent
- Task
- Skill
routing:
force_route: true
not_for: "application code (use workflow), code review (use review)"
triggers:
- "create skill"
- "create agent"
- "scaffold skill"
- "scaffold agent"
- "new skill"
- "new agent"
- "skill template"
- "agent template"
- "eval skill"
- "evaluate agent"
- "benchmark skill"
- "benchmark agents"
- "compare agents"
- "A/B test agents"
- "optimize description"
- "evolve toolkit"
- "toolkit evolution"
- "self-improve"
- "update routing tables"
- "routing maintenance"
- "generate claude.md"
- "create claude.md"
- "compose skills"
- "bake-off"
- "skill quality"
- "weaker model"
- "weak model uplift"
- "improve skill for weaker models"
- "edit skill"
- "routing table governance"
- "ADR management"
- "toolkit maintenance"
- "check coverage"
- "skill compliance"
- "hook standardization"
- "cross-component consistency"
- "upgrade agents"
- "system upgrade"
- "claude update"
- "upgrade skills"
- "adapt workflow"
- "apply claude update"
- "apply update"
- "system health"
- "update system"
- "new claude version"
- "apply retro"
- "create hook"
- "hook development"
- "event handler"
- "PostToolUse"
- "PreToolUse"
- "SessionStart"
- "routing telemetry hook"
- "hook registration"
- "github rules"
- "profile analysis"
- "coding style extraction"
- "github conventions"
- "programming rules"
category: meta-tooling
pairs_with:
- review
- workflow
- assessment
- docs-sync-checker
- testing
- pr-workflow
---
# Toolkit
Thirteen modes covering the full toolkit lifecycle: creating and improving skills
and agents; evaluating agents; maintaining routing tables; generating CLAUDE.md;
composing multi-skill DAGs; running the evolution loop; governance; system upgrades; hook development; and GitHub profile rule extraction. Classify the request
and follow the matching section.
## Mode Selection
| Mode | Signals | Section |
|------|---------|---------|
| **Skill Creator** | create skill, scaffold skill, new skill, build a skill | Create Skill |
| **Agent Creator** | create agent, scaffold agent, new agent | Create Agent |
| **Weak-Model Uplift** | weaker model, uplift skill, make a skill work for Opus 4.6, improve guidance from generated output | Uplift for Weaker Models |
| **Agent Comparison** | compare agents, A/B test agents, benchmark agents, benchmark skill, bake-off | Compare Agents |
| **Agent Evaluation** | evaluate agent quality, audit agent, grade agent, eval skill | Evaluate Agent |
| **Skill Composer** | compose skills, DAG orchestration, skill pipeline | Compose Skills |
| **Routing Tables** | update routing tables, sync routing, routing drift | Update Routing |
| **Toolkit Evolution** | evolve toolkit, self-improve, discover gaps | Evolve Toolkit |
| **Generate CLAUDE.md** | generate claude.md, create claude.md, init | Generate CLAUDE.md |
| **Governance** | edit skill, routing governance, ADR lifecycle, hook standardization, consistency check | Load `references/toolkit-governance.md` |
| **System Upgrade** | claude update, upgrade skills, apply update, apply retro | Load `references/system-upgrade.md` |
| **Hook Development** | create hook, PreToolUse, PostToolUse, SessionStart, hook registration | Load `references/hook-development.md` |
| **GitHub Profile Rules** | github rules, profile analysis, coding style extraction | Load `references/github-profile-rules.md` |
---
## Create Skill
Phases: **INTENT -> DRAFT -> TEST -> REGISTER**
1. **Capture intent.** What should the skill do? When should it trigger? What output? Are outputs objectively verifiable (code, data) or subjective (writing, design)?
2. **Duplicate check.** Run `grep -i "<domain>" skills/*/SKILL.md` to check existing coverage. If an umbrella skill covers the domain, add a reference file instead.
3. **Write SKILL.md.** Follow `references/skill-creator/skill-template.md` for frontmatter structure. Apply Dense-Complete Writing standard. Frontmatter must include: name, description, routing (triggers, not_for, category, pairs_with), allowed-tools.
4. **Test.** Try 3 should-trigger, 2 should-not-trigger, and 2 near-miss prompts with the skill loaded. Revise the SKILL.md until routing and output are right.
5. **Register.** Run `python3 scripts/generate-skill-index.py` to update routing.
Load `references/skill-creator.md` for the full workflow. Deep references in `references/skill-creator/` cover progressive disclosure, artifact schemas, complexity tiers, error catalog, enrichment workflow, and more.
Scripts: `scripts/skill-creator/`
---
## Create Agent
Repo agents are retired (plan 009). A request for a new agent in this repo produces a skill reference, or a new skill when no skill fits. Work runs in the main thread; when dispatch pays, a fresh `general-purpose` worker loads the skill and the reference.
Phases: **DISCOVER -> DESIGN -> SCAFFOLD -> REGISTER -> VALIDATE**
1. **Discover.** Find the owning skill: `grep -ril "<domain>" skills/*/*/SKILL.md skills/*/*/references/`. If a skill reference already covers the domain, extend it instead of adding a new one.
2. **Design.** Decide the owning skill (or a new skill), reference slug, the Reference Loading Table signals that load it (3-6 specific phrases), allowed tools for a worker that loads it, depth files under `references/<slug>/`, and activation cases.
3. **Scaffold.** Write `skills/<category>/<skill>/references/<slug>.md` (depth in `references/<slug>/`) and add its row to the skill's Reference Loading Table. For a new skill, use Create Skill. Follow `docs/PHILOSOPHY.md` for structure.
4. **Register.** Run `python3 scripts/generate-skill-index.py` when skill frontmatter changed.
5. **Validate.** Run `python3 scripts/validate-references.py --check-placeholders`, `--check-size`, and `--check-skill-calls`. Test activation with the 3+2+2 prompt set against the loading-table signals.
**Private overlay agent (owner request only).** When the owner asks for an agent file, write it in `private-agents/`, use `references/agent-creator/agent-frontmatter-template.md`, index it into `agents/INDEX.local.json` with `python3 scripts/generate-agent-index.py --include-private`, and validate it with `python3 scripts/validate-references.py --agent <name> --agents-dir private-agents`.
Load `references/agent-creator.md` for full phases. Deep references in `references/agent-creator/` cover design patterns, frontmatter template (overlay agents), eval design.
---
## Uplift for Weaker Models
Improve a skill, agent, or shared guide until a weaker model produces strong output with it. Load `references/weak-model-uplift.md` and follow its steps:
1. **Pick the target from data.** Query `~/.claude/learning/usage.db` and `learning.db` for heavily used or failing skills.
2. **Build tasks and checks first.** 4–8 tasks plus 1–2 held-out tasks; deterministic checks and a yes/no rubric written before any run.
3. **Run the arms.** No guidance and current guidance, two samples per task minimum, with `python3 scripts/weak_model_run.py`.
4. **Score and look.** Checks, rubric, your own review of every artifact, optional Jev questions on extracted facts.
5. **Turn failures into rules.** Concrete values, before/after examples, runnable checks; delete stale instructions; examples from unrelated products.
6. **Rerun** the guided arm and held-out tasks; stop when gains flatten or after three rounds.
7. **Report and ship** a per-round table with held-out results, cost, and caveats in the PR body.
---
## Compare Agents
Controlled benchmarks comparing agent variants on identical tasks.
1. **Select variants.** Identify the agents to compare (2-4 variants).
2. **Design benchmark.** Load `references/agent-comparison/benchmark-tasks.md`. Select 5-10 representative tasks covering the agent's domain. When the result decides a ship, also load `references/blind-ab-tests.md` and pass its difficulty pilot before the main run.
3. **Execute.** Run each task with each variant. Collect: output quality, token usage, tool calls, time.
4. **Grade.** Apply rubric from `references/agent-comparison/grading-rubric.md`. Score each dimension.
5. **Report.** Use `references/agent-comparison/report-template.md`. Include: methodology, per-task scores, aggregate rankings, cost analysis, recommendation.
6. **Optimize.** Load `references/agent-comparison/optimize-phase.md` to improve the winning variant further.
Load `references/agent-comparison.md` for the full methodology.
---
## Evaluate Agent
Static structural and standards-compliance grading with a 90-point deterministic scorer.
1. **Read the agent file.** Extract frontmatter, body sections, reference files.
2. **Score.** Apply rubric from `references/agent-evaluation/scoring-rubric.md`. Categories: identity (15 pts), expertise (20 pts), routing (15 pts), references (15 pts), workflow (15 pts), standards (10 pts).
3. **Report.** Use `references/agent-evaluation/report-templates.md`. Include: per-category scores, specific findings, improvement recommendations.
4. **Batch mode.** For multiple agents: `references/agent-evaluation/batch-evaluation.md`.
Load `references/agent-evaluation.md` for the full methodology.
---
## Compose Skills
DAG-based multi-skill orchestration with dependency resolution.
1. **Define the DAG.** List skills in execution order. Identify dependencies (skill B needs output from skill A).
2. **Check compatibility.** Load `references/skill-composer/compatibility-matrix.md`. Verify input/output contracts between skills.
3. **Build the pipeline.** Load `references/skill-composer/composition-patterns.md` for orchestration patterns (serial, parallel, fan-out, conditional).
4. **Execute.** Run skills in DAG order. Pass outputs between skills via the defined contracts.
5. **Validate.** Check all skills completed. Verify final output meets the composite goal.
Load `references/skill-composer.md` for the full methodology. See `references/skill-composer/examples.md` for worked examples.
Scripts: `scripts/skill-composer/`
---
## Update Routing
5-phase pipeline: SCAN -> EXTRACT -> GENERATE -> UPDATE -> VERIFY.
1. **SCAN.** Run `python3 scripts/generate-skill-index.py` to discover all skills and agents.
2. **EXTRACT.** Parse frontmatter from each SKILL.md and agent file. Extract triggers, description, category, complexity.
3. **GENERATE.** Build `skills/INDEX.json` and `agents/INDEX.json`.
4. **UPDATE.** Write index files. PostToolUse hooks auto-regenerate on individual edits; this covers bulk changes and drift.
5. **VERIFY.** Compare generated index against discovered files. Report missing entries, conflicts, or stale entries.
Load `references/routing-table-updater.md` for full phases. Deep references in `references/routing-table-updater/` cover routing format, extraction patterns, conflict resolution, batch mode.
---
## Evolve Toolkit
7-phase pipeline: DISCOVER -> DIAGNOSE -> PROPOSE -> CRITIQUE -> BUILD -> VALIDATE -> EVOLVE.
1. **DISCOVER.** Audit recent sessions for routing failures, skill gaps, agent weaknesses, user friction.
2. **DIAGNOSE.** Load `references/toolkit-evolution/diagnose-scripts.md`. Run gap analysis scripts. Identify patterns.
3. **PROPOSE.** Generate 3-5 improvement proposals with expected impact, effort, risk.
4. **CRITIQUE.** Apply multi-perspective review to proposals.
5. **BUILD.** Implement the approved proposals using the appropriate mode above (create skill, create agent, etc.).
6. **VALIDATE.** Run tests and validators on new/changed components.
7. **EVOLVE.** Update evolution history at `references/toolkit-evolution/evolution-history.md`.
Load `references/toolkit-evolution.md` for the full pipeline.
---
## Generate CLAUDE.md
4-phase pipeline: SCAN -> DETECT -> GENERATE -> VALIDATE.
1. **SCAN.** Check for existing CLAUDE.md. If present, write to `CLAUDE.md.generated` for comparison. Detect language, framework, build system from repo files.
2. **DETECT.** Identify domain enrichment opportunities. Load `references/generate-claudemd/examples-and-errors.md` for language-specific patterns.
3. **GENERATE.** Load template from `references/generate-claudemd/CLAUDEMD_TEMPLATE.md`. Fill sections: overview, commands, architecture, conventions, testing, deployment.
4. **VALIDATE.** Run all documented commands. Verify paths exist. Check for secrets in output.
Optional modes: subdirectory CLAUDE.md for monorepos; minimal mode (overview + commands + architecture only).
---
## Dispatching a Toolkit Worker
Repo agents are retired. When a toolkit task goes to a worker (a hook change in an upgrade, a parallel governance sweep), dispatch a fresh `general-purpose` agent whose prompt starts: "Read ~/.claude/skills/toolkit/references/<file> and follow it; load every file its Reference Loading Table matches."
## Deep References
Load when the task needs detailed schemas, templates, or methodology.
| Mode | Key References |
|------|---------------|
| Skill Creator | `references/skill-creator.md`, `references/skill-creator/{skill-template,progressive-disclosure,complexity-tiers,error-catalog,enrichment-workflow}.md` |
| Agent Creator | `references/agent-creator.md`, `references/agent-creator/{agent-design-patterns,agent-frontmatter-template,agent-eval-design}.md` |
| Weak-Model Uplift | `references/weak-model-uplift.md` |
| Agent Comparison | `references/agent-comparison.md`, `references/agent-comparison/{methodology,grading-rubric,benchmark-tasks,report-template,optimize-phase}.md` |
| Blind A/B Tests | `references/blind-ab-tests.md`: difficulty pilot, real-bug corpus, arm parity, grader validation, position bias, power, pre-registration, recompute audit |
| Agent Evaluation | `references/agent-evaluation.md`, `references/agent-evaluation/{scoring-rubric,report-templates,batch-evaluation}.md` |
| Skill Composer | `references/skill-composer.md`, `references/skill-composer/{compatibility-matrix,composition-patterns,skill-patterns,examples}.md` |
| Routing Tables | `references/routing-table-updater.md`, `references/routing-table-updater/{routing-format,extraction-patterns,conflict-resolution,examples}.md` |
| Toolkit Evolution | `references/toolkit-evolution.md`, `references/toolkit-evolution/{diagnose-scripts,evolution-history,evolve-preferred-patterns}.md` |
| Generate CLAUDE.md | `references/generate-claudemd.md`, `references/generate-claudemd/{CLAUDEMD_TEMPLATE,examples-and-errors}.md` |
| Governance | `references/toolkit-governance.md`, `references/toolkit-governance/{frontmatter-compliance,routing-table-patterns,adr-lifecycle,hook-standardization}.md` |
| System Upgrade | `references/system-upgrade.md`, `references/system-upgrade/{upgrade-signal-parsing,component-audit-checklists,upgrade-failure-modes}.md` |
| Hook Development | `references/hook-development.md`, `references/hook-development/{architecture,preferred-patterns,code-examples,telemetry-database}.md` |
| GitHub Profile Rules | `references/github-profile-rules.md`, `references/github-profile-rules/{rule-categories,github-api-patterns}.md` |
## Scripts and Agents
| Mode | Scripts | Agents |
|------|---------|--------|
| Skill Creator | `scripts/skill-creator/` | `agents/skill-creator/` |
| Skill Composer | `scripts/skill-composer/` | -- |
| Weak-Model Uplift | `scripts/weak_model_run.py` (repo root) | -- |
| Routing Tables | `scripts/routing-table-updater/` | -- |
| Agent Comparison | `scripts/agent-comparison/` | -- |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |