Skip to content

notque/vexjoy-agent / skills/meta/toolkit

toolkit

Verified

Toolkit management: create and evaluate skills and agents, manage routing tables, generate Claude.md.

FileKindSize
SKILL.mdskill md15.5 KB
agents/skill-creator/analyzer.mdmarkdown4.8 KB
agents/skill-creator/comparator.mdmarkdown4.7 KB
agents/skill-creator/grader.mdmarkdown3.9 KB
assets/skill-creator/eval_viewer.htmlother57.1 KB
references/agent-comparison.mdmarkdown11.5 KB
references/agent-comparison/benchmark-tasks.mdmarkdown6.8 KB
references/agent-comparison/do-creation-compliance-tasks.jsonconfig8.5 KB
references/agent-comparison/examples-and-errors.mdmarkdown1.0 KB
references/agent-comparison/grading-rubric.mdmarkdown4.3 KB
references/agent-comparison/methodology.mdmarkdown7.4 KB
references/agent-comparison/optimization-guide.mdmarkdown12.0 KB
references/agent-comparison/optimization-tasks.example.jsonconfig853 B
references/agent-comparison/optimize-phase.mdmarkdown9.3 KB
references/agent-comparison/read-only-ops-short-tasks.jsonconfig375 B
references/agent-comparison/report-template.mdmarkdown3.8 KB
references/agent-comparison/socratic-debugging-body-short-tasks.jsonconfig372 B
references/agent-comparison/socratic-debugging-trigger-tasks.jsonconfig2.8 KB
references/agent-creator.mdmarkdown10.9 KB
references/agent-creator/agent-design-patterns.mdmarkdown12.4 KB
references/agent-creator/agent-eval-design.mdmarkdown7.7 KB
references/agent-creator/agent-frontmatter-template.mdmarkdown9.3 KB
references/agent-evaluation.mdmarkdown10.6 KB
references/agent-evaluation/batch-evaluation.mdmarkdown1.6 KB
references/agent-evaluation/common-issues.mdmarkdown2.1 KB
references/agent-evaluation/report-templates.mdmarkdown2.2 KB
references/agent-evaluation/scoring-rubric.mdmarkdown2.4 KB
references/blind-ab-tests.mdmarkdown7.8 KB
references/generate-claudemd.mdmarkdown11.0 KB
references/generate-claudemd/CLAUDEMD_TEMPLATE.mdmarkdown4.1 KB
references/generate-claudemd/examples-and-errors.mdmarkdown7.4 KB
references/github-profile-rules.mdmarkdown7.5 KB
references/github-profile-rules/github-api-patterns.mdmarkdown8.2 KB
references/github-profile-rules/rule-categories.mdmarkdown4.8 KB
references/hook-development.mdmarkdown10.9 KB
references/hook-development/architecture.mdmarkdown3.7 KB
references/hook-development/code-examples.mdmarkdown9.4 KB
references/hook-development/preferred-patterns.mdmarkdown6.5 KB
references/hook-development/telemetry-database.mdmarkdown6.5 KB
references/routing-table-updater.mdmarkdown10.2 KB
references/routing-table-updater/batch-mode.mdmarkdown2.1 KB
references/routing-table-updater/conflict-resolution.mdmarkdown3.8 KB
references/routing-table-updater/error-handling.mdmarkdown1.9 KB
references/routing-table-updater/examples.mdmarkdown4.8 KB
references/routing-table-updater/extraction-patterns.mdmarkdown2.2 KB
references/routing-table-updater/routing-format.mdmarkdown3.6 KB
references/routing-table-updater/skill-examples.mdmarkdown2.7 KB
references/skill-composer.mdmarkdown9.1 KB
references/skill-composer/compatibility-matrix.mdmarkdown13.0 KB
references/skill-composer/composition-patterns.mdmarkdown4.9 KB
references/skill-composer/examples.mdmarkdown17.1 KB
references/skill-composer/skill-patterns.mdmarkdown12.5 KB
references/skill-creator.mdmarkdown28.9 KB
references/skill-creator/agent-template.mdmarkdown18.5 KB
references/skill-creator/artifact-schemas.mdmarkdown8.2 KB
references/skill-creator/bundled-components.mdmarkdown2.3 KB
references/skill-creator/complexity-tiers.mdmarkdown8.0 KB
references/skill-creator/domain-research-targets.mdmarkdown13.2 KB
references/skill-creator/enrichment-workflow.mdmarkdown9.9 KB
references/skill-creator/error-catalog.mdmarkdown9.9 KB
references/skill-creator/preferred-patterns.mdmarkdown11.7 KB
references/skill-creator/progressive-disclosure.mdmarkdown8.5 KB
references/skill-creator/skill-template.mdmarkdown14.6 KB
references/skill-creator/workflow-patterns.mdmarkdown7.6 KB
references/system-upgrade.mdmarkdown11.0 KB
references/system-upgrade/component-audit-checklists.mdmarkdown8.4 KB
references/system-upgrade/upgrade-failure-modes.mdmarkdown8.7 KB
references/system-upgrade/upgrade-signal-parsing.mdmarkdown7.3 KB
references/toolkit-evolution.mdmarkdown14.5 KB
references/toolkit-evolution/diagnose-scripts.mdmarkdown9.5 KB
references/toolkit-evolution/evolution-history.mdmarkdown5.5 KB
references/toolkit-evolution/evolution-report-template.mdmarkdown716 B
references/toolkit-evolution/evolve-preferred-patterns.mdmarkdown6.9 KB
references/toolkit-evolution/evolve-scripts.mdmarkdown3.7 KB
references/toolkit-governance.mdmarkdown16.0 KB
references/toolkit-governance/adr-lifecycle.mdmarkdown10.9 KB
references/toolkit-governance/frontmatter-compliance.mdmarkdown11.1 KB
references/toolkit-governance/hook-standardization.mdmarkdown21.5 KB
references/toolkit-governance/routing-table-patterns.mdmarkdown11.6 KB
references/weak-model-uplift.mdmarkdown10.3 KB
scripts/agent-comparison/compare.pyscript2.3 KB
scripts/agent-comparison/generate_variant.pyscript19.6 KB
scripts/agent-comparison/optimize_loop.pyscript89.3 KB
scripts/agent-comparison/trigger_eval.pyscript19.5 KB
scripts/routing-table-updater/extract_metadata.pyscript10.8 KB
scripts/routing-table-updater/generate_routes.pyscript10.5 KB
scripts/routing-table-updater/scan.pyscript4.7 KB
scripts/routing-table-updater/update_routing.pyscript12.4 KB
scripts/routing-table-updater/validate.pyscript7.3 KB
scripts/skill-composer/build_dag.pyscript12.5 KB
scripts/skill-composer/discover_skills.pyscript12.1 KB
scripts/skill-composer/validate.pyscript12.3 KB
scripts/skill-creator/aggregate_benchmark.pyscript10.0 KB
scripts/skill-creator/eval_compare.pyscript10.2 KB
scripts/skill-creator/optimize_description.pyscript12.5 KB
scripts/skill-creator/package_results.pyscript7.8 KB
scripts/skill-creator/run_eval.pyscript6.3 KB

references/toolkit-governance.md

View on GitHub
# Toolkit governance

Load this reference when you edit a skill, update routing, manage an ADR's lifecycle, audit hook conventions, or check cross-component consistency. Maintain toolkit policy, routing consistency, and ADR conformance. Audit v2.0 frontmatter fields (`name`, `version`, `description`, `routing`, `allowed-tools`), complexity tiers, naming, and progressive disclosure. Hook conventions cover SessionStart, UserPromptSubmit, PostToolUse, PreCompact, Stop, timeouts, and exit code 0 error handling.

## Mandatory Pre-Action Protocol

Before editing, read the applicable sources below. Reuse them while unchanged; reload after a source change or a material scope change:

1. **`docs/PHILOSOPHY.md`** — The project's design philosophy. Every edit must align with these principles: deterministic over LLM execution, handyman principle (context is scarce), specialist selection over generalism, progressive disclosure, anti-rationalization as infrastructure.
2. **The file being edited** — Read the full file before making changes. Understand its current structure, conventions, and purpose before touching it.

## Rules

### Hardcoded Behaviors (Always Apply)

- **Philosophy-First Editing**: Every modification must be defensible against `docs/PHILOSOPHY.md`. If an edit violates a principle (e.g., adding verbose content to a main file instead of references/, bypassing a phase gate), reject or restructure the edit.
- **Read Before Write**: Always read a file before editing it. Always verify file contents rather than relying on naming or memory.
- **Preserve Existing Structure**: When editing SKILL.md files, maintain the existing phase numbering, gate format, and section ordering unless explicitly asked to restructure.
- **Frontmatter Integrity**: Preserve YAML frontmatter integrity at all times. Validate that `---` delimiters are present, required fields exist, and values parse correctly.
- **ADRs Are Local Working Documents**: Keep ADRs as local working artifacts; they stay uncommitted. They are for decision tracking only.
- **Tool Lists Are Advisory**: Claude Code reads `tools:` for subagents and ignores `allowed-tools:`, so an agent's `allowed-tools` documents intended tool use and restricts nothing. When editing agent frontmatter, keep the list matched to the role (reviewers read-only tools, code modifiers full access, orchestrators Read + Agent + Bash), keep read-only rules in the agent body, and never describe the list as enforced.

### Default Behaviors (ON unless disabled)

- **Validation After Edit**: Review the complete diff for lost rules and unintended changes. Parse changed frontmatter and validate affected references after each coherent edit batch; reuse results while inputs remain unchanged. Line counts alone do not prove completeness.
- **Routing Consistency Check**: When updating routing tables, verify that every agent/skill referenced in the table actually exists in the filesystem.
- **Coverage Reporting**: When running INDEX.json operations, report coverage statistics (registered vs total components) and list any unregistered components.

### Optional Behaviors (OFF unless enabled)

- **Full Audit Mode**: Scan ALL agents and skills for compliance issues, not just the ones being edited. Enable for toolkit-wide consistency sweeps.
- **Verbose Diff Output**: Show full unified diffs for every edit. Enable for review-heavy sessions.
- **ADR Consultation Orchestration**: When managing ADRs, dispatch fresh `general-purpose` consultation workers to challenge the decision before status transitions. Enable for consequential architectural decisions.

## Capabilities & Limitations

### Out of Scope
- **Write Go/Python/TypeScript application code** — the `programming` and `frontend` skills cover application development
- **Create brand-new agents or skills from scratch** — toolkit skill handles new component creation with proper template scaffolding
- **Manage CI/CD or deployment** — the `kubernetes` and `deploy` skills cover build pipelines and deployment
- **Review external pull requests** — the `review` skill handles PR review; its workers load `reviewer-code.md`, `reviewer-system.md`, `reviewer-domain.md`, or `reviewer-perspectives.md`
- **Modify the routing system's core logic** — the /do router's implementation is separate from the routing tables governance manages

When asked to perform an out-of-scope action, explain the limitation and name the skill that covers it.

## Reference Loading

Load the relevant reference file before starting any governance task:

| Task Type | Load This Reference | Key Content |
|-----------|--------------------|-|
| Frontmatter audit, `allowed-tools` review, YAML parse errors | `toolkit-governance/frontmatter-compliance.md` | Required fields, advisory tool lists, detection commands |
| Routing table add/update/delete, `pairs_with` validation, INDEX.json | `toolkit-governance/routing-table-patterns.md` | Phantom route detection, trigger conflict checks, index validation |
| ADR status transitions, validation criteria, consultation records | `toolkit-governance/adr-lifecycle.md` | Status line format, transition rules, stale ADR detection commands |
| Hook registration, event types, timeout config, exit code review | `toolkit-governance/hook-standardization.md` | settings.json format, advisory vs blocking exit codes, TTY detection pattern |
| Cross-component consistency sweep | Load all references | Full detection command set |

**Signals that trigger reference loading**:
- Any mention of `allowed-tools`, `frontmatter`, `YAML`, or field compliance → load `frontmatter-compliance.md`
- Any mention of `routing`, `triggers`, `pairs_with`, `INDEX.json`, or phantom routes → load `routing-table-patterns.md`
- Any mention of `ADR`, `status transition`, `Proposed`, `Accepted`, `Implemented`, or `Superseded` → load `adr-lifecycle.md`
- Any mention of `hook`, `settings.json`, `timeout`, `exit code`, `SessionStart`, or `PostToolUse` → load `hook-standardization.md`

---

## Workflow

### Single-File Edit

1. **READ**: Read `docs/PHILOSOPHY.md` and the target file
2. **ANALYZE**: Identify what needs to change and verify it aligns with toolkit principles
3. **EDIT**: Make targeted changes preserving existing structure
4. **VALIDATE**: Re-read file, verify YAML parses, cross-references resolve, no content lost. Run `Grep` to confirm no broken references were introduced.

### Routing Table Update

1. **READ**: Read `docs/PHILOSOPHY.md` and the current routing tables
2. **INVENTORY**: Read frontmatter of each agent/skill being added or modified
3. **DRAFT**: Write entries with intent-based descriptions (what the component does, when to use it, when NOT to use it)
4. **VALIDATE**: Verify every referenced component exists on disk using `Glob` or `ls`

### Cross-Component Consistency Check

1. **SCAN**: Glob for all agents (`agents/*.md`) and skills (`skills/*/*/SKILL.md`, `skills/process/workflow/references/*.md`)
2. **EXTRACT**: Parse YAML frontmatter from each component
3. **CHECK**: Compare against required fields, validate cross-references, check routing coverage
4. **REPORT**: Output compliance summary with specific issues and suggested fixes using the Governance Report format below

### ADR Lifecycle

1. **READ**: Read the ADR file and `docs/PHILOSOPHY.md`
2. **VALIDATE**: Verify the status transition is valid (proposed → accepted → implemented → superseded)
3. **UPDATE**: Modify status, update validation criteria, add consultation notes
4. **VERIFY**: Re-read ADR, confirm changes are correct — keep uncommitted

## Output Format: Governance Report

Use this format for consistency checks, audits, and multi-file operations. Single-file edits report inline.

```markdown
## 1. Scope
[What was checked/modified and why]

## 2. Changes Made
- **[file]**: [what changed] — because [PHILOSOPHY.md principle or governance rule]

## 3. Validation Results
| Check | Result | Evidence |
|-------|--------|----------|
| YAML parses | PASS/FAIL | [tool output or line reference] |
| No content lost | PASS/FAIL | [line count before/after] |
| Cross-refs resolve | PASS/FAIL | [broken links if any] |

## 4. Issues Found (if audit/consistency check)
- **[I1]** [component]: [issue]. Fix: [suggestion].

## 5. VERDICT: [CLEAN / N ISSUES FOUND / BLOCKED]
```

## Reference Loading Table

| Signal | Load These Files | Why |
|---|---|---|
| ADR status transitions, validation criteria, consultation records | [adr-lifecycle.md](toolkit-governance/adr-lifecycle.md) | Status line format, transition rules, stale ADR detection commands |
| frontmatter audit, `allowed-tools` review, YAML parse errors | [frontmatter-compliance.md](toolkit-governance/frontmatter-compliance.md) | Required fields, advisory tool lists, detection commands |
| hook registration, event types, timeout config, exit code review | [hook-standardization.md](toolkit-governance/hook-standardization.md) | settings.json format, advisory vs blocking exit codes |
| routing table edits, `pairs_with` validation, trigger conflicts, INDEX.json | [routing-table-patterns.md](toolkit-governance/routing-table-patterns.md) | Phantom route detection, trigger conflict checks, index validation |
| routing change, measurement change, gate change | [router-ab-runbook.md](../../../../docs/router-ab-runbook.md) | A/B protocol and corpus requirements |
| validator, hook, or CI-gate work | [verify-adversarial-methodology.md](../../../process/testing/references/verify-adversarial-methodology.md) | L1 EXISTS / L3 WIRED / L4 DATA FLOWS verification layers |

## Agent Reference File Validation

Repo agents are retired (plan 009); this applies to private overlay agents. When creating or modifying any agent that has a `references/` directory, run these commands before committing. They cover the structural and orphan-file checks the CI workflow enforces on PR.

```bash
# Structural checks: declared reference files exist, sizes, structure.
python3 scripts/validate-references.py --agent {agent-name}

# Orphans: reference files on disk that no agent declares.
python3 scripts/validate-references.py --all

# Loading-table completeness for skills (signals and rows match files on disk).
# The per-agent pytest cases left with the retired agents (plan 009).
python3 scripts/validate-references.py --check-placeholders
```

Standards enforced:
- Reference files must be <= 500 lines (progressive disclosure budget).
- Framing is joy-checked (no grievance-mode prose in reference bodies).
- The agent body must contain a loading table that maps signals to reference files.

The full spec lives in `skills/meta/do/references/repo-architecture.md`.

## Skill Content Validation

When editing or auditing SKILL.md files, run the content-cleanliness audit alongside the frontmatter and reference validators. It flags non-runtime content (meta-commentary, changelog prose, narration the runtime never reads) that bloats a skill without serving execution.

```bash
# Audit globs <root>/*/SKILL.md — point --root at the skill's category dir.
python3 scripts/audit-skill-content.py --root skills/{category} --severity high
```

Exit 0 with zero high-severity violations is the gate. Drop to `--severity low` to surface every flagged line. This complements `validate-skill-frontmatter.py` (frontmatter) and `validate-references.py` (reference structure + do-framing).

## Error Handling

### Broken YAML Frontmatter
**Cause**: Malformed YAML between `---` delimiters — missing colons, incorrect indentation, unquoted special characters
**Solution**: Read the raw file content, identify the parse error, fix the specific YAML issue. Patch only the broken part of the frontmatter block to preserve the rest and avoid unintended changes.

### Orphaned Cross-References
**Cause**: A routing table entry references an agent or skill file that was renamed or deleted
**Solution**: Glob for the component by partial name to find renames. If deleted, remove the routing entry. Always check both `agents/` and `skills/` directories.

### Stale INDEX.json
**Cause**: Components were added or removed without regenerating the index
**Solution**: Run the index regeneration workflow, then diff the old and new index to report what changed.

### Phase Gate Inconsistency
**Cause**: A skill's phases reference gates that are missing, or gates reference phases that were renumbered
**Solution**: Read the full skill, map phase numbers to gate references, fix numbering to be consistent.

## Preferred Patterns

### Load the Current Philosophy
**What it looks like**: Jumping straight to file edits based on the user's request
**Why wrong**: Edits may violate core principles (progressive disclosure, deterministic execution, specialist separation) — creating technical debt that compounds
**Do instead**: Read `docs/PHILOSOPHY.md` before the task and reload if it changes.

### Rewriting Instead of Patching
**What it looks like**: Rewriting entire sections or files when only a targeted change was needed
**Why wrong**: Risks losing content, breaking cross-references, and introducing unintended changes
**Do instead**: Make minimal, targeted edits. Show before/after for non-trivial changes.

### Routing Table Entry Without Filesystem Verification
**What it looks like**: Adding a routing entry for an agent/skill without verifying the file exists
**Why wrong**: Creates a phantom route — the router selects a component that doesn't exist, causing silent failures
**Do instead**: Always `ls` or `Glob` to verify the referenced file exists before adding a routing entry

### Frontmatter Compliance Without Context
**What it looks like**: Mechanically adding missing fields without understanding the component's purpose
**Why wrong**: Fields like `allowed-tools` and `complexity` depend on what the component does — filling them generically defeats their purpose
**Do instead**: Read the component's body to understand its role, then set fields appropriately

## Anti-Rationalization

| Rationalization Attempt | Why It's Wrong | Required Action |
|------------------------|----------------|-----------------|
| "I know what's in PHILOSOPHY.md" | Memory drifts; the file may have been updated | **Read it before work; reload after changes** |
| "This is a small edit, no need to validate" | Small edits break YAML and cross-references | **Validate each coherent edit batch** |
| "The routing table looks fine" | Visual inspection misses orphaned references | **Verify against filesystem** |
| "ADR status is obvious, just update it" | Status transitions have rules and implications | **Read ADR fully before changing status** |
| "Frontmatter is boilerplate, copy from another agent" | Each component has unique tool needs and routing | **Set fields based on the component's actual role** |
| "I'll fix the cross-references later" | Later rarely arrives; broken links compound | **Fix references in the same edit** |

## Blocker Criteria

Resolve routine issues from repository evidence and existing authorization. Ask only when these situations leave a material decision uncovered:

| Situation | Why Stop | Ask This |
|-----------|----------|----------|
| Edit would change a skill's public interface (phase names, gate criteria) | Downstream consumers may depend on current structure | "This changes the skill's interface — which consumers should I check?" |
| Routing table entry conflicts with existing triggers | Two components claiming the same triggers causes ambiguous routing | "Agent X and Y both trigger on '{keyword}' — which should take priority?" |
| ADR status transition skips a step | May indicate incomplete implementation or review | "ADR is in '{current}' status — should it go through '{intermediate}' first?" |
| Component appears to be deprecated but is still referenced | Removing it may break routing or other components | "This component looks deprecated but is referenced by {list} — safe to remove?" |

## Death Loop Prevention

### Retry Limits
- Maximum 3 attempts for any single edit operation
- If YAML keeps breaking after 3 fixes, show the raw content and ask the user

### Recovery Protocol
1. **Detection**: Validation fails repeatedly on the same file or section
2. **Intervention**: Stop editing, show the current file state, explain what's failing
3. **Prevention**: Read the file fresh (not from memory), identify root cause before attempting another fix
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall