Skip to content

notque/vexjoy-agent / skills/meta/toolkit

toolkit

Verified

Toolkit management: create and evaluate skills and agents, manage routing tables, generate Claude.md.

FileKindSize
SKILL.mdskill md15.5 KB
agents/skill-creator/analyzer.mdmarkdown4.8 KB
agents/skill-creator/comparator.mdmarkdown4.7 KB
agents/skill-creator/grader.mdmarkdown3.9 KB
assets/skill-creator/eval_viewer.htmlother57.1 KB
references/agent-comparison.mdmarkdown11.5 KB
references/agent-comparison/benchmark-tasks.mdmarkdown6.8 KB
references/agent-comparison/do-creation-compliance-tasks.jsonconfig8.5 KB
references/agent-comparison/examples-and-errors.mdmarkdown1.0 KB
references/agent-comparison/grading-rubric.mdmarkdown4.3 KB
references/agent-comparison/methodology.mdmarkdown7.4 KB
references/agent-comparison/optimization-guide.mdmarkdown12.0 KB
references/agent-comparison/optimization-tasks.example.jsonconfig853 B
references/agent-comparison/optimize-phase.mdmarkdown9.3 KB
references/agent-comparison/read-only-ops-short-tasks.jsonconfig375 B
references/agent-comparison/report-template.mdmarkdown3.8 KB
references/agent-comparison/socratic-debugging-body-short-tasks.jsonconfig372 B
references/agent-comparison/socratic-debugging-trigger-tasks.jsonconfig2.8 KB
references/agent-creator.mdmarkdown10.9 KB
references/agent-creator/agent-design-patterns.mdmarkdown12.4 KB
references/agent-creator/agent-eval-design.mdmarkdown7.7 KB
references/agent-creator/agent-frontmatter-template.mdmarkdown9.3 KB
references/agent-evaluation.mdmarkdown10.6 KB
references/agent-evaluation/batch-evaluation.mdmarkdown1.6 KB
references/agent-evaluation/common-issues.mdmarkdown2.1 KB
references/agent-evaluation/report-templates.mdmarkdown2.2 KB
references/agent-evaluation/scoring-rubric.mdmarkdown2.4 KB
references/blind-ab-tests.mdmarkdown7.8 KB
references/generate-claudemd.mdmarkdown11.0 KB
references/generate-claudemd/CLAUDEMD_TEMPLATE.mdmarkdown4.1 KB
references/generate-claudemd/examples-and-errors.mdmarkdown7.4 KB
references/github-profile-rules.mdmarkdown7.5 KB
references/github-profile-rules/github-api-patterns.mdmarkdown8.2 KB
references/github-profile-rules/rule-categories.mdmarkdown4.8 KB
references/hook-development.mdmarkdown10.9 KB
references/hook-development/architecture.mdmarkdown3.7 KB
references/hook-development/code-examples.mdmarkdown9.4 KB
references/hook-development/preferred-patterns.mdmarkdown6.5 KB
references/hook-development/telemetry-database.mdmarkdown6.5 KB
references/routing-table-updater.mdmarkdown10.2 KB
references/routing-table-updater/batch-mode.mdmarkdown2.1 KB
references/routing-table-updater/conflict-resolution.mdmarkdown3.8 KB
references/routing-table-updater/error-handling.mdmarkdown1.9 KB
references/routing-table-updater/examples.mdmarkdown4.8 KB
references/routing-table-updater/extraction-patterns.mdmarkdown2.2 KB
references/routing-table-updater/routing-format.mdmarkdown3.6 KB
references/routing-table-updater/skill-examples.mdmarkdown2.7 KB
references/skill-composer.mdmarkdown9.1 KB
references/skill-composer/compatibility-matrix.mdmarkdown13.0 KB
references/skill-composer/composition-patterns.mdmarkdown4.9 KB
references/skill-composer/examples.mdmarkdown17.1 KB
references/skill-composer/skill-patterns.mdmarkdown12.5 KB
references/skill-creator.mdmarkdown28.9 KB
references/skill-creator/agent-template.mdmarkdown18.5 KB
references/skill-creator/artifact-schemas.mdmarkdown8.2 KB
references/skill-creator/bundled-components.mdmarkdown2.3 KB
references/skill-creator/complexity-tiers.mdmarkdown8.0 KB
references/skill-creator/domain-research-targets.mdmarkdown13.2 KB
references/skill-creator/enrichment-workflow.mdmarkdown9.9 KB
references/skill-creator/error-catalog.mdmarkdown9.9 KB
references/skill-creator/preferred-patterns.mdmarkdown11.7 KB
references/skill-creator/progressive-disclosure.mdmarkdown8.5 KB
references/skill-creator/skill-template.mdmarkdown14.6 KB
references/skill-creator/workflow-patterns.mdmarkdown7.6 KB
references/system-upgrade.mdmarkdown11.0 KB
references/system-upgrade/component-audit-checklists.mdmarkdown8.4 KB
references/system-upgrade/upgrade-failure-modes.mdmarkdown8.7 KB
references/system-upgrade/upgrade-signal-parsing.mdmarkdown7.3 KB
references/toolkit-evolution.mdmarkdown14.5 KB
references/toolkit-evolution/diagnose-scripts.mdmarkdown9.5 KB
references/toolkit-evolution/evolution-history.mdmarkdown5.5 KB
references/toolkit-evolution/evolution-report-template.mdmarkdown716 B
references/toolkit-evolution/evolve-preferred-patterns.mdmarkdown6.9 KB
references/toolkit-evolution/evolve-scripts.mdmarkdown3.7 KB
references/toolkit-governance.mdmarkdown16.0 KB
references/toolkit-governance/adr-lifecycle.mdmarkdown10.9 KB
references/toolkit-governance/frontmatter-compliance.mdmarkdown11.1 KB
references/toolkit-governance/hook-standardization.mdmarkdown21.5 KB
references/toolkit-governance/routing-table-patterns.mdmarkdown11.6 KB
references/weak-model-uplift.mdmarkdown10.3 KB
scripts/agent-comparison/compare.pyscript2.3 KB
scripts/agent-comparison/generate_variant.pyscript19.6 KB
scripts/agent-comparison/optimize_loop.pyscript89.3 KB
scripts/agent-comparison/trigger_eval.pyscript19.5 KB
scripts/routing-table-updater/extract_metadata.pyscript10.8 KB
scripts/routing-table-updater/generate_routes.pyscript10.5 KB
scripts/routing-table-updater/scan.pyscript4.7 KB
scripts/routing-table-updater/update_routing.pyscript12.4 KB
scripts/routing-table-updater/validate.pyscript7.3 KB
scripts/skill-composer/build_dag.pyscript12.5 KB
scripts/skill-composer/discover_skills.pyscript12.1 KB
scripts/skill-composer/validate.pyscript12.3 KB
scripts/skill-creator/aggregate_benchmark.pyscript10.0 KB
scripts/skill-creator/eval_compare.pyscript10.2 KB
scripts/skill-creator/optimize_description.pyscript12.5 KB
scripts/skill-creator/package_results.pyscript7.8 KB
scripts/skill-creator/run_eval.pyscript6.3 KB

references/hook-development.md

View on GitHub
# Hook development

Load this reference when you write or debug a Python hook for the Claude Code event system or its telemetry store. Build event-driven telemetry and governance hooks for Claude Code. Capture dispatches at PostToolUse:Agent. Telemetry uses three-way outcome scoring and SQLite schema/migrations in `hooks/lib/learning_db_v2.py`, with WAL mode and `busy_timeout` on every connection.

## Rules

### Hardcoded Behaviors (Always Apply)
- **Non-Blocking Execution**: Hooks MUST exit with code 0 regardless of internal errors or failures (hard requirement)
- **Sub-50ms Performance**: All hook operations must complete within 50 milliseconds for real-time responsiveness (hard requirement)
- **Atomic File Operations**: File-state updates use write-to-temp-then-rename to prevent corruption; SQLite connections opened in a hook set `PRAGMA busy_timeout` (hard requirement)
- **JSON Safety**: All JSON parsing wrapped in comprehensive error handling with graceful fallbacks
- **Context Injection Pattern**: Solution delivery uses `context_output(EVENT_NAME, text).print_and_exit()` from `hooks/lib/hook_utils.py` — prints JSON to stdout, which Claude Code reads directly
- **Deploy Before Register**: Register a hook in settings.json only after the hook file exists at `~/.claude/hooks/`. Correct order: (1) create file in repo `hooks/`, (2) copy/sync to `~/.claude/hooks/`, (3) verify it runs, (4) THEN register. Reversing this bricks all PreToolUse hooks (Python file-not-found = exit 2 = blocks every tool).
- **Settings via Repo Only**: Edit hook registration through repo-tracked `.claude/settings.json` which syncs via `sync-to-user-claude.py`. Direct edits to `~/.claude/settings.json` can brick the session.
- **Preserve .gitignore**: Keep `.gitignore` unchanged. This file controls repository safety boundaries.
- **Respect Gitignore Boundaries**: Stage only tracked files with `git add` by name. If a file is gitignored, it stays gitignored.

### Default Behaviors (ON unless disabled)
- **Debug Logging**: Write detailed logs to /tmp/claude_hook_debug.log for troubleshooting
- **Outcome Basis Recording**: Record how each outcome was scored alongside the outcome, so a rate can be read against its basis
- **One Marker Per Event**: Emit a lone dispatch marker per event; route-fit scoring reads one marker at a time
- **Telemetry Appends Only**: Record events; leave every skill and reference file to a reviewed human edit

### Verification STOP Blocks
- **After writing a hook**: Run `python3 hooks/{hook-name}.py < /dev/null` and verify exit code 0. A hook that exits non-zero will brick the session.
- **After claiming a fix**: Verify the fix addresses the root cause, not just the symptom. Re-read the original error and confirm it cannot recur.
- **After completing the hook**: Measure execution time (`time python3 hooks/{hook-name}.py < test_event.json`) and verify it is under 50ms. Show the actual timing.
- **Before editing a file**: Read the file first.
- **Before registering in settings.json**: Verify the hook file exists at `~/.claude/hooks/` and runs without error. Registering before deploying deadlocks the session.

### Related Skills

Call the Skill tool with `testing` for hook tests, and with `code-quality` for lint and format gates.

### Optional Behaviors (OFF unless enabled)
- **Extended Timeout Windows**: Allow >50ms execution for complex analysis (violates hard requirement - use cautiously)
- **Memory Profiling**: Enable detailed memory usage tracking and optimization analysis
- **Advanced Analytics**: Generate comprehensive route-health and cohort-delta reports

## Capabilities & Limitations

Design outcome scoring with stacked precision guards on the expensive direction and golden fixtures in both directions. Optimize with lazy loading, efficient JSON processing, minimal allocation, and profiling.

### Out of Scope
- **Modify Claude Code core**: Cannot change Claude Code's hook invocation system or event structure
- **Edit skills or references**: A hook records and gates; knowledge reaches a component file through a reviewed human edit
- **Access Claude Code internals**: Can only work with publicly exposed event data and documented APIs
- **Bypass performance requirements**: Cannot create hooks that violate sub-50ms or non-blocking constraints

When asked to perform unavailable actions, explain the limitation and suggest alternatives within hook system constraints.

## Output Format

Use the **Implementation Schema**.

**Phase 1: ANALYZE**
- Identify the event type and what the hook must record or gate
- Classify hook complexity (Simple single-event vs Complex multi-event coordination)
- Determine telemetry schema requirements

**Phase 2: DESIGN**
- Design hook architecture (event parsing, classification, telemetry writes, context injection)
- Plan performance optimizations for sub-50ms execution
- Design error handling and graceful degradation

**Phase 3: IMPLEMENT**
- Write hook Python code with all safety patterns
- Implement the telemetry writes
- Create test scenarios

**Phase 4: VALIDATE**
- Performance test: Execute time measurement (<50ms)
- Non-blocking test: Verify exit code 0 on all paths
- Error handling test: Malformed JSON, missing files, concurrent access
- Integration test: Context injection and telemetry writes

**Final Output**:
```
═══════════════════════════════════════════════════════════════
 HOOK CREATED: {hook-name}
═══════════════════════════════════════════════════════════════

 Event Type: PostToolUse | PreToolUse | SessionStart
 Performance: {measured-time}ms (target: <50ms)
 Exit Code: 0 (non-blocking ✓)

 Files Created:
   - hooks/{hook-name}.py
   - tests/test_{hook-name}.py
   - settings.json registration entry

 Suggested Next Steps:
   - Test: python hooks/{hook-name}.py < test_event.json
   - Performance: time python hooks/{hook-name}.py < test_event.json
   - Register: Add to settings.json hooks section
═══════════════════════════════════════════════════════════════
```

## Hook Architecture

The event-driven pipeline flows: Session → Event Generation (PostToolUse/PreToolUse/SessionStart) → Hook Registry → Event JSON Input → Gate or Classify → Telemetry Write → optional context injection via `context_output()`. See [architecture.md](hook-development/architecture.md) for the full pipeline diagram and the telemetry directory structure.

See [code-examples.md](hook-development/code-examples.md) for detailed specifications and examples. See [telemetry-database.md](hook-development/telemetry-database.md) for schema and operations.

## Error Handling and Preferred Patterns

See [preferred-patterns.md](hook-development/preferred-patterns.md) for the full pattern catalog: blocking on errors, synchronous heavy operations, direct database writes, registering before deploying, unguarded `main()`, UserPromptSubmit agent-context injection, and the atomic write pattern with code examples.

### Reaction-Detector Design Pattern

When a hook classifies free text into an outcome (e.g., accept/reject), decide the asymmetric cost first: which false direction is cheap? A missed acceptance stays neutral — safe. A false acceptance corrupts telemetry. Gate the expensive direction with stacked precision guards:

1. Marker leads the prompt (acceptance marker must appear at the start).
2. Negation veto (negation near the marker stays neutral).
3. Leading-task-verb veto (a prompt that opens with a task verb is a new instruction; stays neutral).
4. Conditional/instructional-cue veto ("if", "when", "should" cues stay neutral).
5. Short-clause cap (long clauses after the marker stay neutral).

Require golden fixtures in both directions: every marker fires, every veto case stays neutral. Example: `hooks/routing-outcome-finalizer.py`.

Wrap the entire script with `finally: sys.exit(0)`; catching errors only inside `main()` can still leave non-zero exit paths.

## Blocker Criteria

Resolve routine issues from repository evidence and existing authorization. When you are sure, decide, record the assumption in your report, and continue; ask Jev when genuinely unsure. Ask only when these situations leave a material decision uncovered:

Rows marked **(owner)** guard production, destructive, credential, public-exposure, or persistence actions; stop and get explicit owner confirmation before acting on those.

| Situation | Why Stop | Ask This |
|-----------|----------|----------|
| Hook requires >50ms execution | Violates hard requirement | "This operation needs >50ms - simplify or make async?" |
| Unclear outcome classification | A wrong score corrupts the route-health signal | "Should this outcome score as failure or neutral?" |
| A hook would write to an agent or skill file **(owner)** | Automated component edits need a human in the path | "This writes to a component file — should it record a signal instead?" |
| Breaking schema change **(owner)** | Backward compatibility risk | "This changes schema - migrate existing data how?" |

### Verify Before Assuming
- Outcome classification boundaries (failure vs neutral vs success)
- Telemetry schema changes (owner confirms; they change stored data)
- Hook event type selection (PostToolUse vs PreToolUse vs SessionStart)

## Death Loop Prevention

### Retry Limits
- Maximum 3 attempts for telemetry write operations
- Clear failure escalation path to debug logging

### Recovery Protocol
1. Detection: How to identify stuck state (hook timeout, repeated failures)
2. Intervention: Steps to break loop (disable hook, clear corrupted DB)
3. Prevention: Update patterns (add circuit breaker, add a `busy_timeout`, narrow the matcher)

## Reference Loading Table

| Signal | Reference File | When to Load |
|--------|---------------|--------------|
| Pipeline diagram, event flow, telemetry directory structure | `hook-development/architecture.md` | When explaining hook integration or reviewing system design |
| Blocking errors, synchronous ops, direct writes, registration order, unguarded main(), UserPromptSubmit misuse | `hook-development/preferred-patterns.md` | When reviewing hook code or debugging session deadlocks — preferred patterns and detection |
| Production hook template, non-blocking pattern, complete implementations | `hook-development/code-examples.md` | When scaffolding a new hook from scratch |
| Telemetry schema, routing and evidence writes, atomic write ops, read-only CLI | `hook-development/telemetry-database.md` | When implementing telemetry store operations |

## Shared Patterns

[anti-rationalization-core.md](../../../shared-patterns/anti-rationalization-core.md) | [gate-enforcement.md](../../../shared-patterns/gate-enforcement.md) | [verification-checklist.md](../../../shared-patterns/verification-checklist.md)
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall