Skip to content

notque/vexjoy-agent / skills/meta/toolkit

toolkit

Verified

Toolkit management: create and evaluate skills and agents, manage routing tables, generate Claude.md.

FileKindSize
SKILL.mdskill md15.5 KB
agents/skill-creator/analyzer.mdmarkdown4.8 KB
agents/skill-creator/comparator.mdmarkdown4.7 KB
agents/skill-creator/grader.mdmarkdown3.9 KB
assets/skill-creator/eval_viewer.htmlother57.1 KB
references/agent-comparison.mdmarkdown11.5 KB
references/agent-comparison/benchmark-tasks.mdmarkdown6.8 KB
references/agent-comparison/do-creation-compliance-tasks.jsonconfig8.5 KB
references/agent-comparison/examples-and-errors.mdmarkdown1.0 KB
references/agent-comparison/grading-rubric.mdmarkdown4.3 KB
references/agent-comparison/methodology.mdmarkdown7.4 KB
references/agent-comparison/optimization-guide.mdmarkdown12.0 KB
references/agent-comparison/optimization-tasks.example.jsonconfig853 B
references/agent-comparison/optimize-phase.mdmarkdown9.3 KB
references/agent-comparison/read-only-ops-short-tasks.jsonconfig375 B
references/agent-comparison/report-template.mdmarkdown3.8 KB
references/agent-comparison/socratic-debugging-body-short-tasks.jsonconfig372 B
references/agent-comparison/socratic-debugging-trigger-tasks.jsonconfig2.8 KB
references/agent-creator.mdmarkdown10.9 KB
references/agent-creator/agent-design-patterns.mdmarkdown12.4 KB
references/agent-creator/agent-eval-design.mdmarkdown7.7 KB
references/agent-creator/agent-frontmatter-template.mdmarkdown9.3 KB
references/agent-evaluation.mdmarkdown10.6 KB
references/agent-evaluation/batch-evaluation.mdmarkdown1.6 KB
references/agent-evaluation/common-issues.mdmarkdown2.1 KB
references/agent-evaluation/report-templates.mdmarkdown2.2 KB
references/agent-evaluation/scoring-rubric.mdmarkdown2.4 KB
references/blind-ab-tests.mdmarkdown7.8 KB
references/generate-claudemd.mdmarkdown11.0 KB
references/generate-claudemd/CLAUDEMD_TEMPLATE.mdmarkdown4.1 KB
references/generate-claudemd/examples-and-errors.mdmarkdown7.4 KB
references/github-profile-rules.mdmarkdown7.5 KB
references/github-profile-rules/github-api-patterns.mdmarkdown8.2 KB
references/github-profile-rules/rule-categories.mdmarkdown4.8 KB
references/hook-development.mdmarkdown10.9 KB
references/hook-development/architecture.mdmarkdown3.7 KB
references/hook-development/code-examples.mdmarkdown9.4 KB
references/hook-development/preferred-patterns.mdmarkdown6.5 KB
references/hook-development/telemetry-database.mdmarkdown6.5 KB
references/routing-table-updater.mdmarkdown10.2 KB
references/routing-table-updater/batch-mode.mdmarkdown2.1 KB
references/routing-table-updater/conflict-resolution.mdmarkdown3.8 KB
references/routing-table-updater/error-handling.mdmarkdown1.9 KB
references/routing-table-updater/examples.mdmarkdown4.8 KB
references/routing-table-updater/extraction-patterns.mdmarkdown2.2 KB
references/routing-table-updater/routing-format.mdmarkdown3.6 KB
references/routing-table-updater/skill-examples.mdmarkdown2.7 KB
references/skill-composer.mdmarkdown9.1 KB
references/skill-composer/compatibility-matrix.mdmarkdown13.0 KB
references/skill-composer/composition-patterns.mdmarkdown4.9 KB
references/skill-composer/examples.mdmarkdown17.1 KB
references/skill-composer/skill-patterns.mdmarkdown12.5 KB
references/skill-creator.mdmarkdown28.9 KB
references/skill-creator/agent-template.mdmarkdown18.5 KB
references/skill-creator/artifact-schemas.mdmarkdown8.2 KB
references/skill-creator/bundled-components.mdmarkdown2.3 KB
references/skill-creator/complexity-tiers.mdmarkdown8.0 KB
references/skill-creator/domain-research-targets.mdmarkdown13.2 KB
references/skill-creator/enrichment-workflow.mdmarkdown9.9 KB
references/skill-creator/error-catalog.mdmarkdown9.9 KB
references/skill-creator/preferred-patterns.mdmarkdown11.7 KB
references/skill-creator/progressive-disclosure.mdmarkdown8.5 KB
references/skill-creator/skill-template.mdmarkdown14.6 KB
references/skill-creator/workflow-patterns.mdmarkdown7.6 KB
references/system-upgrade.mdmarkdown11.0 KB
references/system-upgrade/component-audit-checklists.mdmarkdown8.4 KB
references/system-upgrade/upgrade-failure-modes.mdmarkdown8.7 KB
references/system-upgrade/upgrade-signal-parsing.mdmarkdown7.3 KB
references/toolkit-evolution.mdmarkdown14.5 KB
references/toolkit-evolution/diagnose-scripts.mdmarkdown9.5 KB
references/toolkit-evolution/evolution-history.mdmarkdown5.5 KB
references/toolkit-evolution/evolution-report-template.mdmarkdown716 B
references/toolkit-evolution/evolve-preferred-patterns.mdmarkdown6.9 KB
references/toolkit-evolution/evolve-scripts.mdmarkdown3.7 KB
references/toolkit-governance.mdmarkdown16.0 KB
references/toolkit-governance/adr-lifecycle.mdmarkdown10.9 KB
references/toolkit-governance/frontmatter-compliance.mdmarkdown11.1 KB
references/toolkit-governance/hook-standardization.mdmarkdown21.5 KB
references/toolkit-governance/routing-table-patterns.mdmarkdown11.6 KB
references/weak-model-uplift.mdmarkdown10.3 KB
scripts/agent-comparison/compare.pyscript2.3 KB
scripts/agent-comparison/generate_variant.pyscript19.6 KB
scripts/agent-comparison/optimize_loop.pyscript89.3 KB
scripts/agent-comparison/trigger_eval.pyscript19.5 KB
scripts/routing-table-updater/extract_metadata.pyscript10.8 KB
scripts/routing-table-updater/generate_routes.pyscript10.5 KB
scripts/routing-table-updater/scan.pyscript4.7 KB
scripts/routing-table-updater/update_routing.pyscript12.4 KB
scripts/routing-table-updater/validate.pyscript7.3 KB
scripts/skill-composer/build_dag.pyscript12.5 KB
scripts/skill-composer/discover_skills.pyscript12.1 KB
scripts/skill-composer/validate.pyscript12.3 KB
scripts/skill-creator/aggregate_benchmark.pyscript10.0 KB
scripts/skill-creator/eval_compare.pyscript10.2 KB
scripts/skill-creator/optimize_description.pyscript12.5 KB
scripts/skill-creator/package_results.pyscript7.8 KB
scripts/skill-creator/run_eval.pyscript6.3 KB

references/toolkit-evolution/diagnose-scripts.md

View on GitHub
# DIAGNOSE Phase Scripts

> **Scope**: Concrete bash/Python commands for each step of Phase 1 DIAGNOSE and Phase 0 DISCOVER (frequency check, briefing data, PR-history mining). Load this reference before running those phases. The SKILL.md contains the prose instructions and decision logic; this file contains the exact commands to execute.

---

## Discovery Frequency Check

Check the last discovery run date before starting Phase 0:

```bash
# Find the most recent discovery report
latest=$(ls -t evolution-reports/discovery-*.md 2>/dev/null | head -1)
if [ -z "$latest" ]; then
  echo "NO_PREVIOUS_DISCOVERY"
else
  # Extract date from filename: discovery-YYYY-MM-DD.md
  report_date=$(basename "$latest" | sed 's/discovery-//;s/\.md//')
  days_ago=$(( ($(date +%s) - $(date -d "$report_date" +%s)) / 86400 ))
  echo "Last discovery: $report_date ($days_ago days ago)"
  [ "$days_ago" -ge 30 ] && echo "DISCOVER_DUE" || echo "DISCOVER_SKIPPED"
fi
```

## DISCOVER Step 1: Briefing Data

Collect current toolkit state to brief all perspective agents:

```bash
# Skill count and category distribution
python3 -c "
import json
with open('skills/INDEX.json') as f:
    idx = json.load(f)
skills = idx.get('skills', {})
print(f'Total skills: {len(skills)}')
categories = {}
for s, meta in skills.items():
    cat = meta.get('category', 'uncategorized')
    categories[cat] = categories.get(cat, 0) + 1
for cat, count in sorted(categories.items(), key=lambda x: -x[1]):
    print(f'  {cat}: {count}')
"

# Agent count
python3 -c "
import json
with open('agents/INDEX.json') as f:
    idx = json.load(f)
agents = idx.get('agents', {})
print(f'Total agents: {len(agents)}')
for a in sorted(agents):
    print(f'  {a}')
"
```

---

## DISCOVER Step 2b: PR-History Reflection

Read-only `gh` queries. No mutation of PRs, comments, or labels.

```bash
# Titles, bodies, and changed files for the last 30 merged PRs
gh pr list --state merged --limit 30 --json number,title,body,files \
  --jq '.[] | {number, title, files: [.files[].path]}'

# Review-comment threads for each of those PRs
for n in $(gh pr list --state merged --limit 30 --json number --jq '.[].number'); do
  echo "--- PR #$n ---"
  gh pr view "$n" --comments
done
```

Read the combined output for three signals:
- **Recurring friction**: the same file, directory, or component named as the diff target across 3+ PRs — a change that keeps needing a change is a design gap, not bad luck.
- **Repeated fix patterns**: review comments requesting the identical correction (a missing test, a missing frontmatter field, a missing validation call) on separate PRs — the fix belongs in a hook, script, or skill step, not in each reviewer's memory.
- **Skill/agent gaps**: a PR manually performs a step (e.g., hand-editing INDEX.json, manually chasing a routing entry) that an existing script or skill should have automated, or that no component covers.

Each signal needs 2+ distinct PRs as evidence (Triple-Validation recurrence check) before it becomes a proposal. Format proposals like other DISCOVER output: one-sentence description, evidence (PR numbers), estimated impact. Tag source `[PR-HISTORY]`.

## DIAGNOSE Step 1: Routing telemetry queries

Run these queries to surface weak routes, error-prone dispatches, and reviewer cost:

```bash
python3 ~/.claude/scripts/learning-db.py route-health
python3 ~/.claude/scripts/learning-db.py route-stats --by agent
python3 ~/.claude/scripts/learning-db.py route-stats --by skill
python3 ~/.claude/scripts/learning-db.py route-stats --by errors
python3 ~/.claude/scripts/learning-db.py review-roi
```

Read `route-health` first: it reports the outcome basis, and a per-route error rate computed mostly from neutral outcomes describes the scorer rather than the router. A route with many dispatches and a high error rate is a routing gap; a component with zero dispatches over a long window is shelf-ware. For prior cycle history and already-shelved proposals, read `references/evolution-history.md` — re-proposing a rejected idea is the failure this step prevents.

## DIAGNOSE Step 2: Git History Scan

```bash
# Frequent fixes to same areas suggest chronic issues
git log --oneline --since="2 weeks ago" | head -40

# Files changed most frequently (churn = potential problems)
git log --since="2 weeks ago" --pretty=format: --name-only | sort | uniq -c | sort -rn | head -20
```

## DIAGNOSE Step 3: Dream Report Check

```bash
ls -t ~/.claude/state/dream-* 2>/dev/null | head -5
# Then read the most recent dream-analysis-*.md file
```

## DIAGNOSE Step 3b: Dream Insight Cross-Validation

For each insight from the dream report, verify it still matches current state before treating it as a proposal signal. A dream report can be days old; the repo moves fast.

For each insight that names a specific file path:

```bash
# Verify the file exists
ls -la {path-mentioned-in-dream} 2>/dev/null || echo "STALE: path does not exist -- exclude this insight"
```

For each insight that claims recent activity on a file or area:

```bash
# Verify recent git activity matches the insight
git log --oneline --since="7 days ago" -- {path} 2>/dev/null | head -5
# If empty: the "recent activity" the dream described may be older than 7 days
```

**Staleness rules:**
- Name a file that no longer exists → mark STALE, exclude from opportunity list
- Claims "recent activity" but git log shows nothing in 7 days → mark STALE
- References a pattern already captured in a merged PR → check git log to confirm, then exclude (it is done)

Only forward dream insights where at least one current-state check passes.

## DIAGNOSE Step 4: Routing-Table Drift Check

```bash
python3 scripts/check-routing-drift.py --verbose
```

The `--verbose` flag prints each skill checked plus the final PASS/FAIL. Exit code 1 means skills are missing from the routing manifest.

## DIAGNOSE Step 4b: Orphaned ADR Session Check

```bash
if [ -f ".adr-session.json" ]; then
  adr_info=$(python3 -c "
import json, sys
try:
    d = json.load(open('.adr-session.json'))
    adr_path = d.get('adr_path', '')
    domain = d.get('domain', adr_path)
    print(f'{adr_path}|{domain}')
except Exception as e:
    print('PARSE_ERROR|PARSE_ERROR')
")
  adr_path="${adr_info%|*}"
  domain="${adr_info#*|}"
  if [ "$domain" = "PARSE_ERROR" ]; then
    echo "WARNING: .adr-session.json exists but is unparseable -- flag as cleanup opportunity"
  elif [ -z "$adr_path" ] || [ ! -f "$adr_path" ]; then
    echo "WARNING: .adr-session.json references '$domain' but '$adr_path' does not exist"
    echo "  Orphaned session file. Add 'Remove orphaned .adr-session.json' to the opportunity list."
  else
    echo "ADR session OK: $domain exists at $adr_path"
  fi
else
  echo "No active ADR session file (OK)"
fi
if [ -f ".adr-session.json.stale" ]; then
  echo "STALE: .adr-session.json.stale exists -- flag as cleanup opportunity"
fi
```

## DIAGNOSE Step 4c: Stub Hook Audit

```bash
python3 -c "
import json, os, re
from pathlib import Path

settings_path = Path('.claude/settings.json')
if not settings_path.exists():
    print('No .claude/settings.json found -- skip hook stub audit')
else:
    with open(settings_path) as f:
        settings = json.load(f)
    hooks = settings.get('hooks', {})
    stubs = []
    for event, groups in hooks.items():
        for group in (groups if isinstance(groups, list) else [groups]):
            entries = group.get('hooks', [group]) if isinstance(group, dict) else [group]
            for entry in entries:
                cmd = entry.get('command', '') if isinstance(entry, dict) else str(entry)
                m = re.search(r'python3 [\"\x27]?([\w/.\$~-]+\.py)[\"\x27]?', cmd)
                if not m:
                    continue
                script = m.group(1).replace('\$HOME', str(Path.home()))
                script = os.path.expandvars(script)
                if not os.path.exists(script):
                    continue
                with open(script) as sf:
                    body = sf.read()
                if 'DISABLED' in body or 'empty_output()' in body:
                    desc = entry.get('description', '(no description)') if isinstance(entry, dict) else ''
                    stubs.append((event, os.path.basename(script), desc))
    if stubs:
        print(f'{len(stubs)} stub hook(s) registered in settings.json:')
        for ev, name, desc in stubs:
            print(f'  [{ev}] {name} -- {desc}')
        print('  Add stub deregistration to the opportunity list.')
    else:
        print('No stub hooks found (OK)')
"
```

## DIAGNOSE Step 4d: Usage and Governance Signals

Two activity feeds turn runtime telemetry into opportunities. Both exit 0 on success.

```bash
# Dormant skills/agents (unused in the window) — feed into gap discovery.
# A long-dormant component is either a discoverability gap or a retirement candidate.
python3 scripts/usage-report.py --dormant --json

# Unresolved governance events (last 30 days) — feed into the "what's failing" diagnosis.
# Repeated hook_blocked / policy_violation events surface enforcement friction worth fixing.
python3 scripts/governance-report.py --days 30 --unresolved
```

**How to use the output:**
- **usage-report `--dormant`**: Each dormant skill/agent is a candidate for the opportunity list — tag the source `[USAGE]`. Cross-check before proposing retirement: dormant may mean low discoverability (routing/trigger gap) rather than low value.
- **governance-report `--unresolved`**: Cluster unresolved events by `TYPE` and `TOOL`. A recurring `policy_violation` or `hook_blocked` cluster is an enforcement-friction signal — tag it `[GOVERNANCE]` on the opportunity list with the event count as evidence.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 470kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall