mxyhi/ok-skills / autoresearch
autoresearch
FlaggedAutonomous iteration loop: modify, verify, keep/discard against any metric
| File | Kind | Size |
|---|---|---|
| LICENSE | other | 1.0 KB |
| SKILL.md | skill md | 7.8 KB |
| agents/openai.yaml | config | 250 B |
| autoresearch.md | markdown | 4.8 KB |
| debug.md | markdown | 4.0 KB |
| evals.md | markdown | 5.0 KB |
| fix.md | markdown | 4.3 KB |
| improve.md | markdown | 6.0 KB |
| learn.md | markdown | 7.9 KB |
| plan.md | markdown | 2.8 KB |
| predict.md | markdown | 3.7 KB |
| probe.md | markdown | 4.8 KB |
| reason.md | markdown | 4.7 KB |
| references/orchestrator-routing.md | markdown | 6.8 KB |
| references/predict-personas.md | markdown | 3.1 KB |
| references/reason-judge-protocol.md | markdown | 3.4 KB |
| references/security-checklist.md | markdown | 3.4 KB |
| regression.md | markdown | 9.3 KB |
| scenario.md | markdown | 4.1 KB |
| scripts/orchestrate.shexecutable | script | 18.5 KB |
| scripts/score-regression.shexecutable | script | 7.5 KB |
| security.md | markdown | 4.6 KB |
| ship.md | markdown | 3.8 KB |
debug.md
View on GitHub---
name: autoresearch:debug
description: "Hunt bugs with scientific method: hypothesize, test, falsify, repeat"
argument-hint: "[Scope: <glob>] [Symptom: <text>] [Iterations: N] [--fix] [--evals]"
---
EXECUTE IMMEDIATELY.
## Parse Arguments
Extract from $ARGUMENTS:
- `Scope:` or `--scope` — file globs to investigate
- `Symptom:` or `--symptom` — error message or behavior description
- `Iterations:` or `--iterations` — default 15. "unlimited" for unbounded.
- `--fix` — shorthand for `--chain fix`
- `--severity` — filter: critical, high, medium, low
- `--technique` — force specific technique
- `--evals`, `--evals-interval N`, `--chain`
## Setup (if required context missing)
If Scope and Symptom both missing:
1. Auto-scan: run tests, lint, typecheck to detect existing failures
2. request_user_input (single batch):
Q1 (Issue): "What's the problem?" — hunt all bugs, specific error, failing tests, CI failure, performance
Q2 (Scope): "Which files?" — suggested globs + entire codebase
Q3 (Depth): "How deep?" — quick (5), standard (15), deep (30+), unlimited
Q4 (After): "When bugs found?" — report only, find and fix (--chain fix), chain to other, ask each time
If all provided → skip.
## Investigation Techniques
| Technique | When to Use |
|---|---|
| Binary search | Know when it worked, find when it broke |
| Differential | Compare working vs broken state |
| Minimal reproduction | Simplify to smallest failing case |
| Trace | Follow execution path through code |
| Pattern search | Grep for known anti-patterns |
| Working backwards | Start from error, trace to root cause |
## Establish Baseline (before loop)
1. Auto-scan for failures if no symptom provided
2. Create output directory: `autoresearch/debug-{YYMMDD}-{HHMM}/`
3. TSV header: `# metric_direction: higher_is_better\niteration\ttimestamp\thypothesis\tstatus\ttechnique\tevidence\tfile_line`
4. Metric = cumulative confirmed findings count
## Iteration Loop
### Phase 1: Review Context
- Read results TSV (past findings)
- Assess: what's been tested, what vectors remain
- If no hypotheses left → early stop
### Phase 2: Hypothesize
- Form ONE specific, falsifiable hypothesis
- Format: "I hypothesize that {X} because {evidence}. Test by {Y}."
- Hypothesis must be testable and different from all previous
### Phase 3: Investigate
- Apply appropriate technique for this hypothesis
- Read relevant code, run targeted tests, check logs
- Collect evidence (file:line references required)
### Phase 4: Classify
- **confirmed** — hypothesis correct, bug found with evidence
- **disproven** — hypothesis wrong, evidence against it
- **inconclusive** — can't prove or disprove, needs different approach
### Phase 5: Log
Append to TSV: iteration, timestamp, hypothesis, status, technique, evidence, file_line
### Eval Checkpoint
If --evals: check if current_iteration % interval == 0 → run checkpoint analysis.
### Bounded Check
If bounded: current_iteration >= max_iterations → exit loop, print summary.
## Summary
Print: total hypotheses tested, confirmed/disproven/inconclusive counts, all confirmed bugs with severity and file:line.
## Eval Checkpoint (--evals flag)
If --evals present:
- Compute interval: floor(max_iterations / 3), min 1. Fixed 10 if unbounded. Override: --evals-interval N.
- Every {interval} iterations, pause and analyze current results TSV.
- Print: `--- Eval Checkpoint (iterations {X}-{Y}) ---\nFindings: {confirmed} confirmed | Trend: {up/flat/down}\n{one-line recommendation}\n---`
- If plateau 3+ checkpoints (no new confirmed) → recommend early stop.
- At loop end → full evals summary to evals-summary.md in output directory.
## Chain Handoff
After completion, write handoff.json to output directory: version "2.1.0", source "debug", timestamp, status (COMPLETE|USER_INTERRUPT|BOUNDED|ERROR), results_tsv path, findings = confirmed bugs with severity + file:line, config{scope, symptom}.
If --fix flag → chain to fix automatically.
Invoke next target in --chain order. Propagate --evals flag.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |