oliver-kriska/claude-elixir-phoenix / lab/autoresearch
lab:autoresearch
VerifiedSelf-improving loop for plugin skills. Reads program.md, proposes one mutation per iteration, evaluates against deterministic scorer, keeps improvements via git, reverts failures. Targets weakest skill+dimension. Use with /loop for overnight runs.
| File | Kind | Size |
|---|---|---|
| .gitignore | other | 130 B |
| SKILL.md | skill md | 4.6 KB |
| program.md | markdown | 3.9 KB |
| references/mutation-strategies.md | markdown | 2.4 KB |
| references/state-management.md | markdown | 1.1 KB |
| retention.py | script | 5.7 KB |
| scripts/checks.shexecutable | script | 3.2 KB |
| scripts/protected_sections.py | script | 3.4 KB |
| scripts/run-iteration.py | script | 25.0 KB |
| scripts/score-skill.py | script | 1.2 KB |
| tests/__init__.py | script | 0 B |
| tests/test_deviation_dispatch.py | script | 3.1 KB |
| tests/test_protected_sections.py | script | 3.3 KB |
scripts/score-skill.py
View on GitHub#!/usr/bin/env python3
"""Thin wrapper around lab/eval/scorer.py for autoresearch use.
Usage:
python3 lab/autoresearch/scripts/score-skill.py plugins/elixir-phoenix/skills/review/SKILL.md
python3 lab/autoresearch/scripts/score-skill.py plugins/elixir-phoenix/skills/review/SKILL.md lab/eval/evals/review.json
Outputs single JSON line to stdout. Errors to stderr.
"""
import os
import sys
# Add project root to path (scripts/ -> autoresearch/ -> lab/ -> project root)
project_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
sys.path.insert(0, project_root)
from lab.eval.scorer import score_skill, find_eval
from lab.eval.schemas import EvalDefinition
def main():
if len(sys.argv) < 2:
print("Usage: score-skill.py <skill_path> [eval_path]", file=sys.stderr)
sys.exit(1)
skill_path = sys.argv[1]
eval_path = sys.argv[2] if len(sys.argv) > 2 else None
if eval_path is None:
skill_name = os.path.basename(os.path.dirname(skill_path))
eval_path = find_eval(skill_name)
eval_def = EvalDefinition.from_file(eval_path) if eval_path else None
score = score_skill(skill_path, eval_def)
print(score.to_json())
if __name__ == "__main__":
main()
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |