tzachbon/srulik-toolkit / plugins/srulik-toolkit/skills/skill-creator
skill-creator
VerifiedCreate or edit a skill, improve its instructions or triggers, evaluate it on real tasks, or benchmark a revision. Every skill creation or edit uses create-plan and implement-plan, including small changes; evaluation compares actual outputs and includes human review.
| File | Kind | Size |
|---|---|---|
| LICENSE.txt | markdown | 11.1 KB |
| SKILL.md | skill md | 4.2 KB |
| agents/analyzer.md | markdown | 10.3 KB |
| agents/grader.md | markdown | 9.1 KB |
| agents/openai.yaml | config | 118 B |
| assets/eval_review.html | other | 6.8 KB |
| eval-viewer/generate_review.py | script | 7.2 KB |
| eval-viewer/viewer.html | other | 40.7 KB |
| references/authoring.md | markdown | 2.8 KB |
| references/claude-code.md | markdown | 3.9 KB |
| references/codex.md | markdown | 2.1 KB |
| references/evaluation.md | markdown | 3.9 KB |
| references/hosts.md | markdown | 861 B |
| references/oh-my-pi.md | markdown | 1.2 KB |
| references/schemas.md | markdown | 3.9 KB |
| references/t3-code.md | markdown | 2.2 KB |
| scripts/__init__.py | script | 0 B |
| scripts/aggregate_benchmark.py | script | 9.3 KB |
| scripts/generate_report.py | script | 12.5 KB |
| scripts/improve_description.py | script | 11.3 KB |
| scripts/run_eval.py | script | 12.6 KB |
| scripts/run_loop.py | script | 13.9 KB |
| scripts/utils.py | script | 3.0 KB |
references/claude-code.md
View on GitHub# Claude Code mapping
Use `/srulik-toolkit:skill-creator`, then the namespaced `create-plan` and
`implement-plan` skills. Verify the resolved skill source; another installed
`skill-creator` may have the same short name. Native questions and permission
handling take precedence over prose prompts when available.
Load an isolated local plugin with `claude --plugin-dir <plugin-root>` and a
process-scoped disposable `CLAUDE_CONFIG_DIR`. Do not copy credential stores or
edit a shared installation. Use authorized runtime credential references if
needed. Verify the actual paths and source identity in a fresh conversation.
For restricted print-mode verification, include the trusted creator directory
with `--add-dir` and authorize its Python helper commands. Direct absolute
entry points, such as `python3 -B <creator>/scripts/aggregate_benchmark.py`, avoid
environment-prefixed module invocations. Keep generated files in the scratch
project. A denied command stops the affected task until its scope is explicitly
resolved; retain the denial in the evidence.
For multi-turn noninteractive tests, use a unique `--session-id` and resume it
with `--resume`; retain session persistence until plan questions and explicit
implementation authorization are answered. Record provider/model/permission
mode and assistant/file evidence. A printed skill list alone is not invocation.
Use native child agents for paired target/baseline tasks when allowed. A fresh
serial session is the fallback when concurrency is unavailable. Keep mutable
outputs in separate directories. Save token/timing information only when
actually exposed; do not assume another host's notification fields.
Generate the static review file; show it using the available artifact surface
or return its clickable absolute path. Import explicit completed feedback.
Do not launch the upstream viewer server. Runtime support must be labeled with
the actual platform/version exercised; source validation is a separate check.
## Optional description optimization
This optional path is Claude-specific and currently POSIX-only because the
upstream trigger runner uses `select` on process pipes. It is never automatic
and is not a prerequisite for portable authoring/evaluation. Missing CLI,
model, authentication or platform support skips this capability explicitly.
Use a disposable project containing its own `.claude` directory. Keep cwd there
so temporary commands cannot land in a parent real project. Set process-scoped
`PYTHONPATH` to the absolute bundled skill directory, then run:
```sh
python3 -m scripts.run_loop --eval-set /absolute/path/trigger-evals.json \
--skill-path /absolute/path/target-skill --model <available-claude-model> \
--num-workers 1 --max-iterations 3 --report none \
--results-dir /absolute/path/optimizer-results
```
Select an actual supported Claude model; do not pass the parent model identifier
from another provider. Review the mixed positive/negative trigger queries with
the human first, using [the template](../assets/eval_review.html). Substitute
plain-text placeholders with HTML escaping and embedded JSON with script-context
escaping (`<` as `\u003c`); do not splice raw query text into script or markup.
The runner returns a proposed description chosen using held-out scores; it does
not authorize applying it. Every application is a skill edit and must be covered
by `create-plan` and executed through `implement-plan`. Use bounded calls and
report test-set size/limits. Do not imply that a live optimizer run was verified
when only local fake tests ran.
With holdout enabled, provide at least two queries per present trigger class so
each class retains training and test examples. For smaller sets, add queries or
explicitly use `--holdout 0`; this disables held-out evaluation.
Capability mapping is based on inspected CLI help and upstream source. Live
creation/improvement evidence, when available, belongs in the repository's
verification receipt; unexercised configurations remain unverified.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |