mgiovani/ai-arsenal / skills/create-skill
create-skill
VerifiedCreate a new agent skill (or Claude Code slash command) from a plain-language description, using live spec fetching, pattern research, and an approval-gated blueprint before any files are written. Use whenever the user wants to build, scaffold, or author a new skill, subagent capability, or slash command, including phrasings like 'make a command for X', 'create a slash command', 'turn this into a reusable skill', or 'package this workflow as a skill'. Not for editing CLAUDE.md/AGENTS.md memory rules (use create-rule) or discovering/installing existing third-party skills (use find-skills).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 20.3 KB |
| evals/evals.json | config | 3.5 KB |
| evals/trigger-eval.json | config | 2.2 KB |
| references/agent-prompts.md | markdown | 4.5 KB |
| references/eval-system.md | markdown | 5.6 KB |
| references/frontmatter-fields.md | markdown | 2.8 KB |
| references/schemas.md | markdown | 6.3 KB |
| references/skill-anatomy.md | markdown | 10.2 KB |
| references/specification-urls.md | markdown | 2.6 KB |
| scripts/generate_report.py | script | 11.5 KB |
| scripts/improve_description.py | script | 16.7 KB |
| scripts/package_skill.py | script | 4.5 KB |
| scripts/quick_validate.pyexecutable | script | 11.1 KB |
| scripts/run_eval.py | script | 8.1 KB |
references/schemas.md
View on GitHub# Skill Schemas
JSON schemas for eval system output formats. Use these when creating or validating eval files.
## Contents
- [evals/evals.json](#evalsevalsjson)
- [evals/trigger-eval.json](#evalstrigger-evaljson)
- [evals/results/{eval_id}/grading.json](#evalsresultsevalidgradingjson)
- [evals/metrics.json](#evalsmetricsjson)
- [evals/benchmark.json](#evalsbenchmarkjson)
## evals/evals.json
The evaluation definition file. Created during Phase 4 of skill creation.
```json
{
"$schema": "https://agentskills.io/schemas/evals.json",
"skill": "skill-name",
"description": "One-sentence description of what this skill does",
"evals": [
{
"id": "eval-1",
"prompt": "Exact trigger phrase a user would type",
"assertions": [
"Output contains expected content or structure",
"Files created at expected paths",
"No placeholder text in output"
]
}
]
}
```
### Field Descriptions
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `skill` | string | yes | Skill name (matches `name` in SKILL.md frontmatter) |
| `description` | string | yes | What the skill does in one sentence |
| `evals` | array | yes | List of eval test cases |
| `evals[].id` | string | yes | Unique identifier (kebab-case, e.g., "eval-basic-usage") |
| `evals[].prompt` | string | yes | Exact trigger phrase a user would type to activate the skill |
| `evals[].assertions` | array | yes | List of natural-language assertions about expected output |
### Writing Good Evals
**Prompts:**
- Write exactly what a real user would type, not an idealized machine-readable query
- Cover the range of trigger phrasings from the skill description
- Include at least one "easy" case and one case with ambiguity
**Assertions:**
- Be specific but not brittle: check structure and content, not exact wording
- Each assertion should fail for a clearly bad output and pass for a good one
- Avoid assertions that would pass even if the skill produced garbage
**Example evals for a commit message skill:**
```json
{
"skill": "git-commit",
"description": "Generates conventional commit messages from staged changes",
"evals": [
{
"id": "basic-feature",
"prompt": "Create a commit message for my staged changes",
"assertions": [
"Commit message follows conventional commits format (type: description)",
"Type is one of: feat, fix, docs, style, refactor, perf, test, build, ci, chore",
"Description is under 72 characters",
"No placeholder text like [description] remains"
]
},
{
"id": "breaking-change",
"prompt": "Write a commit for these API changes that break backwards compatibility",
"assertions": [
"Includes BREAKING CHANGE footer or ! in type",
"Body explains what changed and why"
]
}
]
}
```
## evals/trigger-eval.json
Description-triggering evals. Tests whether the skill's `description` fires (or correctly doesn't) on realistic prompts, a separate concern from evals.json, which tests task completion once the skill is active.
**Top-level JSON array, no wrapper object.** The schema test enforces this shape:
```json
[
{"query": "create a skill that generates changelog entries from commits", "should_trigger": true},
{"query": "add a rule to CLAUDE.md about commit message style", "should_trigger": false}
]
```
| Field | Type | Description |
|-------|------|-------------|
| `query` | string | A realistic prompt a user might type |
| `should_trigger` | bool | Whether the skill's description should cause it to fire on this query |
~20 entries, roughly half `should_trigger: false`. Make the negatives near-misses (phrasings from sibling skills' territory that a loose description would wrongly catch), not obviously unrelated queries; those don't discriminate anything.
## evals/results/{eval_id}/grading.json
Output from the grader agent. Created during eval execution.
```json
{
"eval_id": "eval-1",
"skill_name": "skill-name",
"with_skill_output": "...",
"baseline_output": "...",
"expectations": [
{
"assertion": "Output contains expected content",
"with_skill_pass": true,
"baseline_pass": false,
"reasoning": "The skill output contains X while baseline does not"
}
],
"summary": "Brief explanation of overall quality difference",
"score": 4,
"comparison": "with_skill_better",
"eval_feedback": "Suggestion for improving the skill or evals"
}
```
### Field Descriptions
| Field | Type | Description |
|-------|------|-------------|
| `eval_id` | string | Which eval this grades |
| `skill_name` | string | Skill being evaluated |
| `with_skill_output` | string | Output produced with skill active |
| `baseline_output` | string | Output produced without skill |
| `expectations` | array | Per-assertion verdicts |
| `expectations[].assertion` | string | The assertion being checked |
| `expectations[].with_skill_pass` | bool | Did with-skill output satisfy it? |
| `expectations[].baseline_pass` | bool | Did baseline satisfy it? |
| `expectations[].reasoning` | string | Explanation of the verdict |
| `summary` | string | Overall quality comparison |
| `score` | integer | 1-5 score for with-skill output quality |
| `comparison` | string | One of: `with_skill_better`, `baseline_better`, `equivalent` |
| `eval_feedback` | string | Actionable suggestion for improvement |
## evals/metrics.json
Aggregated metrics across all eval runs.
```json
{
"skill": "skill-name",
"run_timestamp": "2026-03-07T12:00:00Z",
"total_evals": 3,
"pass_rate": 0.67,
"mean_score": 3.8,
"with_skill_wins": 2,
"baseline_wins": 0,
"equivalent": 1,
"per_eval": [
{
"eval_id": "eval-1",
"score": 4,
"comparison": "with_skill_better",
"assertions_passed": 3,
"assertions_total": 3
}
]
}
```
## evals/benchmark.json
Tracks improvement across iterations.
```json
{
"skill": "skill-name",
"iterations": [
{
"iteration": 1,
"description_version": "Original description",
"train_score": 0.6,
"test_score": 0.55,
"timestamp": "2026-03-07T10:00:00Z"
},
{
"iteration": 2,
"description_version": "Improved description with more trigger phrases",
"train_score": 0.8,
"test_score": 0.75,
"timestamp": "2026-03-07T11:00:00Z"
}
],
"best_iteration": 2,
"improvement": 0.2
}
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |