dongshuyan/compass-skills / skills/run-history-skill-builder
run-history-skill-builder
VerifiedTurn a completed task, browser flow, artifact pipeline, failure-recovery trace, or repeatedly refined workflow into a new reusable skill package or a reviewed skill-design plan. Use when the user asks to make a new skill from real run history, extract a reusable workflow from conversation/logs/files, summarize lessons into a new skill, or produce a plan before writing files. Do not use to upgrade an existing skill or to execute the business workflow itself.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 7.2 KB |
| agents/openai.yaml | config | 347 B |
| evals/evals.json | config | 4.1 KB |
| references/examples.md | markdown | 2.7 KB |
| references/history-mining.md | markdown | 3.3 KB |
| references/open-source-pattern-mining.md | markdown | 1.9 KB |
| references/self-repair-and-evals.md | markdown | 3.0 KB |
| references/skill-design-protocol.md | markdown | 2.6 KB |
| scripts/test_validate_skill_package.py | script | 5.1 KB |
| scripts/validate_skill_package.py | script | 13.1 KB |
references/self-repair-and-evals.md
View on GitHub# Self-Repair And Evals
Read this file before delivery, after validation failures, or whenever the generated package still feels ambiguous.
## Structural Validation
Run the bundled validator:
```bash
<python> <skill-dir>/scripts/validate_skill_package.py <target-skill-dir>
```
Then run the platform validator if the current host provides one.
A structural pass proves only that the package is well formed and its deterministic artifact checks succeeded. It does not execute `evals/evals.json` or establish that the skill behaves better than a baseline; use the trigger review and comparison evaluation below for those claims.
## Trigger Review
Start with the smallest useful set:
- one representative should-trigger prompt;
- one nearby should-not-trigger prompt;
- one boundary prompt.
Add cases only when the result is unstable, the trigger is broad, or mistakes are expensive.
If the samples reveal false positives or false negatives, fix the frontmatter `description` first, then the body.
## Adversarial Checklist
Before release, ask:
- Does another agent know exactly when this skill should fire?
- Is the output directory locked before file writes?
- Is the request actually about a new skill rather than an existing-skill upgrade?
- Which branch still depends on a guess instead of user confirmation or machine-checkable evidence?
- Which deterministic checks should be scripted instead of repeated in prose?
- Are there useless resources or placeholder files left behind?
- Is any private path, credential-like string, or hidden internal note still present?
## Eval Design
Create realistic prompts in `evals/evals.json` and make them check:
- trigger accuracy;
- output package shape;
- boundary handling;
- privacy discipline;
- plan-only behavior when requested;
- correct redirection to the upgrader when the user actually wants to modify an existing skill.
Minimal example:
```json
{
"id": 1,
"prompt": "Turn the document-processing workflow we just stabilized into a new skill.",
"expected_output": "A new skill package with trigger wording, state gates, reusable checks, and validation results.",
"expectations": [
"The response creates a package instead of a prose-only summary.",
"The package captures reusable gates and checks.",
"The final response reports real validation results."
]
}
```
## Repair Loop
If validation fails:
1. Find the structure owner: frontmatter, workflow, references, scripts, evals, or final response contract.
2. Fix that owner instead of scattering reminders elsewhere.
3. Update examples or evals if the failure came from a real run.
4. Re-run the smallest sufficient validation set.
5. Report any remaining warnings as residual risk.
## Comparison Evaluations
Run before/after comparisons when:
- the skill will be reused often;
- mistakes are expensive;
- the trigger boundary changed materially;
- the user asks for measurable improvement.
If comparison testing is not practical, provide the prompts and review checklist instead of pretending that the comparison already happened.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |