Skip to content

naman10parikh/agentgrid / .claude/skills/skill-creator

skill-creator

Verified

Auto-generate new skills from learnings, approved patterns, and repeated user preferences. The meta-skill that makes the harness self-improving. Core to AutoLab.

When to Use

  • After discovering a repeatable pattern across 2+ sessions
  • When the user explicitly approves a workflow ("always do X")
  • When LEARNINGS.md has 3+ entries about the same topic
  • When a sub-agent finds a new capability worth capturing
  • During AutoLab nightly improvement cycles

Process

1. Source Mining

Scan these sources for skill candidates:

  • memory/LEARNINGS.md — repeated corrections = skill gaps
  • memory/MEMORY.md — confirmed patterns = skill candidates
  • memory/daily/*.md — workflows that worked = codify them
  • resources/read/*.md — external techniques worth internalizing
  • Session transcript — what did the user approve? correct? praise?
  • packages/web/features.json — feature patterns that recur

2. Skill Candidate Evaluation

For each candidate, score on 4 axes:

AxisQuestionThreshold
FrequencyHow often would this skill be triggered?2+ times/week
ComplexityDoes it need multi-step instructions?3+ steps
Error-proneDo we keep getting it wrong without the skill?2+ mistakes
ReusableDoes it apply across agents, not just one?Cross-agent preferred

Score >= 3/4 axes → create the skill.

3. Skill Template

---
name: { skill-name }
description: { one-line description, under 100 chars }
---

## When to Use

- {trigger condition 1}
- {trigger condition 2}

## Process

### Step 1: {Action}

{What to do, what to check}

### Step 2: {Action}

{What to do, what to check}

## Output

- {What the skill produces}
- {How to verify it worked}

## Model Tier

- {Haiku | Sonnet | Opus} for each step

4. Skill Creation Checklist

  1. Write SKILL.md to .claude/skills/{skill-name}/SKILL.md
  2. Description must be under 100 chars (for ~70 token routing)
  3. Add trigger patterns that the skill-routing system can match
  4. If the skill involves code patterns, include inline examples
  5. If the skill involves external tools, document the setup
  6. Update CLAUDE.md skills count
  7. Test: verify the skill is discoverable via /skill-routing

5. Skill Categories

CategoryPurposeExamples
BuildCode generation patternsapp-factory, agent-runtime
MetaSelf-improvement, researchself-improve, deep-think
OpsSetup, deployment, inframcp-setup, secrets-setup
AgentAgent-specific skillsPer-agent skills in agents/{name}/skills/
CaptureKnowledge capturetranscript-to-insight, resource-digest

6. For Agent-Level Skills

Skills can also be created for individual agents (not just the harness):

agents/{agent-name}/skills/{skill-name}.md

These are loaded into the agent's system prompt via the memory-loader. They follow the same template but are scoped to that agent's domain.

Integration with AutoLab

The Skill Creator is a key component of the AutoLab nightly cycle:

  1. AutoLab analyzes traces from the day's agent runs
  2. Identifies patterns where agents struggled or succeeded
  3. Calls Skill Creator to codify successful patterns
  4. New skills are available to all agents via Group Evolution

YouTube/Article Transcript Pipeline

When a YouTube video or article needs to be processed:

  1. Get the URL from user or resource queue
  2. For YouTube: Use computer use to visit a transcript service (e.g., tavio.io, kome.ai, or youtubetranscript.com)
  3. Enter the URL, download the transcript text
  4. Clean and structure the transcript
  5. Extract insights relevant to Energy platform
  6. Save to resources/read/{topic}.md
  7. If insights warrant a new skill, create it

X/Twitter Content Pipeline

  1. Use computer use to browse the tweet/thread
  2. Extract the full text content
  3. If it's a thread, compile all tweets
  4. Extract actionable insights
  5. Save to resources and update relevant docs

Skill Testing & Evals (from Anthropic's skill-creator enhancements, March 2026)

Two Kinds of Skills

  1. Capability uplift — helps Claude do something the base model can't do consistently (e.g., document creation, complex workflows). May become unnecessary as models improve.
  2. Encoded preference — sequences steps according to your team's process (e.g., NDA review, weekly updates). Durable, value depends on fidelity to actual workflow.

Writing Evals

For each skill, define test cases:

## Evals

### Test 1: {scenario}

- **Prompt:** "{test input that should trigger the skill}"
- **Expected:** {description of correct output}
- **Files:** {any test files needed}

### Test 2: {scenario}

...

Evals check that:

  • The skill triggers when it should (no false negatives)
  • The skill doesn't trigger when it shouldn't (no false positives)
  • Output quality meets the bar defined in "Expected"

Benchmark Mode

Run standardized assessment across all evals:

  • Track eval pass rate, elapsed time, token usage
  • Run after model updates or skill edits
  • Store results locally or integrate with CI

Multi-Agent Eval Execution

Spin up independent agents to run evals in parallel:

  • Each eval in a clean context (no cross-contamination)
  • Own token and timing metrics per eval
  • Use isolation: "worktree" for code-modifying skills

A/B Comparisons

Use comparator agents to judge:

  • Skill v1 vs v2 (after edits)
  • Skill vs no skill (is it still needed?)
  • Judges don't know which output is which → unbiased

Description Tuning

As skill count grows, description precision is critical:

  • Too broad → false triggers on unrelated prompts
  • Too narrow → never fires when needed
  • Analyze description against sample prompts
  • Suggest edits that cut both false positives and false negatives

The Future: Skills → Specifications

As models improve, skills evolve from "how to do X" (implementation) to "what X should produce" (specification). Evals already describe the "what" — eventually the eval IS the skill.

Verification

After creating a skill:

  • Confirm it appears in skill count
  • Confirm skill-routing can find it with relevant keywords
  • Confirm the description fits in ~70 tokens
  • Test with a simulated trigger to verify steps are clear
  • Write at least 2 evals per skill
  • Run benchmark to establish baseline metrics
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 469kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 321kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 321kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 321kUpdated Claude CodeCodexVerifiedInstall