agentsope/SkillAlchemy / skills/agentsop-prompt-compilation
agentsop-prompt-compilation
VerifiedThe compile-readiness gate for prompt auto-optimization. Decide whether you have earned the right to run an optimizer (DSPy MIPROv2 / GEPA / BootstrapFewShot) before spending compute. Two preconditions only — a real metric, and enough examples for the optimizer you picked. Garbage metric in, garbage prompt out. Pick the optimizer by data scale; GEPA inverts the scale assumption (~10 examples + textual feedback).
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 4.1 KB |
| SKILL.md | skill md | 25.5 KB |
| intermediate/operation_candidates.json | config | 9.2 KB |
| references/R1-source-evidence.md | markdown | 7.8 KB |
README.md
View on GitHub# prompt-compilation skill
An **enhancement-overlay** coder skill that surfaces one decision the broad `[[dspy]]` library skill buries under
its API surface: **have you earned the right to run a prompt optimizer yet, and which one?**
**Version**: 0.1.0
**Layer**: enhance overlay (decision rubric only — defers all implementation to `[[dspy]]`, `[[agentsop-dspy]]`, `[[agentsop-metric-design]]`)
## What this skill answers
- **The compile-readiness gate**: two hard preconditions before any optimizer compute is spent —
1. a **metric** that exists *and* is human-validated (≥20 spot-checks), and
2. **enough labeled examples** for the optimizer you intend to run.
- **Which optimizer** by data scale: `LabeledFewShot` (<10) / `GEPA` (~10+ with textual feedback) /
`BootstrapFewShot` (~30–50) / `MIPROv2` (200+).
- **The GEPA inversion**: textual feedback (test diffs, schema violations, judge rationales) lets ~10 examples
suffice — the "you need 200 examples" rule is specific to scalar-feedback optimizers.
- **Cost discipline**: probe `auto="light"` first, never start at `heavy`, use a cheap optimizer LM, escalate only
on ≥2% lift, confirm on a held-out test set.
## What this skill does NOT do
It is **not** a DSPy tutorial and not a metric-construction guide. It defers:
- **`[[dspy]]`** — the library skill (HOW to write a Signature / module / `compile()`).
- **`[[agentsop-dspy]]`** — the full operating workflow (program → evaluate → optimize, save, deploy).
- **`[[agentsop-metric-design]]`** — building, decomposing, bias-probing, and calibrating the metric. This overlay only
*checks the metric exists and is validated*, then uses it as the gate.
This overlay is the sharpened "are you allowed to compile yet?" boundary inside `[[agentsop-dspy]]` §3 (Stage 2 → Stage 3).
## The mental model
> "Compile when you can measure. The optimizer maximizes your metric — garbage metric in, garbage prompt out."
An optimizer is a black-box search with no taste. It chases whatever the metric rewards, biases and all. So the
metric and the example count are *preconditions*, not tunables — hence a **gate**, not a step.
## Files
- `SKILL.md` — main skill, 7 sections: 何时激活 / 核心心智模型 (two-gate picture) / SOP (Gate 1 metric → Gate 2 data
→ pick → budget → probe → decide) / 操作模型 (8 ops + optimizer-by-data-scale table + readiness checklist + cost
budget) / 困境决策案例 (2: optimizer cost-vs-gain; GEPA inverts the data-scale assumption) / 反模式与边界 / 跨框架对照
(DSPy optimizers vs manual few-shot vs OpenAI fine-tuning).
- `references/R1-source-evidence.md` — every claim traced to the local `dspy-sop` skill and the upstream DSPy docs
it cites; overlap check vs `[[dspy]]` and `[[agentsop-metric-design]]`.
- `intermediate/operation_candidates.json` — the 8 operations in Trigger / Action / Output / Evidence form, plus the
optimizer table, dilemma cases, and anti-patterns.
## Quick start
To decide one pipeline today:
1. **Gate 1 — metric**: does a `metric(ex, pred) -> bool|float` exist, validated against a human on ≥20 spot-checks
at ≥80% agreement? No → STOP, route to `[[agentsop-metric-design]]`.
2. **Gate 2 — data**: count labeled examples; is it above the floor of the optimizer you want? (GEPA ~10 + textual
feedback / Bootstrap ~30–50 / MIPROv2 200+.) No → drop to a lower-floor optimizer or collect data.
3. Both gates pass → probe `auto="light"` with a cheap optimizer LM. Escalate only on ≥2% lift; confirm on a
held-out test set. Hand off the compile/deploy mechanics to `[[agentsop-dspy]]`.
## Source basis
- Primary: `/Users/5imp1ex/Desktop/Skill-Workplace/output/dspy-sop-skill/SKILL.md` — optimizers, Dilemma Case A
(optimizer cost vs gain), GEPA (~10 examples suffice with textual feedback), the three-stage gate, cost guardrails.
- Upstream (as cited by the primary): [dspy.ai/learn/optimization/overview/], [dspy.ai/learn/optimization/optimizers/],
[dspy.ai/api/optimizers/GEPA/overview/], [dspy.ai/faqs/], [arxiv.org/abs/2507.19457] (GEPA), [arxiv.org/abs/2310.03714].
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |