dvy1987/agent-loom / .agents/skills/experiment-backlog
experiment-backlog
VerifiedTurn assumptions, funnel opportunities, and product questions into a prioritised, feasibility-checked experiment backlog. Filters by traffic reality, metric latency, and method feasibility — not just ICE/RICE scoring. Maintains a living portfolio with status (idea → designed → running → readout → archived). Load when the user says "what should we test next", "build an experiment backlog", "prioritise our tests", "where should we experiment", "what's worth testing", or when the experimentation orchestrator routes here.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 9.2 KB |
| references/examples.md | markdown | 3.0 KB |
| references/prioritization-rubric.md | markdown | 4.9 KB |
references/examples.md
View on GitHub# Experiment Backlog — Full Worked Examples
Skill: `experiment-backlog` | Enriched from SKILL.md (improve-skills pass, SKIP_RESEARCH).
## Example 1 — Step-by-step execution
**Input:** "Run `experiment-backlog` on [concrete task]"
**Agent actions:**
1. Gather Candidates
2. Tag Each Candidate
3. Score with ICE + Feasibility
4. Apply the Funnel-ROI Map
5. Sort and Write the Backlog File
6. Hand Off
**Impact Report shape:**
```
Backlog updated: docs/experiments/backlog.md
Candidates added: N
Candidates rejected: M (reasons summarised)
Top 3 ready: [list with surface + hypothesis + ICE]
Funnel coverage: [acquisition: N | activation: N | engagement: N | monetisation: N | retention: N | referral: N]
Next recommended: [item — route to experiment-spec]
```
## Example 2 — Gotcha application
**Input:** Task hits a non-obvious edge case
**Apply:**
- **Feasibility is a binary gate, not an ICE multiplier.** A high-ICE item with insufficient traffic at planned MDE is REJECTED — not "deprioritised". Letting it sit on the backlog wastes attention and creates phantom queues.
- **ICE inflation.** Self-proposed ideas score themselves 8/9/8. Anchor scoring against past wins where you already know the lift size — recalibrate every quarter.
- **Retention A/Bs without a holdout are fake.** Retention metrics need a long-running holdout cohort. Reject any retention test that lacks one and surface "fix the holdout first" as a blocking dependency.
- **Don't replace the backlog file — append.** `docs/experiments/backlog.md` is a living portfolio with status. Replacing it loses the lifecycle history that downstream skills (especially `experiment-readout` learnings) depend on.
---
See `SKILL.md` for hard rules and verification checklist.
---
|---|
| Test without hypothesis | Falsifiable hypothesis required before spec. |
| Peek until significant | Peek policy must be pre-committed in spec. |
| Any metric goes | Primary + guardrail metrics defined up front. |
| Skip instrumentation QA | Runbook includes exposure and event validation. |
## Example 3 — Gotcha application
**Input:** Task hits a non-obvious edge case
**Apply:**
- **Feasibility is a binary gate, not an ICE multiplier.** A high-ICE item with insufficient traffic at planned MDE is REJECTED — not "deprioritised". Letting it sit on the backlog wastes attention and creates phantom queues.
- **ICE inflation.** Self-proposed ideas score themselves 8/9/8. Anchor scoring against past wins where you already know the lift size — recalibrate every quarter.
- **Retention A/Bs without a holdout are fake.** Retention metrics need a long-running holdout cohort. Reject any retention test that lacks one and surface "fix the holdout first" as a blocking dependency.
- **Don't replace the backlog file — append.** `docs/experiments/backlog.md` is a living portfolio with status. Replacing it loses the lifecycle history that downstream skills (especially `experiment-readout` learnings) depend on.
---
See `SKILL.md` for hard rules and verification checklist.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |