Skip to content

ash1794/vibe-engineering / plugins/vibe-engineering/skills/vibe-flake-root-cause

vibe-flake-root-cause

Verified

Refuses "flaky" as a diagnosis. Turns an intermittent failure into a reproduction by isolating it from the environment, naming the variable that controls it, forcing that variable, and measuring before and after with enough runs to count. Separates real defects from environment contention with evidence, and never retries to green or widens a threshold. Use when a test, performance check, or deploy step fails intermittently or passes on rerun.

vibe-flake-root-cause

"Flaky" ends an investigation without finding anything, and a retry-to-green hides real defects. An intermittent failure is a hypothesis: some variable you haven't controlled decides the outcome. Find it, force it, and prove the fix with counts.

When to Use This Skill

  • "It passed on rerun"
  • A failure that shows up only in CI
  • A threshold (performance, visual diff, timeout) that fails on some runs
  • You're tempted to add a retry, raise a timeout, or loosen a tolerance

When NOT to Use This Skill

  • The failure reproduces every time (debug it directly)
  • Race conditions visible in the test code itself (use vibe-concurrent-test-safety)
  • The job died before any test ran (checkout, install, runner loss); a single re-run is fine there, but say so

Steps

  1. Record the failure exactly. Commit, environment, profile or device, the failing value, and the threshold. Keep the log.

  2. Run it alone, cleanly. If it passes in isolation, treat it as contention until proven otherwise, and name the resource: CPU, a shared port, a shared database, disk, or network. Record machine load alongside every timing measurement.

  3. List candidate variables and force each in turn. Common ones:

    • timing: network latency, font or asset arrival, animation frames
    • environment: CPU load, parallel agents or tests, shared ports and servers
    • data: ordering, time of day, random seeds, page or payload size
    • process mode: a server started in a mode the framework doesn't support in production
  4. Reproduce at a stated rate before changing anything (for example "10 of 15 runs fail with fonts held back 600 ms"). No reproduction means no root cause yet, so keep looking or report it as unresolved.

  5. Fix the cause, not the symptom. Then rerun the forced condition N times and report the new rate (for example "0 of 240 loads across 4 routes").

  6. Disclose any mitigation. If you clip, retry, or quarantine anything, say what, why, and why it doesn't hide a defect. Never widen a threshold or budget to pass (vibe-anti-rationalization-check).

Output Format

Flake Investigation: [test/check]

Symptom: [value vs threshold, where, how often] Isolated run: pass / fail · Load at failure: [value]

VariableForced howFailure rate
font arrivalheld back 600 ms10/15
CPU loadrun alone0/15

Root cause: [mechanism, in one or two sentences] Fix: [change] · After: [0/N under the forced condition] Mitigations disclosed: [none / list with reasons]

finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 319kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 319kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 319kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 196kUpdated Claude CodeCodexVerifiedInstall