cskwork/supergoal-skill / skills/supergoal
supergoal
UnverifiedRoute objective-driven development, verification, planning, "teach/explain", codebase learning, and skill evaluation through the Supergoal workflow. Use when Supergoal is requested or an autonomous delivery workflow is needed.
templates/harness-eval-cases/fixtures/README.md
View on GitHub# Runnable harness-eval fixtures
Clean-slate, dependency-free fixtures the harness eval can run directly (the RevFactory `.yaml`
files in the parent dir are specs; these are the executable cases). Each is `node --test`-runnable
and ships its visible + hidden suites. Validated to DISCRIMINATE before use (see each case below).
| fixture | tier | starter state | what the hidden suite catches |
|---|---|---|---|
| `revfactory-case-002-async-race/` | hard | visible pass, 2/5 hidden fail (the race) | in-flight dedupe, no over-serialization, concurrent-reject, no caching of failures |
| `revfactory-case-003-refactoring/` | medium | passes ALL (measures preservation) | every coupon/tax/shipping/VIP-floor/rounding edge survives the refactor |
| `underspec-001-deepmerge/` | under-specified | stub throws (greenfield) | deep merge, **prototype-pollution guard**, null source, type replacement |
| `underspec-002-csvline/` | under-specified | stub throws (greenfield) | quoted commas, escaped `""`, empty/trailing fields, quoted spaces |
| `underspec-003-authz-cache/` | under-specified hard-low-effort | visible pass, 1/8 hidden pass | tenant/user/action/version authz leaks, denied-decision staleness, in-flight dedupe |
## Discrimination property (validated)
- bug-fix (002): starter passes visible, fails the planted hidden; a correct fix passes all.
- refactor (003): starter passes ALL; a behavior-breaking refactor regresses a hidden check.
- greenfield (u1/u2): stub fails all; a reference impl passes all; a lazy impl
(`{...t,...s}` / `split(',')`) fails the discriminating hidden checks.
- authz-cache (u3): starter and lazy impl pass visible 3/3 but hidden 1/8; reference passes hidden 8/8.
Re-validate any case with the no-codex path in its runner:
SG_EVAL_VALIDATE=1 SG_EVAL_CASE=<002|003|u1|u3> node templates/harness-eval-cases/run-local-eval.mjs
SG_EVAL_VALIDATE=1 SG_EVAL_CASE=<u1|u2> node templates/harness-eval-cases/run-underspec-n3.mjs
## Runners
- 002, 003, u1, u3: `templates/harness-eval-cases/run-local-eval.mjs`
- u1, u2 (+ equal-compute naive-loop arm): `templates/harness-eval-cases/run-underspec-n3.mjs`
Both runners originated in the 2026-06-07 experiments (medium-hard skill-vs-baseline and
underspecified-n3); the raw experiment dirs were removed 2026-07-06 - conclusions live in
`docs/experiments/README.md`.
Authored case specs (yaml) for the under-spec cases live in `../authored/`. The RevFactory case
specs stay at the corpus root; `revfactory-case-002/003` specs point here via `runnable_fixture:`.
## Key result (2026-06-07, gpt-5.5/low, n=3)
Only `underspec-001-deepmerge` separated baseline from harness: the baseline shipped a
prototype-pollution vuln as a false-GREEN; the harness critic caught it (3/3 seeds). All other
cases tied. See `docs/experiments/README.md` and `reference/harness-eval.md`.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Unverified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |