odinfree/storm-ecdsa-harness / skills/evaluator-scoring-surface-audit
evaluator-scoring-surface-audit
VerifiedRead a benchmark's evaluator, verifier and scoring source line by line for gaps between what is counted, what is checked and what is only assumed, then turn each gap into a priced lever, a tool, or a report to the organisers. Use when joining a benchmark, after any evaluator change, or when the leaderboard moves in a way your cost model does not explain.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 6.6 KB |
| assets/surface-table.md | markdown | 1.4 KB |
assets/surface-table.md
View on GitHub# Scoring-surface audit: <benchmark>
Evaluator commit: `<sha>`; verifier commit: `<sha>`; official runner script: `<path>`; date: `<yyyy-mm-dd>`
Score formula as coded: `<formula>` at `<file:line>`
## 1. Counted quantities
| # | quantity | file:line of the increment | increments when | unit | emitted or executed | notes |
|---|---|---|---|---|---|---|
| Q1 | | | | | | |
| Q2 | | | | | | |
## 2. Checks
| # | property checked | file:line | bound | sampling (all / sample of N / none) | on failure |
|---|---|---|---|---|---|
| C1 | | | | | |
| C2 | | | | | |
## 3. Written rules
| # | rule (quote) | enforced at file:line, or "not enforced" |
|---|---|---|
| R1 | | |
## 4. Randomness
| draw | seeded by | who can influence the seed | file:line |
|---|---|---|---|
| | | | |
## 5. Cross: gaps
| # | counted quantity or check | observation | class (exact lever / sampled lever / harness defect / intent conflict) | price (measured, with date) | action |
|---|---|---|---|---|---|
| G1 | | | | | |
| G2 | | | | | |
No gap found for: `<list the Q and C rows crossed with no finding>`
## 6. Second evaluator path
| item | local command | official runner |
|---|---|---|
| binary and flags | | |
| limits (time, memory, size) | | |
| dependency versions | | |
| differences found | | |
## 7. Reports sent
| date | to | gap | patch or test attached | reply |
|---|---|---|---|---|
| | | | | |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |