cskwork/supergoal-skill / skills/supergoal
supergoal
UnverifiedRoute objective-driven development, verification, planning, "teach/explain", codebase learning, and skill evaluation through the Supergoal workflow. Use when Supergoal is requested or an autonomous delivery workflow is needed.
templates/harness-eval-case.yaml
View on GitHubid: case-001
title: "Short task title"
difficulty: simple
case_type: coding # coding | routing
source: "<pilot, rerun, upstream URL, or repo-specific>"
runtime_adapter: portable
repo_snapshot: "<git sha or immutable snapshot id>"
default_coding_ab:
applies_when: "coding harness eval requested without named cases"
cases:
- fixture: "revfactory-case-002-async-race/"
spec: "revfactory-case-002-bug-fix.yaml"
mode: "DEBUG/concurrency"
reason: "hard runnable fixture with visible-pass hidden-fail starter"
- fixture: "revfactory-case-003-refactoring/"
spec: "revfactory-case-003-refactoring.yaml"
mode: "LEGACY/brownfield preservation"
reason: "runnable preservation fixture whose hidden suite catches behavior drift"
excluded_from_default:
- "underspec-001-deepmerge/"
- "underspec-002-csvline/"
- "underspec-003-authz-cache/"
tie_rule: "If both default cases tie, report Not proven and require authored expert runnable fixtures."
external_task:
applies: false
benchmark: "DeepSWE"
benchmark_url: "https://deepswe.datacurve.ai/"
benchmark_repo: "https://github.com/datacurve-ai/deep-swe"
benchmark_ref: "<immutable benchmark commit>"
task_id: "etree-xml-diff-patch"
task_url: "https://deepswe.datacurve.ai/data/v1/tasks/etree-xml-diff-patch"
upstream_repo: "https://github.com/beevik/etree"
base_commit: "4032e04c8f2e2f35e43ce5d772fcef14a5df4d74"
language: go
category: feature_request
stop_policy:
timeout_seconds: 900
manual_interrupt: invalid_paired_correctness
capture_patch_on_budget_timeout: true
base_task_text_sha256: "<sha256 of original instruction.md>"
harness_source_sha256:
- path: "SKILL.md"
sha256: "<sha256>"
verifier_artifacts:
- "verifier/reward.json"
- "verifier/ctrf.json"
- "verifier/test-stdout.txt"
- "verifier/run.log"
pass_rate_lift:
baseline_pass_rate: 0
harness_pass_rate: 0
absolute_percentage_points: 0
relative_percent: 0
task: |
Exact task text. Reuse this text unchanged for baseline and harness runs.
eval_intent:
goal: "User objective in their own terms, not a diff summary."
constraints:
- "Constraint learned from the conversation or repo rules."
tradeoffs:
- "Decision that affects review judgment."
rejected_approaches:
- "Alternative considered and rejected."
acceptance:
- "Machine-checkable expectation"
- "Human-review expectation"
routing_probe:
applies: false
train_test_split: "60/40"
trials_per_prompt: 3
prompts:
- id: route-001
split: train
expected: should_trigger
text: "/supergoal fix the failing checkout test"
- id: route-002
split: heldout
expected: should_not_trigger
text: "Rewrite this sentence more politely"
machine_checks:
- test
- lint
- build
command_manifest:
- name: test
command: "npm test"
source: frozen_repo
used_by: both
verifies: "visible and hidden acceptance tests"
hidden_checks:
- "Hidden or evaluator-only check the run must pass"
bug_catch_matrix:
planted_or_hidden_bugs:
- "Bug or hidden check the run must catch before shipping"
expected_false_green_risk: "What would count as ship/green while still wrong?"
extra_bug_discovery: "How to record genuine bugs found outside the planted set"
regression_protection:
high_risk_fixed_red: "none, or security/data/concurrency/auth class"
permanent_test_required: true
exception_allowed_only_with_reason: true
decision_gates:
- id: r1
action: auto-fix
status: resolved
description: "Mechanical finding surfaced by the gate."
recheck: "Command proving the fix."
adapter_fixture_replay:
status: not_required
adapter_event_schema: "portable-result-json-v1"
fixtures: []
redaction: "not required"
replay_command: "not required"
rubric:
quality_score: "RevFactory-style 100-point score across 10 dimensions, each 0-10."
correctness: "Does the result meet the task without known bugs?"
tests: "Are relevant regression or acceptance tests present?"
architecture: "Is the structure appropriate for the task size?"
maintainability: "Can a future agent or engineer extend it safely?"
efficiency: "Was overhead justified by quality?"
cost_fields:
- wall_clock_minutes
- duration_ms
- model
- reasoning_effort
- tool_calls
- total_tokens
- token_estimate
persist_path: "docs/changelog/<YYYY-MM>/<DD-harness-eval>/cases/<id>.yaml or .domain-agent/qa/harness-eval/<id>.yaml"
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |