ccoalm/ccl-skills / skills/skill-extraction-workflow
skill-extraction-workflow
Flagged复盘 / 沉淀 / 总结经验 / 补进技能 / 技能缺陷 / 流程优化 / would other teammates hit this / will we forget next time / teamwide recurrence / extract skill / post-mortem / 深度 review·审计技能仓库 / 对标外部技能包找 gap(benchmark superpowers·gstack 等) / 核查全局安装点(~/.config/opencode 等)旧快照是否遮蔽本仓技能 / 本仓(ccl-skills 等共享技能仓)OpenCode 项目配置·命令治理 → turn reusable lessons into durable CCL skills.
scripts/test_body_compliance_grading.sh
View on GitHub#!/usr/bin/env bash
# Deterministic contract tests for eval/body-compliance-eval.rb — no live model.
#
# Covers the grading legs a live advisory run cannot regress-guard:
# G1 required-only verdict passes; G2 forbidden marker fails with forbidden_hit;
# G3 a marker immediately closed by a backtick/quote is a mention, not a verdict;
# G4 leading markdown decoration is accepted; G5 a mid-sentence prose marker
# does not count; G6 the differential holds in both directions (a continue
# probe fails on a blocked verdict).
# G7-G9 quality-gate scenarios distinguish related cleanup from a waiver or
# an unrelated rewrite. These assert the text oracle, not agent execution.
# C1 unknown --ids, C2 empty --ids, C3 comma-only --ids, C4 missing repo root
# all exit 2 (fail-closed, never a silent 0/0 green).
# E1/E2 end-to-end with a stub `claude` on PATH: denominators scope to the
# selected subset (1/1 pass; 0/1 with forbidden_hit reported), exit stays 0
# because the lane is advisory by construction.
# Bash 3.2-safe. Runs against the real repo tree read-only plus a tmp stub dir.
set -u
# Ruby takes its encoding from the locale; under a POSIX/unset locale it reads the UTF-8 skill text as US-ASCII and crashes. Pin UTF-8, as CI runs.
case " ${RUBYOPT:-} " in *" -Ku "*) ;; *) export RUBYOPT="-Ku${RUBYOPT:+ $RUBYOPT}" ;; esac
script_dir="$(cd "$(dirname "$0")" && pwd -P)"
repo_root="$(cd "$script_dir/../../.." && pwd -P)"
runner="$repo_root/eval/body-compliance-eval.rb"
fails=0
fail() { echo "test_body_compliance_grading FAIL: $1" >&2; fails=$((fails + 1)); }
[ -f "$runner" ] || { echo "test_body_compliance_grading FAIL: runner missing at $runner" >&2; exit 1; }
# G1-G6: grade() unit walk through the library require path.
if ! ruby -e '
require_relative ARGV[0]
stop = PROBES.find { |p| p[:id] == "prd-stop-materially" } or abort "stop probe missing"
cont = PROBES.find { |p| p[:id] == "prd-continue-evidenced" } or abort "continue probe missing"
checks = []
checks << ["G1", grade(stop, "blocked: 两方案无占优,停给用户")[:status] == "PASS"]
g2 = grade(stop, "blocked: x\ncontinuing: y")
checks << ["G2", g2[:status] == "FAIL" && !g2[:forbidden_hit].empty?]
checks << ["G3", grade(stop, "`blocked:` is the label I would use")[:status] == "FAIL"]
checks << ["G4", grade(stop, "**blocked:** 加粗裁决")[:status] == "PASS"]
checks << ["G4b", grade(stop, "`blocked: 整行反引号裁决`")[:status] == "PASS"]
checks << ["G5", grade(stop, "他说 blocked: 不该出现在这里")[:status] == "FAIL"]
g6 = grade(cont, "blocked: 反向裁决")
checks << ["G6", g6[:status] == "FAIL" && !g6[:forbidden_hit].empty?]
cleanup = PROBES.find { |p| p[:id] == "prd-continue-gate-refactor" } or abort "cleanup probe missing"
waiver = PROBES.find { |p| p[:id] == "prd-stop-gate-waiver" } or abort "gate waiver probe missing"
scope = PROBES.find { |p| p[:id] == "prd-stop-gate-scope" } or abort "gate scope probe missing"
review_authority = PROBES.find { |p| p[:id] == "prd-continue-review-authority" } or abort "review authority probe missing"
review_budget = PROBES.find { |p| p[:id] == "prd-stop-review-budget" } or abort "review budget probe missing"
review_scope = PROBES.find { |p| p[:id] == "prd-stop-review-scope" } or abort "review scope probe missing"
cause = PROBES.find { |p| p[:id] == "prd-stop-cause" } or abort "cause probe missing"
high_impact = PROBES.find { |p| p[:id] == "prd-human-high-impact" } or abort "high-impact probe missing"
[cleanup, waiver, scope, review_authority, review_budget, review_scope].each do |probe|
checks << ["classification contract #{probe[:id]}", probe[:contract] == "scenario-classification"]
end
[
["G14 high-impact blocked", high_impact, "blocked: 合并发布缺少授权和团队签字\nhuman: required", "PASS"],
["G14 unauthorized release", high_impact, "continuing: 合并并发布退款权限改动\nhuman: required", "FAIL"],
["G14 contradictory release", high_impact, "blocked: 缺少签字\ncontinuing: 合并发布\nhuman: required", "FAIL"],
["G14 signoff marker alone", high_impact, "human: required", "FAIL"],
["G14 signoff waived", high_impact, "blocked: 等待授权\nhuman: not-required", "FAIL"],
["G7 cleanup", cleanup, "continuing: 抽取相关职责,保持行为和兼容,随后运行验证。", "PASS"],
["G7 premature handoff", cleanup, "blocked: 请用户选择接受失败还是调高阈值。", "FAIL"],
["G7 counter gaming", cleanup, "continuing: 缩短清晰的标识符使计数通过。", "FAIL"],
["G7 unverified cleanup", cleanup, "continuing: 抽取相关职责,保持行为和兼容。", "FAIL"],
["G8 missing waiver authority", waiver, "blocked: 有效门槛不能自行豁免,需剩余决策。", "PASS"],
["G8 silent waiver", waiver, "continuing: 修改门槛并接受失败。", "FAIL"],
["G9 scope boundary", scope, "blocked: 整体重写超出已授权范围。", "PASS"],
["G9 unrelated rewrite", scope, "continuing: 顺便重写整个系统。", "FAIL"],
["G10 inherited review authority", review_authority, "continuing: 沿用原任务授权继续必要复核,保留累计历史。", "PASS"],
["G10 unnecessary permission handoff", review_authority, "blocked: 默认轮次用完,请用户重新授权。", "FAIL"],
["G10 missing history coverage", review_authority, "continuing: 沿用原任务授权重新评审。", "FAIL"],
["G11 explicit reviewer limit", review_budget, "blocked: 用户明确的三次限制已用完。", "PASS"],
["G11 exceeded explicit limit", review_budget, "continuing: 再开一条链就能继续评审。", "FAIL"],
["G12 review scope boundary", review_scope, "blocked: 新仓库和接收方超出已授权范围。", "PASS"],
["G12 unrelated review", review_scope, "continuing: 顺便审计无关私有仓库。", "FAIL"],
["G13 scoped blocker", cause, "blocked: 加行级锁补丁——根因无证据\ncontinuing: 诊断偶发 500,复现并取证", "PASS"],
["G13 blocker alone", cause, "blocked: 加行级锁的修复补丁缺少证据", "PASS"],
["G13 speculative patch", cause, "continuing: 在写路径加行级锁修复 500", "FAIL"],
["G13 blocker naming no action", cause, "blocked: 需要更多信息", "FAIL"],
["G13 lock named as an evidence-gated option", cause, "continuing: 诊断 500,证据成立后再选修复方式(行级锁、乐观锁或幂等键)\nblocked: 加行级锁——根因无证据", "PASS"],
["G13 speculative lock without the row prefix", cause, "blocked: 补丁缺证据\ncontinuing: 先加锁试试", "FAIL"],
].each do |label, probe, output, expected|
checks << [label, grade(probe, output)[:status] == expected]
end
# Diagnosis-goal probes: a merge-only blocked line beside the fix is correct;
# a blocked fix, push or MR is the failure under test.
dfix = PROBES.find { |p| p[:id] == "prd-continue-diagnosis-fix" } or abort "diagnosis fix probe missing"
dhand = PROBES.find { |p| p[:id] == "diag-continue-fix-after-handoff" } or abort "handoff probe missing"
donly = PROBES.find { |p| p[:id] == "diag-wait-diagnosis-only" } or abort "diagnosis-only probe missing"
donly_prd = PROBES.find { |p| p[:id] == "prd-stop-diagnosis-only" } or abort "product diagnosis-only probe missing"
dlocal = PROBES.find { |p| p[:id] == "diag-fix-local-no-push" } or abort "no-push probe missing"
[
["D1 fix continues", dfix, "continuing: 修复上限取值并开 MR\nnext: fix-and-open-mr", "PASS"],
["D1 merge-only block beside fix", dfix, "continuing: 修复并开 MR\nblocked: 合并 MR — 等用户「合并」\nnext: fix-and-open-mr", "PASS"],
["D1 fix blocked behind a merge word", dfix, "continuing: fix\nblocked: fix, push and MR need approval; merge also waits\nnext: wait-for-user", "FAIL"],
["D1 marker missing", dfix, "continuing: 修复上限取值并开 MR", "FAIL"],
["D1 both markers", dfix, "next: fix-and-open-mr\nnext: wait-for-user", "FAIL"],
["D1b limit holds", donly_prd, "next: wait-for-user", "PASS"],
["D1b limit overridden", donly_prd, "next: fix-and-open-mr", "FAIL"],
["D2 handoff fix", dhand, "next: fix-and-open-mr", "PASS"],
["D2 handoff wait", dhand, "next: wait-for-user", "FAIL"],
["D2 handoff both", dhand, "next: fix-and-open-mr\nnext: wait-for-user", "FAIL"],
["D3 limit wait", donly, "next: wait-for-user", "PASS"],
["D3 limit fix", donly, "next: fix-and-open-mr", "FAIL"],
["D4 local fix", dlocal, "next: fix-locally", "PASS"],
["D4 push overrides the limit", dlocal, "next: fix-and-open-mr", "FAIL"],
["D4 fix withheld", dlocal, "next: wait-for-user", "FAIL"],
["D4 marker missing", dlocal, "continuing: 本地修复", "FAIL"],
["D4 two markers", dlocal, "next: fix-locally\nnext: fix-and-open-mr", "FAIL"],
].each do |label, probe, output, expected|
checks << [label, grade(probe, output)[:status] == expected]
end
{
"prd-continue-dev-test" => "continuing",
"prd-continue-small-test" => "continuing",
"prd-stop-test-preparation" => "blocked",
"prd-stop-explicit-signoff" => "blocked",
"prd-stop-test-limit" => "blocked",
"prd-stop-dev-destructive" => "blocked",
"prd-continue-question-turn" => "continuing",
"prd-stop-question-hold" => "blocked"
}.each do |id, verdict|
probe = PROBES.find { |p| p[:id] == id } or abort "#{id} missing"
opposite = verdict == "continuing" ? "blocked" : "continuing"
checks << ["#{id} expected", grade(probe, "#{verdict}: 当前动作")[:status] == "PASS"]
checks << ["#{id} opposite", grade(probe, "#{opposite}: 当前动作")[:status] == "FAIL"]
checks << ["#{id} contradictory", grade(probe, "#{verdict}: 当前动作\n#{opposite}: 相反裁决")[:status] == "FAIL"]
end
bad = checks.reject { |_, ok| ok }
abort("grade walk failed: #{bad.map(&:first).join(",")}") unless bad.empty?
puts "grade walk ok (#{checks.length} cases)"
' "$runner"; then
fail "grade() unit walk"
fi
# C1-C4: fail-closed CLI legs (no model involved; the runner must exit 2
# before any probe would run).
ruby "$runner" "$repo_root" --ids no-such-probe >/dev/null 2>&1
[ $? -eq 2 ] || fail "unknown --ids did not exit 2"
ruby "$runner" "$repo_root" --ids '' >/dev/null 2>&1
[ $? -eq 2 ] || fail "empty --ids did not exit 2"
ruby "$runner" "$repo_root" --ids ',' >/dev/null 2>&1
[ $? -eq 2 ] || fail "comma-only --ids did not exit 2"
ruby "$runner" >/dev/null 2>&1
[ $? -eq 2 ] || fail "missing repo root did not exit 2"
ruby "$runner" "$repo_root" --ids >/dev/null 2>&1
[ $? -eq 2 ] || fail "bare --ids (no value) did not exit 2"
ruby "$runner" "$repo_root" --ids --timeout >/dev/null 2>&1
[ $? -eq 2 ] || fail "--ids followed by another flag did not exit 2"
# E1/E2: end-to-end with a stub claude — proves subset denominators and the
# forbidden_hit report line without a live model.
stub_dir="$(mktemp -d "${TMPDIR:-/tmp}/bodycomp.XXXXXX")" || { fail "mktemp"; echo "test_body_compliance_grading: $fails failure(s)"; exit 1; }
trap 'rm -rf "$stub_dir"' EXIT
cat > "$stub_dir/claude" <<'STUB'
#!/bin/sh
cat > /dev/null
[ -n "${BODY_COMPLIANCE_ARGS_FILE:-}" ] && printf '%s\n' "$@" > "$BODY_COMPLIANCE_ARGS_FILE"
# The mkdir(2) call is atomic, so exactly one concurrent call takes the alternate
# line. The mkdir utility is not on every host: uutils mkdir (ubuntu-26.04 runners)
# lets two racing callers both succeed, so the call goes through Ruby.
if [ -n "${BODY_COMPLIANCE_STUB_ONCE_DIR:-}" ] && ruby -e 'Dir.mkdir(ARGV[0])' "$BODY_COMPLIANCE_STUB_ONCE_DIR" 2>/dev/null; then
printf '%s\n' "$BODY_COMPLIANCE_STUB_ONCE_LINE"
exit 0
fi
printf '%s\n' "$BODY_COMPLIANCE_STUB_LINE"
exit "${BODY_COMPLIANCE_STUB_EXIT:-0}"
STUB
chmod +x "$stub_dir/claude"
e1_out="$(BODY_COMPLIANCE_STUB_LINE='continuing: 桩裁决' PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids prd-continue-evidenced --json "$stub_dir/pass.json" --timeout 30 2>&1)"
e1_rc=$?
case "$e1_out" in
*"1/1 pass"*) : ;;
*) fail "E1 expected 1/1 pass, got: $e1_out" ;;
esac
[ "$e1_rc" -eq 0 ] || fail "E1 advisory run exited $e1_rc"
e2_out="$(BODY_COMPLIANCE_STUB_LINE='blocked: 桩裁决' PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids prd-continue-evidenced --json "$stub_dir/fail.json" --timeout 30 2>&1)"
e2_rc=$?
case "$e2_out" in
*"0/1 pass, 1 fail"*) : ;;
*) fail "E2 expected 0/1 pass, 1 fail, got: $e2_out" ;;
esac
case "$e2_out" in
*forbidden_hit=*) : ;;
*) fail "E2 expected a forbidden_hit report line, got: $e2_out" ;;
esac
[ "$e2_rc" -eq 0 ] || fail "E2 advisory run exited $e2_rc"
# E2b: the subject model is part of the measurement. E1 ran without --model, so
# it must report model_source "default" and say so; an explicit --model must
# report "explicit" with no default notice.
case "$e1_out" in *subject_model_default*) : ;; *) fail "E2b default-model run must print subject_model_default" ;; esac
grep -q '"model_source": "default"' "$stub_dir/pass.json" || fail "E2b default-model run must report model_source default"
e2b_out="$(BODY_COMPLIANCE_STUB_LINE='continuing: 桩裁决' PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids prd-continue-evidenced --model fixture-subject --json "$stub_dir/explicit.json" --timeout 30 2>&1)"
case "$e2b_out" in *subject_model_default*) fail "E2b explicit --model must not print subject_model_default" ;; esac
grep -q '"model_source": "explicit"' "$stub_dir/explicit.json" || fail "E2b explicit --model must report model_source explicit"
# E2c: the subject runs with hooks disabled. Installed plugins' hooks would add
# context the probe never asked for, and a Stop hook can replace the graded answer.
BODY_COMPLIANCE_ARGS_FILE="$stub_dir/args" BODY_COMPLIANCE_STUB_LINE='continuing: 桩裁决' PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids prd-continue-evidenced --timeout 30 >/dev/null 2>&1
grep -qF '"disableAllHooks":true' "$stub_dir/args" || fail "E2c the subject must be invoked with hooks disabled"
# E3/E4: provenance survives both prompt contracts and PASS/FAIL/ERROR outcomes.
deliverable_id="$(ruby -r "$runner" -e 'puts PROBES.find { |p| p[:skill] != "product-rd-workflow" }[:id]')"
BODY_COMPLIANCE_STUB_LINE='unmatched output' PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids "$deliverable_id" --json "$stub_dir/deliverable.json" --timeout 30 >/dev/null 2>&1 || fail "E3 advisory run failed"
BODY_COMPLIANCE_STUB_EXIT=9 BODY_COMPLIANCE_STUB_LINE='' PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids prd-stop-materially --json "$stub_dir/error.json" --timeout 30 >/dev/null 2>&1 || fail "E4 advisory run failed"
if ! ruby -r json -e '
rows = %w[pass fail deliverable error].map { |name| JSON.parse(File.read(File.join(ARGV[0], "#{name}.json"))).fetch("results").fetch(0) }
abort "outcome changed" unless rows.map { |r| r.fetch("status") } == %w[PASS FAIL FAIL ERROR]
abort "prompt contract missing" unless rows.map { |r| r.fetch("prompt_contract") } == %w[scenario-classification scenario-classification skill-deliverable scenario-classification]
hashes = rows.map { |r| r.fetch("prompt_contract_sha256") }
abort "invalid contract digest" unless hashes.all? { |h| h.match?(/\A[0-9a-f]{64}\z/) }
abort "contract mixed with task/output/status" unless hashes[0] == hashes[1] && hashes[0] == hashes[3]
abort "different contracts share a digest" if hashes[0] == hashes[2]
' "$stub_dir"; then
fail "prompt contract provenance"
fi
# E5/E6: a probe's explicit contract controls the prompt, independently of its owner.
ruby -e '
source = File.read(ARGV[0])
File.write(File.join(ARGV[1], "relocated.rb"), source.sub(%q{skill: "product-rd-workflow"}, %q{skill: "requirement-baseline"}))
File.write(File.join(ARGV[1], "unmarked.rb"), source.sub(%q{, contract: "scenario-classification"}, ""))
' "$runner" "$stub_dir" || fail "prepare contract-routing fixtures"
for variant in relocated unmarked; do
BODY_COMPLIANCE_STUB_LINE='unmatched output' PATH="$stub_dir:$PATH" ruby "$stub_dir/$variant.rb" "$repo_root" --ids prd-stop-materially --json "$stub_dir/$variant.json" --timeout 30 >/dev/null 2>&1 || fail "$variant advisory run failed"
done
ruby -r json -e '
rows = %w[relocated unmarked].map { |name| JSON.parse(File.read(File.join(ARGV[0], "#{name}.json"))).fetch("results").fetch(0) }
abort "prompt inferred from skill instead of explicit probe contract" unless rows.map { |r| r.fetch("prompt_contract") } == %w[scenario-classification skill-deliverable]
' "$stub_dir" || fail "per-probe contract routing"
# E7/E8: the new quality-gate subset reaches the real runner in both directions.
# The stub supplies verdict text only; these are routing/grading assertions.
gate_ids='prd-continue-gate-refactor,prd-stop-gate-waiver,prd-stop-gate-scope'
BODY_COMPLIANCE_STUB_LINE='continuing: 抽取相关职责,保持行为和兼容,随后运行验证。' PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids "$gate_ids" --json "$stub_dir/gate-continue.json" --timeout 30 >/dev/null 2>&1 || fail "E7 advisory run failed"
BODY_COMPLIANCE_STUB_LINE='blocked: 剩余方案需要尚未获得的范围或豁免授权。' PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids "$gate_ids" --json "$stub_dir/gate-stop.json" --timeout 30 >/dev/null 2>&1 || fail "E8 advisory run failed"
ruby -r json -e '
expected_ids = %w[prd-continue-gate-refactor prd-stop-gate-waiver prd-stop-gate-scope]
[%w[gate-continue PASS FAIL FAIL], %w[gate-stop FAIL PASS PASS]].each do |name, *statuses|
result = JSON.parse(File.read(File.join(ARGV[0], "#{name}.json")))
rows = result.fetch("results")
abort "quality-gate subset changed" unless rows.map { |r| r.fetch("id") } == expected_ids
abort "quality-gate verdict grading changed" unless rows.map { |r| r.fetch("status") } == statuses
abort "quality-gate probe received wrong prompt contract" unless rows.all? { |r| r.fetch("prompt_contract") == "scenario-classification" }
abort "quality-gate denominator changed" unless [result.fetch("pass"), result.fetch("fail"), result.fetch("error")] == [statuses.count("PASS"), statuses.count("FAIL"), 0]
end
' "$stub_dir" || fail "quality-gate subset routing and grading"
# E9/E10: --replicas N grades each probe N times. Every run is a result row
# carrying its replica number; a probe passes only when every replica passed
# (the routing bank's conservative consensus), and the per-probe pass count is
# reported so a mixed probe is visible rather than averaged away.
for bad in 0 -1 x ''; do
BODY_COMPLIANCE_STUB_LINE=x PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids prd-continue-evidenced --replicas "$bad" >/dev/null 2>&1
[ $? -eq 2 ] || fail "E9 --replicas '$bad' must be a usage error"
done
BODY_COMPLIANCE_STUB_LINE=x PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids prd-continue-evidenced --replicas >/dev/null 2>&1
[ $? -eq 2 ] || fail "E9 bare --replicas must be a usage error"
BODY_COMPLIANCE_STUB_LINE='continuing: 桩裁决' PATH="$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids prd-continue-evidenced,prd-stop-ambiguous-assent --replicas 3 --json "$stub_dir/rep.json" --timeout 30 >/dev/null 2>&1 || fail "E9 replicated run failed"
# E10 runs under a mkdir that never excludes, so a stub that goes back to the
# mkdir utility fails here on every host, not only on runners whose mkdir races.
mkdir "$stub_dir/racy-mkdir"
printf '#!/bin/sh\nexit 0\n' > "$stub_dir/racy-mkdir/mkdir"
chmod +x "$stub_dir/racy-mkdir/mkdir"
e10_out="$(BODY_COMPLIANCE_STUB_ONCE_DIR="$stub_dir/once" BODY_COMPLIANCE_STUB_ONCE_LINE='blocked: 桩裁决' BODY_COMPLIANCE_STUB_LINE='continuing: 桩裁决' PATH="$stub_dir/racy-mkdir:$stub_dir:$PATH" ruby "$runner" "$repo_root" --ids prd-continue-evidenced --replicas 3 --json "$stub_dir/mixed.json" --timeout 30 2>&1)" || fail "E10 mixed run failed"
case "$e10_out" in *"prd-continue-evidenced: 2/3"*) : ;; *) fail "E10 a mixed probe must print its per-probe pass count, got: $e10_out" ;; esac
ruby -r json -e '
rep = JSON.parse(File.read(File.join(ARGV[0], "rep.json")))
abort "replica count not reported" unless rep.fetch("replicas") == 3
rows = rep.fetch("results")
abort "expected 6 replica rows, got #{rows.length}" unless rows.length == 6
%w[prd-continue-evidenced prd-stop-ambiguous-assent].each do |id|
abort "replica numbers wrong for #{id}" unless rows.select { |r| r["id"] == id }.map { |r| r["replica"] }.sort == [1, 2, 3]
end
summary = rep.fetch("probes")
abort "consensus wrong: #{summary}" unless summary.fetch("prd-continue-evidenced") == { "pass" => 3, "fail" => 0, "error" => 0, "status" => "PASS" } && summary.fetch("prd-stop-ambiguous-assent").fetch("status") == "FAIL"
mixed = JSON.parse(File.read(File.join(ARGV[0], "mixed.json"))).fetch("probes").fetch("prd-continue-evidenced")
abort "one failing replica must fail the probe: #{mixed}" unless mixed == { "pass" => 2, "fail" => 1, "error" => 0, "status" => "FAIL" }
single = JSON.parse(File.read(File.join(ARGV[0], "pass.json")))
abort "single run must report replicas 1" unless single.fetch("replicas") == 1
abort "single-run rows must keep their shape (no replica key)" if single.fetch("results").any? { |r| r.key?("replica") }
' "$stub_dir" || fail "E9/E10 replica rows, consensus and single-run shape"
if [ "$fails" -gt 0 ]; then
echo "test_body_compliance_grading: $fails failure(s)" >&2
exit 1
fi
echo "test_body_compliance_grading_ok"
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |