Skip to content

ZeKaiNie/universal-examprep-skill / full

universal-exam-cram-coach-full

Verified

帮助学生在临考前进行结构化极速复习:解析课程资料/大纲/重点,按章节生成 wiki 知识库与标准题库,组织针对性刷题与判分,并记录复习进度和错题。当用户即将考试、需要快速复习计划、练习题、错题复盘或考前小抄时使用(关键词:期末/备考/复习/刷题/划重点/错题;exam, cram, study plan, quiz, review)。不适用于长期学习规划、与考试无关的写作或编程任务。

FileKindSize
.gitattributesother1.5 KB
.github/workflows/ci.ymlconfig1.9 KB
.gitignoreother659 B
AGENTS.mdmarkdown38.8 KB
CHANGELOG.en.mdmarkdown25.5 KB
CHANGELOG.mdmarkdown20.5 KB
CONTRIBUTING.mdmarkdown5.1 KB
CONTRIBUTING.zh.mdmarkdown4.9 KB
LICENSEother1.0 KB
README.mdmarkdown24.5 KB
README.zh.mdmarkdown22.8 KB
SKILL.mdskill md10.4 KB
assets/exam-panic.pngbinary934.2 KB
benchmark/.gitignoreother1.2 KB
benchmark/README.en.mdmarkdown12.7 KB
benchmark/README.mdmarkdown10.8 KB
benchmark/REPORT.en.mdmarkdown7.1 KB
benchmark/REPORT.mdmarkdown6.4 KB
benchmark/aggregate_matrix.pyscript16.3 KB
benchmark/behavior_smoke/README.en.mdmarkdown9.9 KB
benchmark/behavior_smoke/README.mdmarkdown7.4 KB
benchmark/behavior_smoke/fixtures/mini_course/references/assets/venn_prompt.svgother677 B
benchmark/behavior_smoke/fixtures/mini_course/references/quiz_bank.jsonconfig4.5 KB
benchmark/behavior_smoke/fixtures/mini_course/references/wiki/ch1_stack_queue.mdmarkdown447 B
benchmark/behavior_smoke/fixtures/mini_course/references/wiki/ch2_trees.mdmarkdown493 B
benchmark/behavior_smoke/fixtures/mini_course/study_plan.mdmarkdown550 B
benchmark/behavior_smoke/fixtures/mini_course/study_progress.mdmarkdown429 B
benchmark/behavior_smoke/fixtures/mini_course/study_state.jsonconfig390 B
benchmark/behavior_smoke/fixtures/mini_course_nostate/references/assets/venn_prompt.svgother677 B
benchmark/behavior_smoke/fixtures/mini_course_nostate/references/quiz_bank.jsonconfig3.2 KB
benchmark/behavior_smoke/fixtures/mini_course_nostate/references/wiki/ch1_stack_queue.mdmarkdown447 B
benchmark/behavior_smoke/fixtures/mini_course_nostate/references/wiki/ch2_trees.mdmarkdown493 B
benchmark/behavior_smoke/fixtures/mini_course_nostate/study_plan.mdmarkdown550 B
benchmark/behavior_smoke/fixtures/mini_course_nostate/study_progress.mdmarkdown429 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_chat_auto_pdf_bad.txtmarkdown433 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_chat_good.txtmarkdown462 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_chat_not_persisted_bad.txtmarkdown186 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_one_shot_good.txtmarkdown490 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_one_shot_persist_bad.txtmarkdown605 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_one_shot_probe_error_bad.txtmarkdown507 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_visual_native_good.txtmarkdown495 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_visual_preflight_failed_bad.txtmarkdown487 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_visual_subscription_guess_bad.txtmarkdown507 B
benchmark/behavior_smoke/mock/sample_outputs/hint_skip_offer.txtmarkdown349 B
benchmark/behavior_smoke/mock/sample_outputs/language_ask_good.txtmarkdown461 B
benchmark/behavior_smoke/mock/sample_outputs/language_ask_missing_line.txtmarkdown270 B
benchmark/behavior_smoke/mock/sample_outputs/language_ask_unglossed.txtmarkdown211 B
benchmark/behavior_smoke/mock/sample_outputs/language_persist_good.txtmarkdown216 B
benchmark/behavior_smoke/mock/sample_outputs/language_persist_noncanonical.txtmarkdown157 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_bilingual.txtmarkdown261 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_bilingual_explicit.txtmarkdown372 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_bilingual_negated.txtmarkdown306 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_question.txtmarkdown224 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_silent.txtmarkdown364 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_wrong_language.txtmarkdown262 B
benchmark/behavior_smoke/mock/sample_outputs/notebook_persist_chat_only.txtmarkdown952 B
benchmark/behavior_smoke/mock/sample_outputs/notebook_persist_good.txtmarkdown744 B
benchmark/behavior_smoke/mock/sample_outputs/progress_after_confusion.mdmarkdown395 B
benchmark/behavior_smoke/mock/sample_outputs/progress_after_mistake.mdmarkdown403 B
benchmark/behavior_smoke/mock/sample_outputs/provenance_answer.txtmarkdown492 B
benchmark/behavior_smoke/mock/sample_outputs/quiz_output_good.txtmarkdown351 B
benchmark/behavior_smoke/mock/sample_outputs/quiz_output_invented.txtmarkdown317 B
benchmark/behavior_smoke/mock/sample_outputs/resume_message.txtmarkdown224 B
benchmark/behavior_smoke/mock/sample_outputs/scope_override_good.txtmarkdown258 B
benchmark/behavior_smoke/mock/sample_outputs/scope_override_missing.txtmarkdown127 B
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_ai_answer_good.txtmarkdown1.2 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_formula_first.txtmarkdown1.1 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_good.txtmarkdown1.2 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_liberal_good.txtmarkdown1.4 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_missing_warn.txtmarkdown608 B
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_no_source.txtmarkdown1.1 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_optin_closers.txtmarkdown1.5 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_skip_ask.txtmarkdown948 B
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_unlabeled_source.txtmarkdown517 B
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_unsolicited_closers.txtmarkdown1.5 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_warn_title.txtmarkdown581 B
benchmark/behavior_smoke/mock/sample_outputs/time_budget_1day_bad.txtmarkdown340 B
benchmark/behavior_smoke/mock/sample_outputs/time_budget_1day_bank_checkpoint.txtmarkdown341 B
benchmark/behavior_smoke/mock/sample_outputs/time_budget_1day_good.txtmarkdown695 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_answer_after_prompt.txtmarkdown264 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_answer_before_prompt.txtmarkdown264 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_answer_side_first.txtmarkdown261 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_answer_text_before_prompt.txtmarkdown243 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_good.txtmarkdown232 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_missing_asset.txtmarkdown182 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_path_only.txtmarkdown163 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_prose_before_image.txtmarkdown234 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_question_label_late_asset.txtmarkdown250 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_unlabeled_solution_before_prompt.txtmarkdown233 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_unsafe_url.txtmarkdown178 B
benchmark/behavior_smoke/mock/sample_outputs/window_recheck_bad.txtmarkdown346 B
benchmark/behavior_smoke/mock/sample_outputs/window_recheck_good.txtmarkdown665 B
benchmark/behavior_smoke/mock/sample_outputs/window_recheck_recall_only_bad.txtmarkdown333 B
benchmark/behavior_smoke/mock/sample_outputs/window_recheck_test_good.txtmarkdown580 B
benchmark/behavior_smoke/mock/sample_outputs/workspace_confirm_good.txtmarkdown797 B
benchmark/behavior_smoke/mock/sample_outputs/workspace_confirm_silent.txtmarkdown453 B
benchmark/behavior_smoke/mock/sample_outputs/zero_basic_explain.txtmarkdown1.2 KB
benchmark/behavior_smoke/mock/sample_outputs/zero_basic_legacy_only.txtmarkdown442 B
benchmark/behavior_smoke/mock/sample_transcript.jsonlother517 B
benchmark/behavior_smoke/mock/state_after_confusion.jsonconfig203 B
benchmark/behavior_smoke/mock/state_after_mistake.jsonconfig215 B
benchmark/behavior_smoke/run_behavior_smoke.pyscript97.2 KB
benchmark/behavior_smoke/scenarios.jsonconfig12.7 KB
benchmark/calibrate.pyscript7.3 KB
benchmark/calibrate_matrix.pyscript20.0 KB
benchmark/calibration/README.en.mdmarkdown3.2 KB
benchmark/calibration/README.mdmarkdown2.6 KB
benchmark/calibration/kappa_n24_summary.jsonconfig1.5 KB
benchmark/config.example.jsonconfig431 B
benchmark/config.matrix.example.jsonconfig1.1 KB
benchmark/docs/coverage-matrix.en.mdmarkdown17.6 KB
benchmark/docs/coverage-matrix.mdmarkdown15.2 KB
benchmark/docs/img/hard_algo_correct_en.svgother2.7 KB
benchmark/docs/img/hard_algo_correct_zh.svgother2.7 KB
benchmark/docs/img/hard_algo_hallu_en.svgother2.7 KB
benchmark/docs/img/hard_algo_hallu_zh.svgother2.7 KB
benchmark/docs/img/hard_psyc_correct_en.svgother2.7 KB
benchmark/docs/img/hard_psyc_correct_zh.svgother2.7 KB
benchmark/docs/img/hard_psyc_hallu_en.svgother2.7 KB
benchmark/docs/img/hard_psyc_hallu_zh.svgother2.7 KB
benchmark/docs/img/oos_algo_abst_en.svgother2.7 KB
benchmark/docs/img/oos_algo_abst_zh.svgother2.7 KB
benchmark/docs/img/oos_psyc_abst_en.svgother2.7 KB
benchmark/docs/img/oos_psyc_abst_zh.svgother2.7 KB
benchmark/docs/img/realexam_algo_correct_en.svgother2.7 KB
benchmark/docs/img/realexam_algo_correct_zh.svgother2.7 KB
benchmark/docs/judge-calibration.en.mdmarkdown13.1 KB
benchmark/docs/judge-calibration.mdmarkdown9.9 KB
benchmark/docs/materials_sources.en.mdmarkdown3.2 KB
benchmark/docs/materials_sources.mdmarkdown2.7 KB
benchmark/docs/matrix_pipeline.en.mdmarkdown9.5 KB
benchmark/docs/matrix_pipeline.mdmarkdown8.4 KB
benchmark/docs/related_benchmarks.en.mdmarkdown5.5 KB
benchmark/docs/related_benchmarks.mdmarkdown4.6 KB
benchmark/docs/running-real-runs.en.mdmarkdown10.0 KB
benchmark/docs/running-real-runs.mdmarkdown8.0 KB
benchmark/docs/test_tiers.en.mdmarkdown7.7 KB
benchmark/docs/test_tiers.mdmarkdown6.8 KB
benchmark/docs/testing-audit.en.mdmarkdown17.2 KB
benchmark/docs/testing-audit.mdmarkdown14.6 KB
benchmark/docs/testing-walkthrough.en.mdmarkdown23.5 KB
benchmark/docs/测试流程详解.mdmarkdown19.0 KB
benchmark/drift/README.en.mdmarkdown14.6 KB
benchmark/drift/README.mdmarkdown12.3 KB
benchmark/drift/convert_session_log.pyscript12.9 KB
benchmark/drift/docs/live_agent_pilot.mdmarkdown6.6 KB
benchmark/drift/fixtures/live_logs/good_session.expected.jsonlother1.5 KB
benchmark/drift/fixtures/live_logs/good_session.mdmarkdown1.6 KB
benchmark/drift/fixtures/live_logs/live_smoke_golden.jsonlother2.2 KB
benchmark/drift/fixtures/live_logs/live_smoke_golden.mdmarkdown2.3 KB
benchmark/drift/fixtures/mini_course_long/references/quiz_bank.jsonconfig1.3 KB
benchmark/drift/fixtures/mini_course_long/references/wiki/ch1_stack_queue.mdmarkdown541 B
benchmark/drift/fixtures/mini_course_long/references/wiki/ch2_trees.mdmarkdown413 B
benchmark/drift/fixtures/mini_course_long/study_plan.mdmarkdown426 B
benchmark/drift/fixtures/mini_course_long/study_progress.final.bad.mdmarkdown151 B
benchmark/drift/fixtures/mini_course_long/study_progress.final.good.mdmarkdown272 B
benchmark/drift/fixtures/mini_course_long/study_progress.initial.mdmarkdown151 B
benchmark/drift/fixtures/mini_course_long_state/references/quiz_bank.jsonconfig1.3 KB
benchmark/drift/fixtures/mini_course_long_state/references/wiki/ch1_stack_queue.mdmarkdown541 B
benchmark/drift/fixtures/mini_course_long_state/references/wiki/ch2_trees.mdmarkdown413 B
benchmark/drift/fixtures/mini_course_long_state/study_plan.mdmarkdown426 B
benchmark/drift/fixtures/mini_course_long_state/study_progress.final.bad.mdmarkdown151 B
benchmark/drift/fixtures/mini_course_long_state/study_progress.final.good.mdmarkdown272 B
benchmark/drift/fixtures/mini_course_long_state/study_progress.initial.mdmarkdown151 B
benchmark/drift/fixtures/mini_course_long_state/study_state.jsonconfig145 B
benchmark/drift/run_drift.pyscript79.7 KB
benchmark/drift/run_live_smoke.pyscript26.4 KB
benchmark/drift/scenarios/live_smoke_basic.jsonconfig681 B
benchmark/drift/scenarios/long_session_basic.jsonconfig793 B
benchmark/drift/scenarios/long_session_state.jsonconfig994 B
benchmark/drift/scenarios/mode_urgent_no_questions.jsonconfig1.1 KB
benchmark/drift/scenarios/window_persist.jsonconfig1.0 KB
benchmark/drift/templates/live_session_template.mdmarkdown1.3 KB
benchmark/drift/templates/live_smoke_turns.jsonconfig1.6 KB
benchmark/drift/transcripts/bad_checkpoint_reset.jsonlother877 B
benchmark/drift/transcripts/bad_goal_drift.jsonlother226 B
benchmark/drift/transcripts/bad_plan_drift.jsonlother729 B
benchmark/drift/transcripts/bad_progress_loss.jsonlother1013 B
benchmark/drift/transcripts/bad_provenance_drift.jsonlother583 B
benchmark/drift/transcripts/bad_quiz_invention.jsonlother590 B
benchmark/drift/transcripts/bad_urgent_1day_questions.jsonlother840 B
benchmark/drift/transcripts/bad_wiki_overread.jsonlother430 B
benchmark/drift/transcripts/good_session.jsonlother5.5 KB
benchmark/drift/transcripts/good_session_state.jsonlother7.0 KB
benchmark/drift/transcripts/good_session_urgent_1day.jsonlother2.8 KB
benchmark/drift/transcripts/window_persist_session.jsonlother7.3 KB
benchmark/fixtures/mini_course_matrix/.gitignoreother250 B
benchmark/fixtures/mini_course_matrix/config.jsonconfig394 B
benchmark/fixtures/mini_course_matrix/items.jsonlother1.6 KB
benchmark/fixtures/mini_course_matrix/materials/_combined.txtmarkdown502 B
benchmark/fixtures/mini_course_matrix/raw_ws/lecture01.mdmarkdown379 B
benchmark/fixtures/mini_course_matrix/skill_ws/references/wiki/ch01_minios.mdmarkdown485 B
benchmark/gen.pyscript13.0 KB
benchmark/items/README.en.mdmarkdown3.3 KB
benchmark/items/README.mdmarkdown2.6 KB
benchmark/items/items.example.jsonlother2.0 KB
benchmark/items/items_algo_full_q.jsonlother13.5 KB
benchmark/items/items_pilot_6006.jsonlother4.5 KB
benchmark/items/items_pilot_psyc110.jsonlother4.5 KB
benchmark/items/items_psyc_full_q.jsonlother7.5 KB

benchmark/calibrate.py

View on GitHub
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Human-vs-judge calibration (Cohen's kappa) — the gate before trusting any LLM-judged number.

We just caught the LLM judge marking exact-match answers as hallucinations; this tool quantifies how
much the (fixed) judge can now be trusted, by comparing it against YOUR hand labels on a sample.

Two steps:

  1) sample — draw a STRATIFIED sample (half judge-correct, half judge-wrong, so kappa isn't
              degenerate) and write a calibration sheet that HIDES the judge's verdict. You fill the
              `human_correct` column with 1 (correct/acceptable) or 0 (wrong) — judging ONLY from the
              question, the gold answer, and the reference span.

         python calibrate.py sample --n 24 --course both --seed 7

  2) kappa  — after you fill the sheet, compute Cohen's kappa(human, judge) + raw agreement, and list
              every item where you and the judge DISAGREE (those are the judge's likely errors).

         python calibrate.py kappa

Files (under calibration/, gitignored — they embed answer details):
  calibration/calibration_sheet.csv   <- you edit the human_correct column here
  calibration/.calibration_key.jsonl  <- hidden judge verdicts, written by `sample`, read by `kappa`

Rule of thumb (README): trust the judge's numbers only at kappa >= ~0.6.
"""
import os
import sys
import csv
import json
import random
import argparse

import stats as S
import rejudge as RJ

for _s in ("stdout", "stderr"):
    try:
        getattr(sys, _s).reconfigure(encoding="utf-8")
    except Exception:
        pass

HERE = os.path.dirname(os.path.abspath(__file__))
CAL = os.path.join(HERE, "calibration")
SHEET = os.path.join(CAL, "calibration_sheet.csv")
KEY = os.path.join(CAL, ".calibration_key.jsonl")
CACHE = os.path.join(HERE, "results", "matrix", "judge_cache.jsonl")

_FIELDS = ["ref_id", "course", "answerable", "question", "gold_answer", "reference_span",
           "model_answer", "human_correct"]


def _flat(s):
    return " ".join(str(s or "").split())


def load_cache():
    cache = {}
    if os.path.exists(CACHE):
        for line in open(CACHE, encoding="utf-8"):
            line = line.strip()
            if line:
                d = json.loads(line)
                cache[d["k"]] = d["score"]
    return cache


def build_pool(course):
    gold = {"algo": RJ.load_gold(RJ.ALGO_GOLD), "psyc": RJ.load_gold(RJ.PSYC_GOLD)}
    cache = load_cache()
    pool = []
    for row in RJ.unified_rows(course):
        item = gold[row["course"]].get(row["id"])
        if not item:
            continue
        if RJ.is_infra_error(row["answer"]):          # skip rate-limit/API error strings — not answers
            continue
        sc = cache.get(RJ.cache_key(row["id"], row["answer"]))
        if not sc:                                    # only items with an authoritative verdict
            continue
        pool.append({
            "course": row["course"], "model": row["model"], "arm": row["arm"], "id": row["id"],
            "answerable": bool(item.get("answerable", True)),
            "question": item.get("question", ""), "gold_answer": item.get("gold_answer", ""),
            "reference_span": item.get("supporting_span", ""), "answer": row["answer"],
            "judge_correct": 1 if sc.get("correct") else 0,
        })
    return pool


def cmd_sample(args):
    pool = build_pool(args.course)
    if not pool:
        sys.exit("[-] 没有可抽样的条目(先跑 rejudge.py --llm 生成 judge_cache.jsonl)")
    rng = random.Random(args.seed)
    pos = [p for p in pool if p["judge_correct"] == 1]
    neg = [p for p in pool if p["judge_correct"] == 0]
    rng.shuffle(pos); rng.shuffle(neg)
    half = args.n // 2
    pick = pos[:half] + neg[:args.n - half]
    rng.shuffle(pick)

    os.makedirs(CAL, exist_ok=True)
    with open(SHEET, "w", encoding="utf-8-sig", newline="") as f, open(KEY, "w", encoding="utf-8") as kf:
        w = csv.DictWriter(f, fieldnames=_FIELDS)
        w.writeheader()
        for i, p in enumerate(pick, 1):
            ref = f"cal_{i:03d}"
            note = "" if p["answerable"] else "  ←【越界题:材料无答案,正确=老实弃答】"
            w.writerow({"ref_id": ref, "course": p["course"], "answerable": int(p["answerable"]),
                        "question": _flat(p["question"]) + note, "gold_answer": _flat(p["gold_answer"]),
                        "reference_span": _flat(p["reference_span"]), "model_answer": _flat(p["answer"]),
                        "human_correct": ""})
            kf.write(json.dumps({"ref_id": ref, "judge_correct": p["judge_correct"]}, ensure_ascii=False) + "\n")
    print(f"[+] 抽样 {len(pick)} 条(判对 {min(half,len(pos))} / 判错 {len(pick)-min(half,len(pos))})"
          f",已写出待填表:\n    {SHEET}")
    print("    用 Excel/编辑器打开,给 human_correct 列填 1(对/可接受)或 0(错);填完跑:python calibrate.py kappa")
    print("    判定标准:只看 question + gold_answer + reference_span 判 model_answer 对不对;越界题以「是否老实弃答」为准。")


def cmd_kappa(args):
    if not (os.path.exists(SHEET) and os.path.exists(KEY)):
        sys.exit("[-] 找不到 calibration_sheet.csv / .calibration_key.jsonl,先跑:python calibrate.py sample")
    key = {d["ref_id"]: d["judge_correct"] for d in (json.loads(l) for l in open(KEY, encoding="utf-8") if l.strip())}
    rows = list(csv.DictReader(open(SHEET, encoding="utf-8-sig")))
    human, judge, disagree, blank = [], [], [], 0
    for r in rows:
        hv = (r.get("human_correct") or "").strip()
        if hv not in ("0", "1"):
            blank += 1
            continue
        ref = r["ref_id"]
        if ref not in key:
            continue
        h, j = int(hv), key[ref]
        human.append(h); judge.append(j)
        if h != j:
            disagree.append((ref, j, h, r.get("question", "")[:70]))
    n = len(human)
    if n == 0:
        sys.exit(f"[-] 还没有已填的 human_correct({blank} 行为空)。先在 {SHEET} 填好再跑。")
    agree = sum(1 for h, j in zip(human, judge) if h == j) / n
    k = S.cohen_kappa(human, judge)
    print(f"=== 人工 vs 裁判一致性(n={n},未填 {blank})===")
    print(f"  原始一致率 agreement = {agree*100:.1f}%")
    print(f"  Cohen's kappa        = {k:.3f}   ->  {'可信(>=0.6)' if k >= 0.6 else '偏低,先改进裁判/题目再信任数字'}")
    if disagree:
        print(f"\n  人机分歧 {len(disagree)} 条(judge=裁判判, human=你判;这些是裁判最可能错的地方):")
        for ref, j, h, q in disagree:
            print(f"    {ref}: judge={j} human={h} | {q}")
    else:
        print("\n  无分歧 —— 裁判与你完全一致。")


def main():
    ap = argparse.ArgumentParser(description="人工 vs 裁判 kappa 校准")
    sub = ap.add_subparsers(dest="cmd", required=True)
    sp = sub.add_parser("sample", help="抽样生成待填校准表")
    sp.add_argument("--n", type=int, default=24)
    sp.add_argument("--course", default="both", choices=["algo", "psyc", "both"])
    sp.add_argument("--seed", type=int, default=7)
    sp.set_defaults(func=cmd_sample)
    kp = sub.add_parser("kappa", help="读已填表,算 kappa + 列分歧")
    kp.set_defaults(func=cmd_kappa)
    args = ap.parse_args()
    args.func(args)


if __name__ == "__main__":
    main()
receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementationStars: 322kUpdated Claude CodeCodexVerifiedInstall
requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirementsStars: 322kUpdated Claude CodeCodexVerifiedInstall
using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallbackStars: 322kUpdated Claude CodeCodexVerifiedInstall
verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions alwaysStars: 322kUpdated Claude CodeCodexVerifiedInstall