ZeKaiNie/universal-examprep-skill / full
universal-exam-cram-coach-full
Verified帮助学生在临考前进行结构化极速复习:解析课程资料/大纲/重点,按章节生成 wiki 知识库与标准题库,组织针对性刷题与判分,并记录复习进度和错题。当用户即将考试、需要快速复习计划、练习题、错题复盘或考前小抄时使用(关键词:期末/备考/复习/刷题/划重点/错题;exam, cram, study plan, quiz, review)。不适用于长期学习规划、与考试无关的写作或编程任务。
benchmark/calibration/kappa_n24_summary.json
View on GitHub{
"what": "24 题四层分层盲测人工校准(human vs Sonnet 裁判)——脱敏摘要,不含任何课程内容/题面/答案文本",
"date": "2026-07-04",
"method": {
"tool": "calibrate_matrix.py 同款四层分层(可答判对/可答判错/越界弃答/越界未弃答 = 8/8/4/4,判对:判错各半、可答:越界≈2:1)",
"blind": "裁判判定与 model/arm 对标注者隐藏(藏 .calibration_key.jsonl)",
"pool": "真实 PSYC 110 / MIT 6.006 结果(Sonnet 权威判定);先剔除 29 条配额错误答案(infra 残留非模型作答);可答的数值题/词法快路按 B5 政策排除",
"annotator": "Siyun Chen(人工逐条盲判)"
},
"n": 24,
"agreement": 0.9167,
"cohen_kappa": 0.8333,
"strata": {
"answerable_judge_correct": 8,
"answerable_judge_wrong": 8,
"oos_judge_abstained": 4,
"oos_judge_not_abstained": 4
},
"oos_agreement": "8/8",
"disagreements": [
{
"item_id": "psyc_27",
"model": "sonnet",
"arm": "closedbook",
"direction": "human=对 / judge=错(判分偏严)"
},
{
"item_id": "psyc_27",
"model": "opus",
"arm": "closedbook",
"direction": "human=对 / judge=错(判分偏严)"
}
],
"caveat": "分歧方向(全部判分偏严)是抽样迹象、非对全部判分的证明;判分与被测同属 Claude 家族是已知局限。",
"corroborates": "早先 16 题抽查 kappa=0.875(README『判分如何保证可信』)",
"raw_artifacts": "盲表/隐藏key/人工判定 属真实数据不入库(本目录 gitignored),此摘要为唯一入库件"
}
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementation | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirements | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |