Skip to content

ZeKaiNie/universal-examprep-skill / full

universal-exam-cram-coach-full

Verified

帮助学生在临考前进行结构化极速复习:解析课程资料/大纲/重点,按章节生成 wiki 知识库与标准题库,组织针对性刷题与判分,并记录复习进度和错题。当用户即将考试、需要快速复习计划、练习题、错题复盘或考前小抄时使用(关键词:期末/备考/复习/刷题/划重点/错题;exam, cram, study plan, quiz, review)。不适用于长期学习规划、与考试无关的写作或编程任务。

FileKindSize
.gitattributesother1.5 KB
.github/workflows/ci.ymlconfig1.9 KB
.gitignoreother659 B
AGENTS.mdmarkdown38.8 KB
CHANGELOG.en.mdmarkdown25.5 KB
CHANGELOG.mdmarkdown20.5 KB
CONTRIBUTING.mdmarkdown5.1 KB
CONTRIBUTING.zh.mdmarkdown4.9 KB
LICENSEother1.0 KB
README.mdmarkdown24.5 KB
README.zh.mdmarkdown22.8 KB
SKILL.mdskill md10.4 KB
assets/exam-panic.pngbinary934.2 KB
benchmark/.gitignoreother1.2 KB
benchmark/README.en.mdmarkdown12.7 KB
benchmark/README.mdmarkdown10.8 KB
benchmark/REPORT.en.mdmarkdown7.1 KB
benchmark/REPORT.mdmarkdown6.4 KB
benchmark/aggregate_matrix.pyscript16.3 KB
benchmark/behavior_smoke/README.en.mdmarkdown9.9 KB
benchmark/behavior_smoke/README.mdmarkdown7.4 KB
benchmark/behavior_smoke/fixtures/mini_course/references/assets/venn_prompt.svgother677 B
benchmark/behavior_smoke/fixtures/mini_course/references/quiz_bank.jsonconfig4.5 KB
benchmark/behavior_smoke/fixtures/mini_course/references/wiki/ch1_stack_queue.mdmarkdown447 B
benchmark/behavior_smoke/fixtures/mini_course/references/wiki/ch2_trees.mdmarkdown493 B
benchmark/behavior_smoke/fixtures/mini_course/study_plan.mdmarkdown550 B
benchmark/behavior_smoke/fixtures/mini_course/study_progress.mdmarkdown429 B
benchmark/behavior_smoke/fixtures/mini_course/study_state.jsonconfig390 B
benchmark/behavior_smoke/fixtures/mini_course_nostate/references/assets/venn_prompt.svgother677 B
benchmark/behavior_smoke/fixtures/mini_course_nostate/references/quiz_bank.jsonconfig3.2 KB
benchmark/behavior_smoke/fixtures/mini_course_nostate/references/wiki/ch1_stack_queue.mdmarkdown447 B
benchmark/behavior_smoke/fixtures/mini_course_nostate/references/wiki/ch2_trees.mdmarkdown493 B
benchmark/behavior_smoke/fixtures/mini_course_nostate/study_plan.mdmarkdown550 B
benchmark/behavior_smoke/fixtures/mini_course_nostate/study_progress.mdmarkdown429 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_chat_auto_pdf_bad.txtmarkdown433 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_chat_good.txtmarkdown462 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_chat_not_persisted_bad.txtmarkdown186 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_one_shot_good.txtmarkdown490 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_one_shot_persist_bad.txtmarkdown605 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_one_shot_probe_error_bad.txtmarkdown507 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_visual_native_good.txtmarkdown495 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_visual_preflight_failed_bad.txtmarkdown487 B
benchmark/behavior_smoke/mock/sample_outputs/artifact_mode_visual_subscription_guess_bad.txtmarkdown507 B
benchmark/behavior_smoke/mock/sample_outputs/hint_skip_offer.txtmarkdown349 B
benchmark/behavior_smoke/mock/sample_outputs/language_ask_good.txtmarkdown461 B
benchmark/behavior_smoke/mock/sample_outputs/language_ask_missing_line.txtmarkdown270 B
benchmark/behavior_smoke/mock/sample_outputs/language_ask_unglossed.txtmarkdown211 B
benchmark/behavior_smoke/mock/sample_outputs/language_persist_good.txtmarkdown216 B
benchmark/behavior_smoke/mock/sample_outputs/language_persist_noncanonical.txtmarkdown157 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_bilingual.txtmarkdown261 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_bilingual_explicit.txtmarkdown372 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_bilingual_negated.txtmarkdown306 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_question.txtmarkdown224 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_silent.txtmarkdown364 B
benchmark/behavior_smoke/mock/sample_outputs/language_urgent_wrong_language.txtmarkdown262 B
benchmark/behavior_smoke/mock/sample_outputs/notebook_persist_chat_only.txtmarkdown952 B
benchmark/behavior_smoke/mock/sample_outputs/notebook_persist_good.txtmarkdown744 B
benchmark/behavior_smoke/mock/sample_outputs/progress_after_confusion.mdmarkdown395 B
benchmark/behavior_smoke/mock/sample_outputs/progress_after_mistake.mdmarkdown403 B
benchmark/behavior_smoke/mock/sample_outputs/provenance_answer.txtmarkdown492 B
benchmark/behavior_smoke/mock/sample_outputs/quiz_output_good.txtmarkdown351 B
benchmark/behavior_smoke/mock/sample_outputs/quiz_output_invented.txtmarkdown317 B
benchmark/behavior_smoke/mock/sample_outputs/resume_message.txtmarkdown224 B
benchmark/behavior_smoke/mock/sample_outputs/scope_override_good.txtmarkdown258 B
benchmark/behavior_smoke/mock/sample_outputs/scope_override_missing.txtmarkdown127 B
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_ai_answer_good.txtmarkdown1.2 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_formula_first.txtmarkdown1.1 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_good.txtmarkdown1.2 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_liberal_good.txtmarkdown1.4 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_missing_warn.txtmarkdown608 B
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_no_source.txtmarkdown1.1 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_optin_closers.txtmarkdown1.5 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_skip_ask.txtmarkdown948 B
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_unlabeled_source.txtmarkdown517 B
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_unsolicited_closers.txtmarkdown1.5 KB
benchmark/behavior_smoke/mock/sample_outputs/teaching_template_warn_title.txtmarkdown581 B
benchmark/behavior_smoke/mock/sample_outputs/time_budget_1day_bad.txtmarkdown340 B
benchmark/behavior_smoke/mock/sample_outputs/time_budget_1day_bank_checkpoint.txtmarkdown341 B
benchmark/behavior_smoke/mock/sample_outputs/time_budget_1day_good.txtmarkdown695 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_answer_after_prompt.txtmarkdown264 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_answer_before_prompt.txtmarkdown264 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_answer_side_first.txtmarkdown261 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_answer_text_before_prompt.txtmarkdown243 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_good.txtmarkdown232 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_missing_asset.txtmarkdown182 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_path_only.txtmarkdown163 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_prose_before_image.txtmarkdown234 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_question_label_late_asset.txtmarkdown250 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_unlabeled_solution_before_prompt.txtmarkdown233 B
benchmark/behavior_smoke/mock/sample_outputs/visual_first_unsafe_url.txtmarkdown178 B
benchmark/behavior_smoke/mock/sample_outputs/window_recheck_bad.txtmarkdown346 B
benchmark/behavior_smoke/mock/sample_outputs/window_recheck_good.txtmarkdown665 B
benchmark/behavior_smoke/mock/sample_outputs/window_recheck_recall_only_bad.txtmarkdown333 B
benchmark/behavior_smoke/mock/sample_outputs/window_recheck_test_good.txtmarkdown580 B
benchmark/behavior_smoke/mock/sample_outputs/workspace_confirm_good.txtmarkdown797 B
benchmark/behavior_smoke/mock/sample_outputs/workspace_confirm_silent.txtmarkdown453 B
benchmark/behavior_smoke/mock/sample_outputs/zero_basic_explain.txtmarkdown1.2 KB
benchmark/behavior_smoke/mock/sample_outputs/zero_basic_legacy_only.txtmarkdown442 B
benchmark/behavior_smoke/mock/sample_transcript.jsonlother517 B
benchmark/behavior_smoke/mock/state_after_confusion.jsonconfig203 B
benchmark/behavior_smoke/mock/state_after_mistake.jsonconfig215 B
benchmark/behavior_smoke/run_behavior_smoke.pyscript97.2 KB
benchmark/behavior_smoke/scenarios.jsonconfig12.7 KB
benchmark/calibrate.pyscript7.3 KB
benchmark/calibrate_matrix.pyscript20.0 KB
benchmark/calibration/README.en.mdmarkdown3.2 KB
benchmark/calibration/README.mdmarkdown2.6 KB
benchmark/calibration/kappa_n24_summary.jsonconfig1.5 KB
benchmark/config.example.jsonconfig431 B
benchmark/config.matrix.example.jsonconfig1.1 KB
benchmark/docs/coverage-matrix.en.mdmarkdown17.6 KB
benchmark/docs/coverage-matrix.mdmarkdown15.2 KB
benchmark/docs/img/hard_algo_correct_en.svgother2.7 KB
benchmark/docs/img/hard_algo_correct_zh.svgother2.7 KB
benchmark/docs/img/hard_algo_hallu_en.svgother2.7 KB
benchmark/docs/img/hard_algo_hallu_zh.svgother2.7 KB
benchmark/docs/img/hard_psyc_correct_en.svgother2.7 KB
benchmark/docs/img/hard_psyc_correct_zh.svgother2.7 KB
benchmark/docs/img/hard_psyc_hallu_en.svgother2.7 KB
benchmark/docs/img/hard_psyc_hallu_zh.svgother2.7 KB
benchmark/docs/img/oos_algo_abst_en.svgother2.7 KB
benchmark/docs/img/oos_algo_abst_zh.svgother2.7 KB
benchmark/docs/img/oos_psyc_abst_en.svgother2.7 KB
benchmark/docs/img/oos_psyc_abst_zh.svgother2.7 KB
benchmark/docs/img/realexam_algo_correct_en.svgother2.7 KB
benchmark/docs/img/realexam_algo_correct_zh.svgother2.7 KB
benchmark/docs/judge-calibration.en.mdmarkdown13.1 KB
benchmark/docs/judge-calibration.mdmarkdown9.9 KB
benchmark/docs/materials_sources.en.mdmarkdown3.2 KB
benchmark/docs/materials_sources.mdmarkdown2.7 KB
benchmark/docs/matrix_pipeline.en.mdmarkdown9.5 KB
benchmark/docs/matrix_pipeline.mdmarkdown8.4 KB
benchmark/docs/related_benchmarks.en.mdmarkdown5.5 KB
benchmark/docs/related_benchmarks.mdmarkdown4.6 KB
benchmark/docs/running-real-runs.en.mdmarkdown10.0 KB
benchmark/docs/running-real-runs.mdmarkdown8.0 KB
benchmark/docs/test_tiers.en.mdmarkdown7.7 KB
benchmark/docs/test_tiers.mdmarkdown6.8 KB
benchmark/docs/testing-audit.en.mdmarkdown17.2 KB
benchmark/docs/testing-audit.mdmarkdown14.6 KB
benchmark/docs/testing-walkthrough.en.mdmarkdown23.5 KB
benchmark/docs/测试流程详解.mdmarkdown19.0 KB
benchmark/drift/README.en.mdmarkdown14.6 KB
benchmark/drift/README.mdmarkdown12.3 KB
benchmark/drift/convert_session_log.pyscript12.9 KB
benchmark/drift/docs/live_agent_pilot.mdmarkdown6.6 KB
benchmark/drift/fixtures/live_logs/good_session.expected.jsonlother1.5 KB
benchmark/drift/fixtures/live_logs/good_session.mdmarkdown1.6 KB
benchmark/drift/fixtures/live_logs/live_smoke_golden.jsonlother2.2 KB
benchmark/drift/fixtures/live_logs/live_smoke_golden.mdmarkdown2.3 KB
benchmark/drift/fixtures/mini_course_long/references/quiz_bank.jsonconfig1.3 KB
benchmark/drift/fixtures/mini_course_long/references/wiki/ch1_stack_queue.mdmarkdown541 B
benchmark/drift/fixtures/mini_course_long/references/wiki/ch2_trees.mdmarkdown413 B
benchmark/drift/fixtures/mini_course_long/study_plan.mdmarkdown426 B
benchmark/drift/fixtures/mini_course_long/study_progress.final.bad.mdmarkdown151 B
benchmark/drift/fixtures/mini_course_long/study_progress.final.good.mdmarkdown272 B
benchmark/drift/fixtures/mini_course_long/study_progress.initial.mdmarkdown151 B
benchmark/drift/fixtures/mini_course_long_state/references/quiz_bank.jsonconfig1.3 KB
benchmark/drift/fixtures/mini_course_long_state/references/wiki/ch1_stack_queue.mdmarkdown541 B
benchmark/drift/fixtures/mini_course_long_state/references/wiki/ch2_trees.mdmarkdown413 B
benchmark/drift/fixtures/mini_course_long_state/study_plan.mdmarkdown426 B
benchmark/drift/fixtures/mini_course_long_state/study_progress.final.bad.mdmarkdown151 B
benchmark/drift/fixtures/mini_course_long_state/study_progress.final.good.mdmarkdown272 B
benchmark/drift/fixtures/mini_course_long_state/study_progress.initial.mdmarkdown151 B
benchmark/drift/fixtures/mini_course_long_state/study_state.jsonconfig145 B
benchmark/drift/run_drift.pyscript79.7 KB
benchmark/drift/run_live_smoke.pyscript26.4 KB
benchmark/drift/scenarios/live_smoke_basic.jsonconfig681 B
benchmark/drift/scenarios/long_session_basic.jsonconfig793 B
benchmark/drift/scenarios/long_session_state.jsonconfig994 B
benchmark/drift/scenarios/mode_urgent_no_questions.jsonconfig1.1 KB
benchmark/drift/scenarios/window_persist.jsonconfig1.0 KB
benchmark/drift/templates/live_session_template.mdmarkdown1.3 KB
benchmark/drift/templates/live_smoke_turns.jsonconfig1.6 KB
benchmark/drift/transcripts/bad_checkpoint_reset.jsonlother877 B
benchmark/drift/transcripts/bad_goal_drift.jsonlother226 B
benchmark/drift/transcripts/bad_plan_drift.jsonlother729 B
benchmark/drift/transcripts/bad_progress_loss.jsonlother1013 B
benchmark/drift/transcripts/bad_provenance_drift.jsonlother583 B
benchmark/drift/transcripts/bad_quiz_invention.jsonlother590 B
benchmark/drift/transcripts/bad_urgent_1day_questions.jsonlother840 B
benchmark/drift/transcripts/bad_wiki_overread.jsonlother430 B
benchmark/drift/transcripts/good_session.jsonlother5.5 KB
benchmark/drift/transcripts/good_session_state.jsonlother7.0 KB
benchmark/drift/transcripts/good_session_urgent_1day.jsonlother2.8 KB
benchmark/drift/transcripts/window_persist_session.jsonlother7.3 KB
benchmark/fixtures/mini_course_matrix/.gitignoreother250 B
benchmark/fixtures/mini_course_matrix/config.jsonconfig394 B
benchmark/fixtures/mini_course_matrix/items.jsonlother1.6 KB
benchmark/fixtures/mini_course_matrix/materials/_combined.txtmarkdown502 B
benchmark/fixtures/mini_course_matrix/raw_ws/lecture01.mdmarkdown379 B
benchmark/fixtures/mini_course_matrix/skill_ws/references/wiki/ch01_minios.mdmarkdown485 B
benchmark/gen.pyscript13.0 KB
benchmark/items/README.en.mdmarkdown3.3 KB
benchmark/items/README.mdmarkdown2.6 KB
benchmark/items/items.example.jsonlother2.0 KB
benchmark/items/items_algo_full_q.jsonlother13.5 KB
benchmark/items/items_pilot_6006.jsonlother4.5 KB
benchmark/items/items_pilot_psyc110.jsonlother4.5 KB
benchmark/items/items_psyc_full_q.jsonlother7.5 KB

benchmark/docs/test_tiers.en.md

View on GitHub
# Test Tiers

English · [中文](test_tiers.md)

> **This file = the concise, authoritative definition of the tiers (canonical tier taxonomy).** For the detailed audit of the current state see
> [`testing-audit.en.md`](testing-audit.en.md); for the capability × tier coverage table see [`coverage-matrix.en.md`](coverage-matrix.en.md).
> All three documents must define each tier (especially **Tier 2 = behavior smoke**) the same way.

The full anti-hallucination benchmark (real runs across models) is **expensive**: one full single-round matrix costs tens of dollars and takes hours,
and long-horizon drift testing costs subscription quota measured in days. To avoid "running everything for every small change", we split testing into 5 tiers:
**the earlier the tier, the cheaper it is and the more often it should run; the later the tier, the more expensive it is and the more it should be triggered by hand**.

| Tier | Contents | Cost | When to run |
| --- | --- | --- | --- |
| **Tier 0 unit/static tests** | `python -m unittest discover -s tests` (ingest, validator, skill structure, language policy, etc.; pure stdlib, no network/LLM) | Seconds, $0 | **Required on every commit / in CI** |
| **Tier 1 workspace validation** | `python scripts/validate_workspace.py <ws>` (a built workspace's structure / question-bank schema / provenance labels / path safety) | Seconds, $0 | When changing schema/ingest/skills (locally or by hand); its validation logic is already covered by Tier 0 on `tests/fixtures/` |
| **Tier 2 behavior smoke (behavioral smoke)** | **Scripted agent behavior smoke**: a tiny **self-authored fixture workspace** + **scripted prompts** + **deterministic assertions on the produced files/output** (covering quiz_bank-only / provenance labels / hint·skip·mistake logging / confusion tracking / checkpoint resume / no-Python fallback / beginner walkthrough). **The deterministic mock layer has landed** (see [`../behavior_smoke/`](../behavior_smoke/); pure stdlib, in CI); **since T5c there is an opt-in real-agent smoke runner** (`../drift/run_live_smoke.py`: drive → record → T4 scoring in one command; env-gated, not in CI); **the behavior_smoke `--llm` single-turn real-agent smoke is wired up (B2; the wiring is tested deterministically with a stub agent)**; real paid runs remain opt-in and have not yet been run against a real model; lazy loading is best-effort | Near $0 by default | Mock layer whenever skill behavior changes (in CI); real-agent smoke is opt-in and manual |
| **Tier 3 full benchmark matrix** | **6.006 has all three arms in full** (`closedbook`/`rawfiles`/`skill`); **PSYC is partial** (`closedbook`/`skill` in full; `rawfiles` complete only for Opus, a small amount for Sonnet, none for Haiku; `report_matrix.py` also defines only closedbook/skill for PSYC). **Since T3, the aggregation layer [`aggregate_matrix.py`](matrix_pipeline.en.md) + fixture pipeline are committed, and the mechanism is reproducible from a clean checkout**; but **the full published matrix still needs private/intermediate artifacts + paid runs**. For the data see [`testing-audit.en.md`](testing-audit.en.md) | Tens of dollars, hours (fixture mechanism $0) | **Manual trigger only**, when published data/methods change |
| **Tier 4 long-horizon drift benchmark** | Measures drift across multi-turn long sessions. **Since T4, the deterministic replay harness has landed** ([`../drift/`](../drift/): replays scripted transcripts + snapshots; pure stdlib; in root-level tests); **real-LLM long sessions remain opt-in, unimplemented, and not in CI** (see below) | Replay $0; real LLM costs days of quota | Replay can run whenever skill behavior changes; real LLM is manual |

> **Tier 2 ≠ "benchmark pipeline mock check"**: `run_benchmark.py --mock` (a dry run on `items/items.example.jsonl` that
> only verifies that the benchmark pipeline connects end to end) is a **separate "benchmark pipeline mock check"**.
> It only proves that the pipeline runs. It is **not** the canonical Tier 2 behavior smoke and does not replace it: Tier 2 tests **skill behavior**, not pipeline connectivity.

## CI policy (stated honestly)
- **CI (`.github/workflows/ci.yml`) currently runs only Tier 0** (`python -m unittest discover -s tests -v`): zero cost, cross-platform, no secrets.
- **The Tier 1 validator's logic is covered by Tier 0 tests + `tests/fixtures/`**; but **there is currently no** separate CI step that runs `validate_workspace.py` on "a workspace produced by a real ingest" (this PR adds no CI step).
- **Tier 2's deterministic mock layer is in CI** (included in `tests/test_behavior_smoke.py`; pure stdlib, zero cost); but **Tier 2's real-LLM behavior smoke (opt-in) and Tiers 3–4 are not in automated CI**, so that no PR has to pay for them.

## How Tier 2 differs from Tier 4
Both test "behavior", but **at a different granularity and cost**: Tier 2 is a **single-scenario, deterministic, near-zero-cost** behavior smoke (one scripted interaction + assertions on the artifacts);
Tier 4 measures drift across **multi-turn long sessions** (next section). Tier 2 is the cheap early warning for Tier 4. **Tier 2's deterministic layer landed in PR T2** (real-LLM behavior is still opt-in and not in CI); **Tier 4's deterministic replay harness landed in PR T4** (it replays scripted transcripts; pure stdlib; in root-level tests); **real-LLM long sessions remain opt-in, unimplemented, and not in CI**.

## Tier 4: long-horizon drift benchmark (deterministic replay has landed; real-LLM long sessions are still future work)
This targets the blind spot raised in the PR #7 discussion: the current benchmark measures only "single-question accuracy" and cannot reproduce "gradual breakdown over a multi-turn long session"
(goal drift, unauthorized plan changes, inventing questions outside the question bank, failed checkpoint recovery). **T4 has landed a deterministic replay harness**
[`../drift/`](../drift/): it replays scripted multi-turn transcripts + workspace snapshots (self-authored, non-copyrighted fixtures) and measures the following metrics
**deterministically**, comparing them against thresholds (pure stdlib, zero cost, in root-level tests; `python benchmark/drift/run_drift.py --all`):

- **Goal retention**: after N turns, does it still follow the original plan/goal?
- **Plan adherence**: does it avoid rewriting the phase sequence in `study_plan.md` without the user's consent?
- **Quiz-bank fidelity / invention rate**: can every question the AI asks be found in `quiz_bank.json`, in the right phase? (Invention rate = the share that cannot be found.)
- **Checkpoint recovery**: after a restart, can it continue from the current phase in `study_progress.md` (instead of falling back to phase 1)?
- **Provenance fidelity**: does every later explanation turn carry the 🟢/🟡/⚠️ content labels? (Uses the same check as T2; the legend does not count.)
- **Mistake/confusion persistence**: once mistakes/confusions are recorded, are they **never silently deleted**?
- **Per-turn cost / context growth (optional)** and **wiki lazy loading / out-of-chapter reads**: the cost curve as turns accumulate, and whether it reads only the chapter for the current phase.

> Current state: **Tiers 0–1** low-cost structural validation + **the Tier 2 deterministic mock layer** (PR T2) + **the Tier 4 deterministic replay harness**
> (PR T4) have all landed in (root-level) tests; **Tier 2's real-LLM behavior validation and Tier 4's real-LLM long sessions remain opt-in, are not in CI,
> and have not yet been run against a real model** (behavior_smoke `--llm` single-turn is **wired up** and tested deterministically with a stub, and the drift multi-turn live runner is also ready; real paid runs are always opt-in and never in CI); **the full long-horizon LLM benchmark (costing days of quota) is still future work**.
receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementationStars: 322kUpdated Claude CodeCodexVerifiedInstall
requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirementsStars: 322kUpdated Claude CodeCodexVerifiedInstall
using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallbackStars: 322kUpdated Claude CodeCodexVerifiedInstall
verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions alwaysStars: 322kUpdated Claude CodeCodexVerifiedInstall