Skip to content

MiniMax-AI/mcode-stack / stacks/jiachen/skills/pr-review-tuning

pr-review-tuning

Verified

The weekly self-tuning loop for the MCode Review bot on GitHub or GitLab. Measures the bot on a repository's recent PRs (threads per PR, which kinds people answer or dismiss, what was unanswered at merge, cost per round, time to merge, follow-up fixes), finds noise and misses, proposes at most three rule changes with their expected effect on each goal, backtests them on past PRs with known outcomes within an agreed token budget, has an independent check recompute the numbers, and changes rules only after a person approves. The first run only measures. Use for requests to run pr-review-tuning or to measure or tune the review bot, for the weekly scheduled run, and for settling a proposal (approve or reject P1).

scripts/test/fixtures/metrics/prs.jsonl

View on GitHub
{"host":"github","project":"acme/app","number":101,"url":"https://github.com/acme/app/pull/101","title":"Add a login retry limit","author":"alice","state":"merged","created_at":"2026-01-05T09:00:00Z","merged_at":"2026-01-06T09:00:00Z","source_branch":"feature/login-limit","target_branch":"main","draft":false,"summary":{"verdict":"pass","tier":"M","round":2,"phase":"shadow"},"history":[{"sha":"a1b2c3d4e5f60718293a4b5c6d7e8f9012345678","verdict":"request_changes","round":1,"model_called":true,"at":"2026-01-05T10:00:00Z","arm":null,"tier":"M","usage":{"input":100000,"output":5000,"cache_read":20000}},{"sha":"b2c3d4e5f60718293a4b5c6d7e8f901234567890","verdict":"pass","round":2,"model_called":true,"at":"2026-01-05T15:00:00Z","arm":null,"tier":"M","usage":{"input":60000,"output":3000,"cache_read":40000}}],"threads":[{"id":"F-1a2b3c","kind":"finding","source":"model","severity":"P1","type":"defect","class":"retry-semantics","lens":"general","title":"A failed login is counted twice","path":"src/auth/login.ts","created_at":"2026-01-05T10:00:00Z","resolved":true,"resolved_by_bot":true,"resolved_by":"github-actions","human_replies":1,"first_human_reply_at":"2026-01-05T11:00:00Z","human_reply_texts":["Good catch, fixed in 3f9a2c1."]},{"id":"F-4d5e6f","kind":"finding","source":"model","severity":"P2","type":"maintainability","class":"duplicate-implementation","lens":"general","title":"The retry limit is defined in two places","path":"src/auth/session.ts","created_at":"2026-01-05T10:00:00Z","resolved":false,"resolved_by_bot":false,"resolved_by":null,"human_replies":1,"first_human_reply_at":"2026-01-05T12:00:00Z","human_reply_texts":["Will do this in a follow-up PR."]},{"id":"NH-risk-auth","kind":"risk_path","source":"condition","severity":null,"type":null,"class":null,"lens":"","title":"Risk path touched: permissions and authentication","path":"src/auth/login.ts","created_at":"2026-01-05T10:00:00Z","resolved":true,"resolved_by_bot":false,"resolved_by":"alice","human_replies":0,"first_human_reply_at":null,"human_reply_texts":[]}],"quiz":{"posted":1,"answered":true},"fixes":[],"files":["src/auth/login.ts","src/auth/session.ts"]}
{"host":"github","project":"acme/app","number":102,"url":"https://github.com/acme/app/pull/102","title":"Fix rounding in the cart total","author":"bob","state":"merged","created_at":"2026-01-06T08:00:00Z","merged_at":"2026-01-06T10:00:00Z","source_branch":"fix/cart-total","target_branch":"main","draft":false,"summary":{"verdict":"needs_human","tier":"S","round":1,"phase":"shadow"},"history":[{"sha":"c3d4e5f60718293a4b5c6d7e8f9012345678901a","verdict":"needs_human","round":1,"model_called":true,"at":"2026-01-06T08:30:00Z","arm":null,"tier":"S","usage":{"input":30000,"output":2000,"cache_read":10000}}],"threads":[{"id":"F-7a8b9c","kind":"finding","source":"model","severity":"P2","type":"defect","class":"other","lens":"general","title":"An empty cart skips the minimum check","path":"src/cart/total.ts","created_at":"2026-01-06T08:30:00Z","resolved":false,"resolved_by_bot":false,"resolved_by":null,"human_replies":0,"first_human_reply_at":null,"human_reply_texts":[]},{"id":"NH-0d1e2f","kind":"design_direction","source":"model","severity":null,"type":null,"class":null,"lens":"general","title":"Should totals round per line or per cart?","path":"src/cart/total.ts","created_at":"2026-01-06T08:30:00Z","resolved":false,"resolved_by_bot":false,"resolved_by":null,"human_replies":1,"first_human_reply_at":"2026-01-06T11:00:00Z","human_reply_texts":["这是设计如此,不需要改"]}],"quiz":null,"fixes":[99],"files":["src/cart/total.ts"]}
{"host":"github","project":"acme/app","number":103,"url":"https://github.com/acme/app/pull/103","title":"Update the README","author":"carol","state":"merged","created_at":"2026-01-07T00:00:00Z","merged_at":"2026-01-07T06:00:00Z","source_branch":"docs/readme","target_branch":"main","draft":false,"summary":null,"history":[],"threads":[],"quiz":null,"fixes":[],"files":["README.md"]}
{"host":"github","project":"acme/app","number":104,"url":"https://github.com/acme/app/pull/104","title":"Merge dev into main","author":"alice","state":"merged","created_at":"2026-01-05T00:00:00Z","merged_at":"2026-01-09T00:00:00Z","source_branch":"dev","target_branch":"main","draft":false,"summary":{"verdict":"request_changes","tier":"XL","round":2,"phase":"shadow"},"history":[{"sha":"d4e5f60718293a4b5c6d7e8f9012345678901a2b","verdict":"request_changes","round":1,"model_called":true,"at":"2026-01-05T01:00:00Z","arm":null,"tier":"XL","usage":{"input":400000,"output":20000,"cache_read":100000}},{"sha":"e5f60718293a4b5c6d7e8f9012345678901a2b3c","verdict":"request_changes","round":2,"model_called":true,"at":"2026-01-07T01:00:00Z","arm":null,"tier":"XL","usage":{"input":200000,"output":10000,"cache_read":300000}},{"sha":"f60718293a4b5c6d7e8f9012345678901a2b3c4d","verdict":"request_changes","round":2,"model_called":false,"at":"2026-01-08T01:00:00Z","arm":null,"tier":"XL"}],"threads":[{"id":"F-2b3c4d","kind":"finding","source":"model","severity":"P0","type":"defect","class":"contract-break","lens":"contracts","title":"Old clients cannot read the new order field","path":"api/schema.proto","created_at":"2026-01-05T01:00:00Z","resolved":false,"resolved_by_bot":false,"resolved_by":null,"human_replies":0,"first_human_reply_at":null,"human_reply_texts":[]},{"id":"F-5e6f7a","kind":"finding","source":"model","severity":"P2","type":"leftover","class":"other","lens":"behavior","title":"The removed flag is still read at startup","path":"old/flag.ts","created_at":"2026-01-05T01:00:00Z","resolved":true,"resolved_by_bot":true,"resolved_by":"github-actions","human_replies":0,"first_human_reply_at":null,"human_reply_texts":[]},{"id":"NH-ui","kind":"ui_evidence","source":"condition","severity":null,"type":null,"class":null,"lens":"","title":"The UI changed without a screenshot","path":"","created_at":"2026-01-07T01:00:00Z","resolved":false,"resolved_by_bot":false,"resolved_by":null,"human_replies":1,"first_human_reply_at":"2026-01-07T05:00:00Z","human_reply_texts":["Screenshot added in the description."]},{"id":"NH-desc","kind":"description","source":"condition","severity":null,"type":null,"class":null,"lens":"","title":"The description has no rollback section","path":"","created_at":"2026-01-05T01:00:00Z","resolved":true,"resolved_by_bot":true,"resolved_by":"github-actions","human_replies":0,"first_human_reply_at":null,"human_reply_texts":[]},{"id":"NH-8b9c0d","kind":"deploy_prerequisite","source":"model","severity":null,"type":null,"class":null,"lens":"contracts","title":"Who runs the migration before the deploy?","path":"db/migrations/0042_orders.sql","created_at":"2026-01-07T01:00:00Z","resolved":false,"resolved_by_bot":false,"resolved_by":null,"human_replies":2,"first_human_reply_at":"2026-01-07T06:00:00Z","human_reply_texts":["Not needed: the migration runs automatically.","ok"]}],"quiz":{"posted":2,"answered":false},"fixes":[],"files":["api/schema.proto","old/flag.ts","db/migrations/0042_orders.sql","apps/web/src/Order.tsx"]}
{"host":"github","project":"acme/app","number":105,"url":"https://github.com/acme/app/pull/105","title":"Share the amount formatter","author":"bob","state":"open","created_at":"2026-01-08T00:00:00Z","merged_at":null,"source_branch":"refactor/format","target_branch":"main","draft":false,"summary":{"verdict":"request_changes","tier":"L","round":1,"phase":"shadow"},"history":[{"sha":"0718293a4b5c6d7e8f9012345678901a2b3c4d5e","verdict":"request_changes","round":1,"model_called":true,"at":"2026-01-08T01:00:00Z","arm":null,"tier":"L","usage":{"input":150000,"output":8000,"cache_read":50000}}],"threads":[{"id":"F-3c4d5e","kind":"finding","source":"model","severity":"P2","type":"maintainability","class":"duplicate-implementation","lens":"duplication","title":"The formatter copies the rounding rule","path":"src/util/format.ts","created_at":"2026-01-08T01:00:00Z","resolved":false,"resolved_by_bot":false,"resolved_by":null,"human_replies":1,"first_human_reply_at":"2026-01-08T02:00:00Z","human_reply_texts":["Later."]},{"id":"F-6f7a8b","kind":"finding","source":"model","severity":"P1","type":"defect","class":"behavior-regression","lens":"behavior","title":"Negative amounts lose their sign","path":"src/util/format.ts","created_at":"2026-01-08T01:00:00Z","resolved":false,"resolved_by_bot":false,"resolved_by":null,"human_replies":0,"first_human_reply_at":null,"human_reply_texts":[]}],"quiz":{"posted":1,"answered":false},"fixes":[],"files":["src/util/format.ts"]}
{"host":"github","project":"acme/app","number":106,"url":"https://github.com/acme/app/pull/106","title":"fix: count a failed login once","author":"alice","state":"merged","created_at":"2026-01-09T00:00:00Z","merged_at":"2026-01-09T12:00:00Z","source_branch":"fix/login-retry","target_branch":"main","draft":false,"summary":{"verdict":"pass","tier":"S","round":1,"phase":"shadow"},"history":[{"sha":"18293a4b5c6d7e8f9012345678901a2b3c4d5e6f","verdict":"pass","round":1,"model_called":true,"at":"2026-01-09T01:00:00Z","arm":null,"tier":"S","usage":{"input":20000,"output":1000,"cache_read":5000}}],"threads":[],"quiz":null,"fixes":[101],"files":["src/auth/login.ts"]}
{"host":"github","project":"acme/app","number":107,"url":"https://github.com/acme/app/pull/107","title":"fix: handler crash on an empty order","author":"carol","state":"merged","created_at":"2026-01-10T00:00:00Z","merged_at":"2026-01-10T03:00:00Z","source_branch":"hotfix/empty-order","target_branch":"main","draft":false,"summary":null,"history":[],"threads":[],"quiz":null,"fixes":[103,104],"files":["docs/README.md","src/server/handler.ts"]}
{"host":"github","project":"acme/app","number":108,"url":"https://github.com/acme/app/pull/108","title":"Try a new cache layer","author":"bob","state":"closed","created_at":"2026-01-03T00:00:00Z","merged_at":null,"source_branch":"spike/cache","target_branch":"main","draft":false,"summary":{"verdict":"needs_human","tier":"S","round":1,"phase":"shadow"},"history":[{"sha":"293a4b5c6d7e8f9012345678901a2b3c4d5e6f70","verdict":"needs_human","round":1,"model_called":true,"at":"2026-01-03T01:00:00Z","arm":null,"tier":"S","usage":{"input":10000,"output":500,"cache_read":0}}],"threads":[{"id":"NH-1e2f3a","kind":"other","source":"model","severity":null,"type":null,"class":null,"lens":"general","title":"Who owns the cache invalidation?","path":"src/cache/layer.ts","created_at":"2026-01-03T01:00:00Z","resolved":false,"resolved_by_bot":false,"resolved_by":null,"human_replies":1,"first_human_reply_at":"2026-01-03T02:00:00Z","human_reply_texts":["@bob can you take a look?"]}],"quiz":null,"fixes":[],"files":["src/cache/layer.ts"]}
receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementationStars: 322kUpdated Claude CodeCodexVerifiedInstall
requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirementsStars: 322kUpdated Claude CodeCodexVerifiedInstall
using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallbackStars: 322kUpdated Claude CodeCodexVerifiedInstall
verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions alwaysStars: 322kUpdated Claude CodeCodexVerifiedInstall