me2resh/apexyard / .claude/skills/eval-agents
eval-agents
UnverifiedScore a review agent (Rex/Hakim/Tariq) against a labeled PR corpus — ground-truth defect-set overlap, never a prose rubric. Headline metric is approve-precision.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 28.8 KB |
| lib/mine_disagreements.pyexecutable | script | 23.1 KB |
| lib/snapshot-diff.shexecutable | script | 3.3 KB |
| lib/source-snapshot.shexecutable | script | 6.7 KB |
| lib/validate-corpus.shexecutable | script | 8.7 KB |
| tests/fixtures/mine-disagreements/edge-cases.json | config | 4.9 KB |
| tests/fixtures/mine-disagreements/merged-prs.json | config | 10.8 KB |
| tests/smoke.shexecutable | script | 3.4 KB |
| tests/test-snapshot-diff.shexecutable | script | 3.2 KB |
| tests/test_mine_disagreements.py | script | 17.0 KB |
| tests/test_mine_disagreements.shexecutable | script | 5.6 KB |
| tests/test_source_snapshot.shexecutable | script | 11.8 KB |
tests/fixtures/mine-disagreements/merged-prs.json
View on GitHub{
"repo": "example-org/example-repo",
"prs": [
{
"number": 101,
"url": "https://github.com/example-org/example-repo/pull/101",
"reviews": [
{
"id": 1011,
"state": "COMMENTED",
"body": "## Code Review: PR #0\n\n**Commit**: `aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa`\n**Scope**: `Full`\n\n### Summary\nThe change looks fine.\n\n### Verdict\n**APPROVED**\nThe findings support this verdict.\n\n---\nReviewed by Rex (Code Reviewer Agent)\nReviewed commit: `aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa`\n",
"commit_id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"submitted_at": "2026-09-01T10:00:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/101#pullrequestreview-1011",
"user": {
"login": "example-bot"
}
},
{
"id": 1012,
"state": "COMMENTED",
"body": "## Security Review: PR #0\n\n**Commit**: `aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa`\n\n### Summary\nHIGH: the prefix skip fails open.\n\n### Security Issues Found\nSee below.\n\n### Verdict\n**CHANGES REQUESTED**\n\n---\nReviewed by Hakim (Security Auditor)\nReviewed commit: `aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa`\n",
"commit_id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"submitted_at": "2026-09-01T10:10:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/101#pullrequestreview-1012",
"user": {
"login": "example-bot"
}
},
{
"id": 1013,
"state": "APPROVED",
"body": "",
"commit_id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"submitted_at": "2026-09-01T11:00:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/101#pullrequestreview-1013",
"user": {
"login": "example-bot"
}
}
]
},
{
"number": 102,
"url": "https://github.com/example-org/example-repo/pull/102",
"reviews": [
{
"id": 1021,
"state": "COMMENTED",
"body": "## Security Review: PR #0\n\n**Commit**: `bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb`\n\n### Summary\nSummary text.\n\n### Security Issues Found\nSee below.\n\n### Verdict\n**APPROVED**\n\n---\nReviewed by Hakim (Security Auditor)\nReviewed commit: `bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb`\n",
"commit_id": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"submitted_at": "2026-09-02T10:00:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/102#pullrequestreview-1021",
"user": {
"login": "example-bot"
}
},
{
"id": 1022,
"state": "COMMENTED",
"body": "## Code Review: PR #0\n\n**Commit**: `bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb`\n**Scope**: `Full`\n\n### Summary\nThe change looks fine.\n\n### Verdict\n**CHANGES REQUESTED**\nThe findings support this verdict.\n\n---\nReviewed by Rex (Code Reviewer Agent)\nReviewed commit: `bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb`\n",
"commit_id": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"submitted_at": "2026-09-02T10:05:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/102#pullrequestreview-1022",
"user": {
"login": "example-bot"
}
}
]
},
{
"number": 103,
"url": "https://github.com/example-org/example-repo/pull/103",
"reviews": [
{
"id": 1031,
"state": "COMMENTED",
"body": "## Code Review: PR #0\n\n**Commit**: `cccccccccccccccccccccccccccccccccccccccc`\n**Scope**: `Full`\n\n### Summary\nThe change looks fine.\n\n### Verdict\n**APPROVED**\nThe findings support this verdict.\n\n---\nReviewed by Rex (Code Reviewer Agent)\nReviewed commit: `cccccccccccccccccccccccccccccccccccccccc`\n",
"commit_id": "cccccccccccccccccccccccccccccccccccccccc",
"submitted_at": "2026-09-03T10:00:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/103#pullrequestreview-1031",
"user": {
"login": "example-bot"
}
},
{
"id": 1032,
"state": "COMMENTED",
"body": "## Security Review: PR #0\n\n**Commit**: `cccccccccccccccccccccccccccccccccccccccc`\n\n### Summary\nRound 1 was CHANGES REQUESTED. All four BLOCKING findings are fixed.\n\n### Security Issues Found\nSee below.\n\n### Verdict\n**APPROVED**\n\n---\nReviewed by Hakim (Security Auditor)\nReviewed commit: `cccccccccccccccccccccccccccccccccccccccc`\n",
"commit_id": "cccccccccccccccccccccccccccccccccccccccc",
"submitted_at": "2026-09-03T10:05:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/103#pullrequestreview-1032",
"user": {
"login": "example-bot"
}
}
]
},
{
"number": 104,
"url": "https://github.com/example-org/example-repo/pull/104",
"reviews": [
{
"id": 1041,
"state": "COMMENTED",
"body": "## Code Review: PR #0\n\n**Commit**: `dddddddddddddddddddddddddddddddddddddddd`\n**Scope**: `Full`\n\n### Summary\nThe change looks fine.\n\n### Verdict\n**APPROVED**\nThe findings support this verdict.\n\n---\nReviewed by Rex (Code Reviewer Agent)\nReviewed commit: `dddddddddddddddddddddddddddddddddddddddd`\n",
"commit_id": "dddddddddddddddddddddddddddddddddddddddd",
"submitted_at": "2026-09-04T10:00:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/104#pullrequestreview-1041",
"user": {
"login": "example-bot"
}
},
{
"id": 1042,
"state": "COMMENTED",
"body": "## Security Review: PR #0\n\n**Commit**: `eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee`\n\n### Summary\nSummary text.\n\n### Security Issues Found\nSee below.\n\n### Verdict\n**CHANGES REQUESTED**\n\n---\nReviewed by Hakim (Security Auditor)\nReviewed commit: `eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee`\n",
"commit_id": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
"submitted_at": "2026-09-04T10:05:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/104#pullrequestreview-1042",
"user": {
"login": "example-bot"
}
}
]
},
{
"number": 105,
"url": "https://github.com/example-org/example-repo/pull/105",
"reviews": [
{
"id": 1051,
"state": "COMMENTED",
"body": "The verdict is APPROVED. Next action is design review.\n\n## Code Review: PR #0\n\n**Commit**: `f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0`\n**Scope**: `Full`\n\n### Summary\nThe change looks fine.\n\n### Verdict\n**APPROVED**\nThe findings support this verdict.\n\n---\nReviewed by Rex (Code Reviewer Agent)\nReviewed commit: `f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0`\n",
"commit_id": "f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0",
"submitted_at": "2026-09-05T10:00:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/105#pullrequestreview-1051",
"user": {
"login": "example-bot"
}
},
{
"id": 1052,
"state": "COMMENTED",
"body": "## Design Review: PR #0\n\n**Commit**: `f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0`\n\n### Blocking Findings\nB1: the design skips the rollback path.\n\n### Verdict\n**CHANGES REQUESTED**\nFix B1.\n\n---\nReviewed by Tariq (Solution Architect)\nReviewed commit: `f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0`\n",
"commit_id": "f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0",
"submitted_at": "2026-09-05T10:05:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/105#pullrequestreview-1052",
"user": {
"login": "example-bot"
}
}
]
},
{
"number": 106,
"url": "https://github.com/example-org/example-repo/pull/106",
"reviews": [
{
"id": 1061,
"state": "COMMENTED",
"body": "## Code Review: PR #0\n\n**Commit**: `11111111`\n**Scope**: `Full`\n\n### Summary\nThe change looks fine.\n\n### Verdict\n**APPROVED**\nThe findings support this verdict.\n\n---\nReviewed by Rex (Code Reviewer Agent)\nReviewed commit: `11111111`\n",
"commit_id": "1111111111111111111111111111111111111111",
"submitted_at": "2026-09-06T10:00:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/106#pullrequestreview-1061",
"user": {
"login": "example-bot"
}
},
{
"id": 1062,
"state": "COMMENTED",
"body": "## Security Review: PR #0\n\n**Commit**: `1111111111111111111111111111111111111111`\n\n### Summary\nSummary text.\n\n### Security Issues Found\nSee below.\n\n### Verdict\n**BLOCKING**\n\n---\nReviewed by Hakim (Security Auditor)\nReviewed commit: `1111111111111111111111111111111111111111`\n",
"commit_id": "2222222222222222222222222222222222222222",
"submitted_at": "2026-09-06T10:30:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/106#pullrequestreview-1062",
"user": {
"login": "example-bot"
}
}
]
},
{
"number": 107,
"url": "https://github.com/example-org/example-repo/pull/107",
"reviews": [
{
"id": 1071,
"state": "COMMENTED",
"body": "## Code Review: PR #0\n\n**Commit**: `3333333333333333333333333333333333333333`\n**Scope**: `Full`\n\n### Summary\nThe change looks fine.\n\n### Verdict\n**APPROVED**\nThe findings support this verdict.\n\n---\nReviewed by Rex (Code Reviewer Agent)\nReviewed commit: `3333333333333333333333333333333333333333`\n",
"commit_id": "3333333333333333333333333333333333333333",
"submitted_at": "2026-09-07T10:00:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/107#pullrequestreview-1071",
"user": {
"login": "example-bot"
}
},
{
"id": 1072,
"state": "COMMENTED",
"body": "## Security Review: PR #0\n\n**Commit**: `3333333333333333333333333333333333333333`\n\n### Summary\nSummary text.\n\n### Security Issues Found\nSee below.\n\n### Verdict\n**[APPROVED / CHANGES REQUESTED / COMMENT]**\n\n---\nReviewed by Hakim (Security Auditor)\nReviewed commit: `3333333333333333333333333333333333333333`\n",
"commit_id": "3333333333333333333333333333333333333333",
"submitted_at": "2026-09-07T10:05:00Z",
"html_url": "https://github.com/example-org/example-repo/pull/107#pullrequestreview-1072",
"user": {
"login": "example-bot"
}
}
]
}
]
}
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Unverified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |