Skip to content

me2resh/apexyard / .claude/skills/eval-agents

eval-agents

Unverified

Score a review agent (Rex/Hakim/Tariq) against a labeled PR corpus — ground-truth defect-set overlap, never a prose rubric. Headline metric is approve-precision.

FileKindSize
SKILL.mdskill md28.8 KB
lib/mine_disagreements.pyexecutablescript23.1 KB
lib/snapshot-diff.shexecutablescript3.3 KB
lib/source-snapshot.shexecutablescript6.7 KB
lib/validate-corpus.shexecutablescript8.7 KB
tests/fixtures/mine-disagreements/edge-cases.jsonconfig4.9 KB
tests/fixtures/mine-disagreements/merged-prs.jsonconfig10.8 KB
tests/smoke.shexecutablescript3.4 KB
tests/test-snapshot-diff.shexecutablescript3.2 KB
tests/test_mine_disagreements.pyscript17.0 KB
tests/test_mine_disagreements.shexecutablescript5.6 KB
tests/test_source_snapshot.shexecutablescript11.8 KB

tests/fixtures/mine-disagreements/edge-cases.json

View on GitHub
{
  "repo": "example-org/example-repo",
  "prs": [
    {
      "number": 201,
      "url": "https://github.com/example-org/example-repo/pull/201",
      "reviews": [
        {
          "id": 2011,
          "state": "COMMENTED",
          "body": "## Code Review: PR #201\n\n### Verdict\n**APPROVED**\n\n---\nLast reviewed commit: `aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa`\nReviewed commit: `bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb`\n",
          "commit_id": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
          "submitted_at": "2026-09-01T10:01:00Z",
          "html_url": "https://github.com/example-org/example-repo/pull/201#pullrequestreview-2011",
          "user": {
            "login": "example-bot"
          }
        },
        {
          "id": 2012,
          "state": "COMMENTED",
          "body": "## Security Review: PR #201\n\n### Verdict\n**CHANGES REQUESTED**\n\n---\nReviewed commit: `bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb`\n",
          "commit_id": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
          "submitted_at": "2026-09-01T10:02:00Z",
          "html_url": "https://github.com/example-org/example-repo/pull/201#pullrequestreview-2012",
          "user": {
            "login": "example-bot"
          }
        }
      ]
    },
    {
      "number": 202,
      "url": "https://github.com/example-org/example-repo/pull/202",
      "reviews": [
        {
          "id": 2021,
          "state": "COMMENTED",
          "body": "## Code Review: PR #202\n\n### Verdict\n**APPROVED**\n\n---\nReviewed commit: `cccccccccccccccccccccccccccccccccccccccc`\n",
          "commit_id": "cccccccccccccccccccccccccccccccccccccccc",
          "submitted_at": "2026-09-01T10:01:00Z",
          "html_url": "https://github.com/example-org/example-repo/pull/202#pullrequestreview-2021",
          "user": {
            "login": "example-bot"
          }
        },
        {
          "id": 2022,
          "state": "COMMENTED",
          "body": "## Security Review: PR #202\n\n### Verdict\nNo blocking findings remain.\n\n---\nReviewed commit: `cccccccccccccccccccccccccccccccccccccccc`\n",
          "commit_id": "cccccccccccccccccccccccccccccccccccccccc",
          "submitted_at": "2026-09-01T10:02:00Z",
          "html_url": "https://github.com/example-org/example-repo/pull/202#pullrequestreview-2022",
          "user": {
            "login": "example-bot"
          }
        }
      ]
    },
    {
      "number": 203,
      "url": "https://github.com/example-org/example-repo/pull/203",
      "reviews": [
        {
          "id": 2031,
          "state": "COMMENTED",
          "body": "## Code Review: PR #203\n\n### Verdict\n**APPROVED**\n\n---\nReviewed commit: `dddddddddddddddddddddddddddddddddddddddd`\n",
          "commit_id": "dddddddddddddddddddddddddddddddddddddddd",
          "submitted_at": "2026-09-01T10:01:00Z",
          "html_url": "https://github.com/example-org/example-repo/pull/203#pullrequestreview-2031",
          "user": {
            "login": "example-bot"
          }
        },
        {
          "id": 2032,
          "state": "COMMENTED",
          "body": "## Security Review: PR #203\n\n### Verdict\n**CHANGES REQUESTED**\n\n---\nReviewed commit: `dddddddd`\n",
          "commit_id": null,
          "submitted_at": "2026-09-01T10:02:00Z",
          "html_url": "https://github.com/example-org/example-repo/pull/203#pullrequestreview-2032",
          "user": {
            "login": "example-bot"
          }
        }
      ]
    },
    {
      "number": 1080,
      "url": "https://example.test/pr/1080",
      "reviews": [
        {
          "body": "## Code Review: PR #1080\n\n**Commit**: `63177daa`\n\n### Verdict\n**APPROVED**\n\n---\nReviewed by Rex\nReviewed commit: `63177daa`\n",
          "commit_id": "596a95b4",
          "submitted_at": "2026-09-01T10:01:00Z",
          "html_url": "https://example.test/pr/1080/review/1"
        },
        {
          "body": "## Security Review: PR #1080\n\n**Commit**: `63177daa`\n\n### Verdict\n**CHANGES REQUESTED**\n\n---\nReviewed by Hakim\nReviewed commit: `63177daa`\n",
          "commit_id": "63177daa",
          "submitted_at": "2026-09-01T10:02:00Z",
          "html_url": "https://example.test/pr/1080/review/2"
        }
      ]
    },
    {
      "number": 1292,
      "url": "https://example.test/pr/1292",
      "reviews": [
        {
          "body": "## Code Review: PR #1292\n\n### Verdict\n**APPROVED**\n\n---\nReviewed by Rex\nReviewed commit: `abcdef01`\n",
          "commit_id": "8e5c33ea",
          "submitted_at": "2026-09-01T10:01:00Z",
          "html_url": "https://example.test/pr/1292/review/1"
        },
        {
          "body": "## Security Review: PR #1292\n\n**Commit**: `8e5c33ea`\n\n### Verdict\n**CHANGES REQUESTED**\n\n---\nReviewed by Hakim\nReviewed commit: `8e5c33ea`\n",
          "commit_id": "8e5c33ea",
          "submitted_at": "2026-09-01T10:02:00Z",
          "html_url": "https://example.test/pr/1292/review/2"
        }
      ]
    }
  ]
}
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall