existential-birds/beagle / plugins/beagle-analysis/skills/llm-judge
llm-judge
VerifiedUse when comparing two or more code implementations against a spec or requirements doc. Triggers on "which repo is better", "compare these implementations", "evaluate both solutions", "rank these codebases", or "judge which approach wins". Also covers choosing between competing PRs or vendor submissions solving the same problem. Does NOT review a single codebase for quality — use code review skills instead. Does NOT evaluate strategy docs — use strategy-review. Requires a spec file and 2+ repo paths.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 10.7 KB |
| references/fact-schema.md | markdown | 4.3 KB |
| references/judge-agents.md | markdown | 2.7 KB |
| references/repo-agent.md | markdown | 3.1 KB |
| references/scoring-rubrics.md | markdown | 4.1 KB |
references/judge-agents.md
View on GitHub# Judge Agent Instructions
Instructions for Phase 2 agents that score implementations on a single dimension.
## Role
You are a scoring judge. You receive facts gathered from ALL repositories and score each one on YOUR specific dimension using the rubrics in [scoring-rubrics.md](scoring-rubrics.md).
## Inputs You Receive
1. **Spec Document**: The original requirements
2. **Facts Array**: JSON facts from all repos (output of Phase 1)
3. **Your Dimension**: One of: functionality, security, tests, overengineering, dead_code
## Your Task
Produce a JSON object with scores and justifications for each repo.
## Output Schema
```json
{
"dimension": "functionality",
"scores": {
"RepoLabel1": {
"score": 4,
"justification": "Clear explanation of why this score was assigned",
"evidence": ["Specific facts that support this score"]
},
"RepoLabel2": {
"score": 5,
"justification": "...",
"evidence": ["..."]
}
},
"ranking": ["RepoLabel2", "RepoLabel1"],
"notes": "Optional comparative notes"
}
```
## Scoring Process
1. Read the rubric for your dimension from [scoring-rubrics.md](scoring-rubrics.md)
2. For each repo's facts:
- Extract the relevant section (e.g., `facts.functionality` for functionality judge)
- Apply the rubric criteria
- Assign a 1-5 score
- Write a clear justification citing specific evidence
3. Rank the repos by score (highest first)
## Dimension-Specific Instructions
### Functionality Judge
Focus on `facts.functionality`:
- Compare `spec_requirements` to `implemented` and `missing`
- Weight test results heavily (`test_results.passed` vs `failed`)
- Consider `partially_implemented` as half credit
### Security Judge
Focus on `facts.security`:
- Count and weight `findings` by severity
- High severity = major deduction
- Positive `patterns_observed` can offset minor issues
### Tests Judge
Focus on `facts.tests`:
- Evaluate `coverage_estimate`
- Count `dry_violations` (more = worse)
- Consider `mocking_approach` quality
- Raw `test_count` relative to codebase size
### Overengineering Judge
Focus on `facts.overengineering`:
- Count `abstractions` issues
- Count `defensive_code` issues
- Consider `config_complexity`
- FEWER issues = HIGHER score (inverse)
### Dead Code Judge
Focus on `facts.dead_code`:
- Sum all unused items
- Weight `unused_functions` > `unused_imports`
- Count `todo_comments` and `commented_code_blocks`
- FEWER issues = HIGHER score (inverse)
## Important Rules
1. **Use the rubric** - Don't invent criteria
2. **Be consistent** - Apply the same standards to all repos
3. **Cite evidence** - Every score needs justification from facts
4. **Be comparative** - Rankings should reflect relative quality
5. **Valid JSON only** - Output must be parseable
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-changelog-updatemorluto/reaPrepare or rewrite REA release changelogs and GitHub release notes from pinned Git history, with verified contributor thanks and Release Please synchronization. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| academy-guideanthropics/skillsStop and check this skill before finishing any reply to a question about how to use Claude or a Claude product — it recommends matching courses, tutorials, and use cases from Claude Academy (academy.claude.com), Anthropic's learning hub. Trigger on: "how do I", "how can I", "getting started with", "what can Claude do", "teach me", "learn to use"; questions about artifacts, projects, skills, plugins, connectors, MCP; requests about rolling Claude out to a team, class, or organization; and any ask for training materials, onboarding content, or learning resources. Use it when the user is learning how to use a feature or product — not when they are mid-task and just want the task done. This skill composes with other skills: after consulting product documentation to answer how a Claude feature works, also check here for a matching course or tutorial — a docs-grounded answer and an Academy recommendation belong together. Only recommend on a strong match; never invent Academy content. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| claude-apianthropics/skillsReference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model migration. TRIGGER — read BEFORE opening the target file; don't skip because it "looks like a one-liner" — whenever: the prompt names Claude/Anthropic in any form (Claude, Anthropic, Fable, Opus, Sonnet, Haiku, `anthropic`, `@anthropic-ai`, `claude-*`, `us.anthropic.*`, `[1m]`); the user asks about an LLM (pricing/model choice/limits/caching) — never answer from memory; OR the task is LLM-shaped with provider unstated (agent/MCP/tool-definition/multi-agent/RAG/LLM-judge/computer-use; generate/summarize/extract/classify/rewrite/converse over NL; debugging refusals/cutoffs/streaming/tool-calls/tokens). SKIP only when another provider is being worked on (overrides all triggers): OpenAI/GPT/Gemini/Llama/Mistral/Cohere/Ollama named in the query; OR `grep -rE 'openai|langchain_openai|google.generativeai|genai|mistralai|cohere|ollama'` over the project hits (run this grep FIRST if no provider named — don't Read the file). | Stars: 198k | Updated | Claude CodeCodex | Unverified | Install |
| doc-coauthoringanthropics/skillsGuide users through a structured workflow for co-authoring documentation. Use when user wants to write documentation, proposals, technical specs, decision docs, or similar structured content. This workflow helps users efficiently transfer context, refine content through iteration, and verify the doc works for readers. Trigger when user mentions writing docs, creating proposals, drafting specs, or similar documentation tasks. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |