The-AI-Directory-Company/agents-and-skills / skills/hiring-rubric
hiring-rubric
VerifiedBuild structured hiring rubrics — defining evaluation dimensions, behavioral interview questions, scoring criteria, and calibration processes that maximize signal and minimize bias.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 6.4 KB |
| assets/scorecard-template.md | markdown | 3.2 KB |
| examples/product-manager-mid-level.md | markdown | 8.7 KB |
| examples/senior-backend-engineer.md | markdown | 4.6 KB |
| references/calibration-workshop-guide.md | markdown | 5.9 KB |
references/calibration-workshop-guide.md
View on GitHub# Calibration Workshop Guide
How to run a pre-interview calibration session that aligns the interview panel on scoring standards before any candidate walks through the door.
## Why Calibrate
Without calibration, a "3" means something different to every interviewer. One person's "Hire" is another's "Strong Hire." The debrief becomes a negotiation about standards instead of a discussion about the candidate. Calibration fixes this by anchoring the panel to shared examples before interviews begin.
## When to Run
- **Before every new role's interview loop begins.** Not once per quarter — once per role.
- **When the panel changes.** A new interviewer joining mid-loop needs calibration, even if others already completed it.
- **Duration:** 60-90 minutes. Do not try to compress this into 30 minutes.
## Pre-Workshop Preparation
### 1. Select Mock Clips (Owner: Hiring Manager)
Choose 2-3 recorded interview clips or written transcripts that represent different performance levels. Ideal sources:
- Past interview recordings (anonymized) at different score levels
- Role-played interviews recorded by team members
- Written vignettes if recordings are unavailable
**Clip selection criteria:**
| Clip | Target Score | Why This Clip |
|------|-------------|---------------|
| Clip A | 2 (Weak) | Shows a common failure mode for this role — plausible but flawed |
| Clip B | 3 (Hire) | Solid performance with minor gaps — the hardest score to calibrate |
| Clip C | 4 (Strong Hire) | Clearly exceptional, but the specific "why" is worth discussing |
The 3 is the most important clip. Most disagreements happen at the Hire/Weak boundary.
### 2. Distribute Materials (2 days before)
Send to all panelists:
- The role's hiring rubric (dimensions, weights, scoring criteria)
- Blank scorecards
- The mock clips or transcripts
- Instructions: "Score each clip independently before the workshop. Bring your completed scorecards."
## Workshop Agenda
### Part 1: Independent Scoring Review (15 minutes)
Confirm everyone has scored the clips. Do not allow scoring during the meeting — it defeats the purpose. If someone did not complete the pre-work, they observe but do not participate in scoring discussions.
### Part 2: Score Reveal — Clip by Clip (30-40 minutes)
For each clip:
1. **Silent reveal.** Everyone holds up their score simultaneously (use a shared doc, chat, or literal fingers). No one speaks first.
2. **Identify disagreements.** Note where scores differ by 2+ points. Minor variance (3 vs. 4) is normal; large gaps (2 vs. 4) indicate a calibration problem.
3. **Lowest scorer speaks first.** The person who gave the lowest score explains their reasoning with specific evidence from the clip. This prevents anchoring to the majority view.
4. **Highest scorer responds.** They explain what they saw differently, citing specific moments.
5. **Group discussion.** Focus on: "What specific behavior would move this from a 2 to a 3?" or "What was missing that kept this from being a 4?"
6. **Agree on the reference score.** The group does not need to reach unanimous agreement, but the hiring manager should document the consensus score and the key behavioral markers that justify it.
### Part 3: Anchor Language (15-20 minutes)
For each dimension, write down 1-2 sentences that complete these phrases:
- "A 3 on [dimension] means the candidate..."
- "The difference between a 2 and a 3 on [dimension] is..."
- "A 4 requires the candidate to... without prompting."
Write these in shared notes. Interviewers should reference them when filling out scorecards during actual interviews.
### Part 4: Process Alignment (10 minutes)
Confirm logistics:
- Scorecard submission deadline (within 24 hours of interview)
- Debrief format (most junior interviewer presents first)
- Escalation path for split decisions
- Reminder: scores must be submitted before debrief — no post-hoc adjustments
## Disagreement Resolution Patterns
When the panel cannot agree on a reference score, use these techniques:
### Behavioral Decomposition
Break the disagreement into specific behaviors: "We agree they identified the edge case (that's a 3-level behavior for problem-solving). The question is whether proposing the fix unprompted makes it a 4 or whether the fix itself was too shallow to qualify."
### Threshold Questions
Ask: "If this were the only data point we had for this dimension, would you hire this person?" If the answer splits, the clip is genuinely borderline — document it as a 2-3 boundary example rather than forcing consensus.
### Role-Specific Calibration
For dimensions that are unusually important to this role, tighten the standard: "For a senior infrastructure engineer, a 3 on system design requires multi-region considerations. For a junior role, single-region with acknowledged limitations is a 3."
## Anti-Patterns to Avoid
- **Calibrating on perfect candidates.** Use clips that show flawed-but-reasonable performance. A clip where everything goes right does not help distinguish 3 from 4.
- **Hiring manager anchoring.** The hiring manager should reveal their score last, not first. If they speak first, the room adjusts to them.
- **Skipping the 3.** Everyone can identify a 1 and a 4. The 3 is where calibration matters most. Spend the majority of your time on Hire-level clips.
- **One-time calibration for ongoing roles.** If you run the same role across multiple quarters, re-calibrate when the panel changes or when debrief disagreements become frequent.
- **Rushing the workshop.** Cutting it to 30 minutes means you skip the anchor language step, which is where the lasting value is. Protect the full 60-90 minutes.
## Output Artifacts
By the end of the workshop, the hiring manager should have:
1. **Reference scores** for 2-3 clips with documented reasoning
2. **Anchor language** for each dimension at the 2/3 and 3/4 boundaries
3. **Process confirmation** — submission deadlines, debrief order, escalation path
4. **Attendance record** — who calibrated, who still needs to before interviewing
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |