Skip to content

deanpeters/ai-product-operating-model-skills / skills/aipom-eval-scorecard-builder

aipom-eval-scorecard-builder

Verified

Define calibrated AI evaluation metrics, rubrics, judges, thresholds, sampling, uncertainty, ownership, and decision rules tied to behavior and consequences.

FileKindSize
SKILL.mdskill md4.9 KB
examples/weak-example.mdmarkdown269 B
examples/worked-example.mdmarkdown572 B
template.mdmarkdown649 B

template.md

View on GitHub
# AI Evaluation Scorecard

## Decision and Boundary
- Decision, behavior, population, consequence, owner:

## Measures
| Dimension | Metric / rubric | Unit and denominator | Judge | Sample | Threshold | Critical? |
|---|---|---|---|---|---|---|

## Calibration
- Reference decisions and reviewer qualification:
- Agreement, disagreement, and uncertainty:
- Model-judge checks:

## Views
- Normal, edge, adversarial, subgroup, and production slices:
- Aggregation limits:

## Decision Rules
| Result | Continue / revise / constrain / rollback / stop | Authority | Required evidence | Date |
|---|---|---|---|---|
- Exception and recalibration rules:
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 196kUpdated Claude CodeCodexVerifiedInstall