wshobson/agents / plugins/llm-finetuning/skills/eval-harness-first
eval-harness-first
VerifiedBuild the evaluation harness that gates every fine-tuning run — golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when converting traces into an eval set, or when calibrating a judge against human labels.
Security report
VerifiedStatic analysis of one exact version. It reduces risk and shows evidence; it does not certify that a skill is safe.
- Commit
9c61ab243def98f72da12abd41cfcf5bcd91af9d- Content hash
0e6af8dfaffb00be15160a7941e05b032e418ca67a0c978cf3b404f1729e0896- Scanned
- 2026-11-17
- Scanner version
- skillxone-scan-1.0.0
- Policy version
- 0.1-draft
- Decision
- Every required check completed with no open blocking or medium findings.
- Verification expires
- 2026-12-17
Findings (none)
No findings were recorded.
Checks run
- prompt-injection
- dangerous-commands
- credentials
- network
- file-access
- integrity
Checks skipped
- dependencies: No dependency manifest in the skill directory.
- gitleaks: gitleaks is not installed on this host.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |