topprismdata/cultivating-ml-agent / skills/examples/cv-lb-gap-acknowledgment
cv-lb-gap-acknowledgment
VerifiedCV-LB Gap Acknowledgment: CV improvement does NOT equal LB improvement. This is one of the most important MLOps principles. Use when: (1) OOF score keeps improving but LB score plateaus or drops, (2) Spending days tuning hyperparameters without LB improvement, (3) Comparing models on CV only without LB validation, (4) Trusting cross-validation as the "final" metric. Covers the mathematical reasons for CV-LB gap (overfitting to OOF noise, distribution shift train vs test, hyperparameter over-tuning to CV), empirical validation (0.005-0.01 gap observed in tabular competitions), the 5-stage validation pipeline (CV → submission → LB → analysis), when to stop iterating on CV alone, and the 80/20 rule for time allocation between CV optimization and LB validation.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 8.6 KB |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |