sajor2000/methodsmith / plugins/methodsmith/skills/method-model-choice
method-model-choice
VerifiedChooses and justifies the statistical model (regression, mixed/GEE, survival, causal, prediction, Bayesian) after the estimand is set. Use when the user asks which model to fit.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 9.8 KB |
| agents/openai.yaml | config | 252 B |
| references/advanced-models.md | markdown | 4.7 KB |
| references/evidence-access.md | markdown | 3.4 KB |
| references/forward-temporal-validation.md | markdown | 1.5 KB |
| references/mixed-effects.md | markdown | 8.1 KB |
| references/ml-prediction-decisions.md | markdown | 3.0 KB |
| references/ml-prediction-decisions.schema.json | config | 3.4 KB |
| references/ml-training-protocol.md | markdown | 23.4 KB |
| references/model-routing.md | markdown | 8.1 KB |
| references/python-implementation.md | markdown | 3.6 KB |
| references/r-implementation.md | markdown | 2.9 KB |
references/forward-temporal-validation.md
View on GitHub# Forward temporal validation
Fix the intended validation target before selecting a splitter.
- Assess future deployment on observations later than all model-development data, using either a frozen model with an independent later cohort or rolling/expanding calendar windows. Fit preprocessing, tuning, and calibration only within the applicable earlier development data.
- Keep patients grouped across training and assessment. Apply a gap or purge when needed for overlapping prediction windows, delayed labels, or outcome maturation; for a 90-day outcome, confirm training labels would be available at the assessment origin.
- `LeaveOneGroupOut` on calendar period uses every other period for training, including future periods. It can assess a justified historical period-transport target, but cannot establish forward performance. Label that target explicitly.
- `TimeSeriesSplit` assumes ordered samples and equally spaced samples for comparable fold durations. Irregular clinical cohorts may require explicit calendar windows. Check its gap units and patient boundaries; a row-count gap does not automatically represent a clinical time interval.
- Preserve chronology even when outcome stratification would improve class balance. Report period-specific sample sizes, events, calibration, and discrimination.
Source: [scikit-learn TimeSeriesSplit documentation](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html). Site and period transport remain distinct from forward prediction.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |