ericrisco/rsc-harness / skills/machine-learning
machine-learning
VerifiedUse when predicting a column from rows of tabular features with classic models — scikit-learn pipelines, RandomForest, XGBoost/LightGBM, leak-free cross-validation, metrics for imbalanced classes, or a model that aced CV then collapsed in production. NOT PyTorch neural nets (that is `deep-learning`), NOT cleaning the dirty table first (that is `data-cleaning`), NOT forecasting a dated series (that is `forecasting`), NOT text/token modeling (that is `nlp`).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 16.6 KB |
| evals/README.md | markdown | 1.2 KB |
| evals/cases.yaml | config | 4.7 KB |
| references/gbdt-and-tuning.md | markdown | 4.5 KB |
| references/metrics-and-imbalance.md | markdown | 4.4 KB |
| references/pipelines-and-cv.md | markdown | 5.7 KB |
references/gbdt-and-tuning.md
View on GitHub# GBDTs: parameters, early stopping, categoricals, importance (depth)
Three interchangeable sklearn-API GBDTs. Versions verified 2026-07 — XGBoost 3.x, LightGBM 4.x, and
sklearn's `HistGradientBoosting*` (~1.9). Verify current versions before pinning; APIs shift between majors.
## What the knobs mean (they translate across libraries)
| Concept | HistGradientBoosting | XGBoost | LightGBM |
| --- | --- | --- | --- |
| Step size | `learning_rate` | `learning_rate` (`eta`) | `learning_rate` |
| Number of trees | `max_iter` | `n_estimators` | `n_estimators` |
| Tree size | `max_leaf_nodes` / `max_depth` | `max_depth` | `num_leaves` (leaf-wise!) / `max_depth` |
| L2 penalty | `l2_regularization` | `reg_lambda` / `reg_alpha` | `reg_lambda` / `reg_alpha` |
| Min data per leaf | `min_samples_leaf` | `min_child_weight` | `min_child_samples` |
| Row/col subsample | (n/a) | `subsample` / `colsample_bytree` | `bagging_fraction` / `feature_fraction` |
Tuning order that pays off: get `learning_rate` low-ish with enough trees (use early stopping to pick the
count), then control complexity (`num_leaves`/`max_depth`, `min_*_leaf`), then regularization and subsampling.
**Lower `learning_rate` + more trees + early stopping** beats a high learning rate almost always. LightGBM
grows **leaf-wise**, so cap `num_leaves` (and/or `max_depth`) or it overfits fast.
## Early stopping (stop adding trees when validation stops improving)
Needs a validation signal. Keep it out of the sacred test set — use a train-derived eval split.
```python
# XGBoost 3.x: early_stopping_rounds is a CONSTRUCTOR param; pass eval_set to fit()
from xgboost import XGBClassifier
xgb = XGBClassifier(n_estimators=2000, learning_rate=0.05,
early_stopping_rounds=50, eval_metric="aucpr")
xgb.fit(X_tr, y_tr, eval_set=[(X_val, y_val)], verbose=False)
print(xgb.best_iteration)
# LightGBM 4.x: callbacks
from lightgbm import LGBMClassifier, early_stopping, log_evaluation
lgbm = LGBMClassifier(n_estimators=2000, learning_rate=0.05, num_leaves=31)
lgbm.fit(X_tr, y_tr, eval_set=[(X_val, y_val)], eval_metric="average_precision",
callbacks=[early_stopping(50), log_evaluation(0)])
# HistGradientBoosting: built-in, no eval_set needed (carves its own validation slice)
from sklearn.ensemble import HistGradientBoostingClassifier
hgb = HistGradientBoostingClassifier(max_iter=2000, learning_rate=0.05,
early_stopping=True, validation_fraction=0.1,
n_iter_no_change=50, random_state=0)
```
## Native categorical handling (skip one-hot)
- **HistGradientBoosting**: `categorical_features="from_dtype"` — columns with pandas `category` dtype are
handled natively. No `OneHotEncoder`, no explosion of columns.
- **XGBoost**: `enable_categorical=True` + columns as pandas `category` dtype; `tree_method` `hist`/`approx`.
- **LightGBM**: pass `categorical_feature=` (names/indices) or use `category` dtype; native by default.
One-hot only when cardinality is low and a *linear* model needs it. For trees, native categorical or, for
very high cardinality, `TargetEncoder` (cross-fit — see pipelines-and-cv.md) beats a thousand dummy columns.
## Monotonic constraints (domain priors)
Force the model's response to move only one way with a feature (price ↑ ⇒ risk never ↓). Guards against
noise inverting a known relationship and helps trust/compliance.
```python
# HistGradientBoosting: dict of feature -> {-1, 0, +1}
HistGradientBoostingClassifier(monotonic_cst={"income": +1, "num_late_payments": -1})
# XGBoost: monotone_constraints="(1,0,-1,...)" LightGBM: monotone_constraints=[1,0,-1,...]
```
## Feature importance — don't trust the default blindly
- **Impurity / gain importance** (`feature_importances_`): fast but biased toward high-cardinality and
continuous features; computed on train.
- **Permutation importance** (`sklearn.inspection.permutation_importance`): model-agnostic, measures the
metric drop when a feature is shuffled — run it **on held-out data**, and beware correlated features
sharing credit.
- **SHAP** (separate `shap` library): per-prediction attributions; the most faithful local explanations,
the heaviest to compute. Use for "why this row" and to sanity-check for leakage — a feature dominating
SHAP that shouldn't be predictive is a leak flag.
```python
from sklearn.inspection import permutation_importance
r = permutation_importance(model, X_test, y_test, scoring="average_precision",
n_repeats=10, random_state=0)
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |