Skip to content

topprismdata/cultivating-ml-agent / skills/examples/autogluon-first

autogluon-first

Verified

AutoGluon-First Strategy: Always run AutoGluon `best_quality` preset as the first step in any tabular ML competition (5-15 min baseline). Validated 3/4 times vs manual ensembles on small/medium tabular datasets. Use when: (1) Starting any new tabular competition, (2) Need a strong baseline in <15 minutes, (3) Manual GBDT work takes hours but gives similar results, (4) Want to focus manual effort on complementary techniques (deep learning, external data, special features) rather than reimplementing what AutoGluon already does automatically. Covers 5 core reasons for success (multi-algorithm diversity, multi-level stacking, automated preprocessing, bagging/multi-fold, Bayesian optimization), when to AVOID AutoGluon (text/image data, multi-output regression, >1M rows with limited RAM), and how to use it as a Silver signal in custom pipelines.

AutoGluon-First Strategy

Problem

Manual ML on tabular problems often wastes time:

  • Hours tuning GBDT hyperparameters (LightGBM, XGBoost)
  • Manual feature engineering trial-and-error
  • Manual ensemble design (which models? which weights?)
  • Reaches a plateau, then struggles to break through

Reality: AutoGluon's best_quality preset achieves in 5-15 minutes what takes humans days.

Context / Trigger Conditions

Use this skill when:

  • Starting any new tabular competition (regression or classification)
  • Need a strong baseline in <15 minutes
  • Manual GBDT work takes hours but gives similar results
  • Want to focus manual effort on complementary techniques (deep learning, external data)
  • Unsure which algorithm to use (LightGBM vs XGBoost vs CatBoost vs NeuralNet)
  • Need to compare your work against a known strong baseline

Solution

Step 1: Run AutoGluon Baseline (5-15 minutes)

import time
from autogluon.tabular import TabularPredictor

label = 'target'  # Your target column
save_path = f'ag_baseline_{int(time.time())}'

predictor = TabularPredictor(
    label=label,
    path=save_path,
    eval_metric='accuracy',  # or 'rmse', 'roc_auc', 'log_loss'
    verbosity=1
).fit(
    train_data,
    presets='best_quality',  # 10+ algorithms, multi-level stacking
    time_limit=900  # 15 minutes
)

That's it. In 5-15 minutes you have:

  • 10+ models trained (LightGBM, XGBoost, CatBoost, RF, ExtraTrees, KNN, NN)
  • Multi-level stacking (Level 1, 2, 3)
  • OOF predictions for validation
  • Test predictions for submission

Step 2: Validate and Compare

# Get OOF score
oof = predictor.predict_oof()
test_pred = predictor.predict(test_data)

# Compare to your manual work
# - If AutoGluon matches you: Stop, don't waste time
# - If AutoGluon beats you: Learn from its ensemble
# - If AutoGluon loses: Use it as Silver signal, focus on what's different

5 Core Reasons AutoGluon Works

1. Multi-Algorithm Diversity

AlgorithmInductive BiasStrengths
LightGBM/XGBoost/CatBoostTree-based splitsCategorical features, missing values
Random ForestBagging + treesAnti-overfitting, feature selection
ExtraTreesRandom splitsLow variance
KNNLocal smoothingSmall samples, manifold structure
NeuralNet (MLP)Universal approximationComplex non-linearity
Linear modelsLinearSparse data, extrapolation

Manual ensembles fail because they use only 3 GBDTs (high correlation → high Cov). AutoGluon wins because it uses 10+ algorithms (low correlation → low Cov → low ensemble variance).

2. Multi-Level Stacking

Level 1: y1_i = model_i(X)   for i in 10 algorithms
Level 2: y2 = meta_learner(y1_1, y1_2, ..., y1_10)
Level 3: y3 = meta_meta_learner(y2, y1_combined)

Single-level averaging only learns linear combinations. Multi-level stacking learns conditional combinations:

  • "If LightGBM predicts high AND KNN predicts low, use XGBoost"
  • "If dataset is small, weight NeuralNet higher"

3. Automated Preprocessing

Each model needs different preprocessing:

  • Tree models: Categorical encoding, missing value imputation
  • NeuralNet: Standardization, OneHot
  • KNN: Distance normalization
  • Linear: Standardization

AutoGluon automatically does per-model preprocessing, while manual ensembles use uniform preprocessing (often suboptimal).

4. Bagging + Multi-Fold

For each model:
    for each fold (5-fold):
        train on 4 folds, predict on 1 fold
    average predictions

For small datasets especially, this dramatically reduces variance (the biggest bottleneck for small data).

5. Bayesian Optimization + Early Stopping

  • Bayesian optimization predicts next hyperparams based on history
  • Early stopping terminates bad models early
  • Resource allocation trains good models more, bad models less

For small datasets: each train is cheap → can try more models → finds global optimum.

Validation Results (2026-06-14)

CompetitionDataset SizeAutoGluon OOFManual BestVerdict
Titanic891 rows0.8552 acc~0.82 typical✅ AutoGluon wins
Leaf Classification990 rows0.9889 acc, 0.0300 loglossGold 1.0⚠️ Close (0.0111 gap)
House Prices1460 rows0.118033 RMSE logV18 0.119350✅ AutoGluon wins
TPS May 2022900K rows0.8757 accSilver 0.99754❌ Manual wins (special method)

Win rate: 2/4 AutoGluon wins, 1/4 close, 1/4 manual wins (special method needed)

When to AVOID AutoGluon Tabular

Problem TypeReasonAlternative
Text/NLPAutoGluon tabular doesn't do NLPXLM-R, BERT, TabPFN
ImageNeeds CNN/ViT, not tabular featuresResNet, ViT
Sequence (text→text)Seq2seq, not classificationT5, BART
Multi-output regressionAutoGluon doesn't natively supportTrain one model per output
Custom domain (chemistry, physics)Needs domain-specific featuresDomain knowledge + GBDT
Image-based regressionNeeds CNN features, not raw pixelsPretrained CNN + GBDT
>1M rows, <16GB RAMResource constraints, slowSubsample or use GPU

How to Use AutoGluon in Custom Pipelines

Option 1: AutoGluon as Baseline (5-15 min)

  • Run AutoGluon → get baseline OOF
  • Compare to your manual work
  • If AutoGluon matches you: Stop, don't waste time
  • If AutoGluon beats you: Use it directly, focus elsewhere

Option 2: AutoGluon as Silver Signal (House Prices SST-style)

  • Run AutoGluon → get OOF + test predictions
  • Train your manual ensemble separately
  • Blend: 0.30-0.50 × AutoGluon + 0.50-0.70 × Manual
  • This adds diversity that AutoGluon's internal ensemble might miss

Option 3: AutoGluon + Special Methods

  • Run AutoGluon first
  • Add complementary approach (deep learning, external data, special features)
  • Validate via CV that special method adds signal beyond AutoGluon

Anti-Patterns to Avoid

❌ Don't skip AutoGluon to "save time on small data" — Titanic 5 min AutoGluon = 0.8552 ❌ Don't trust AutoGluon for text/image — needs different approach ❌ Don't assume AutoGluon = best — TPS May 2022 had manual Silver 12% better ❌ Don't use AutoGluon as final without LB validation — CV-LB gap exists ❌ Don't spend days tuning GBDT if AutoGluon matches in 15 min

Empirical Pattern: When Manual Wins Over AutoGluon

  • Special domain knowledge (chemistry, physics, biology)
  • External data integration that AutoGluon doesn't have access to
  • Custom feature engineering that captures domain-specific signals
  • Advanced ensemble techniques beyond AutoGluon's multi-level stacking

Related Skills

  • catboost-first-tabular — When AutoGluon doesn't work, try CatBoost (House Prices V17 lesson)
  • cv-lb-validation — Always validate AutoGluon on LB (CV-LB gap exists)
  • multi-model-diversity — AutoGluon implements this internally (10+ algorithms)
  • asymmetric-blending — AutoGluon can be one part of a larger blend
  • ml-sweet-spot — When to stop optimizing and accept AutoGluon baseline
  • kaggle-competition-best-practices — Full competition workflow with AutoGluon

Validation Source

  • House Prices: AutoGluon OOF 0.1180 vs V18 manual 0.1194 (2026-06-12)
  • Leaf Classification: AutoGluon 0.9889 vs Gold 1.0 (2026-06-14)
  • Titanic: AutoGluon 0.8552 vs classic 0.82 (2026-06-14)
  • TPS May 2022: AutoGluon 0.8757 vs Silver 0.9975 (2026-06-14)
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 321kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall