Skip to content

DAAF-Contribution-Community/daaf / .claude/skills/scikit-learn

scikit-learn

Verified

Machine learning: clustering, PCA/t-SNE/UMAP, classification, prediction regression (Ridge/Lasso/ensemble), cross-validation, Pipelines. For unsupervised analysis, classification, or prediction. For econometric regression use pyfixest/statsmodels. R equivalent: tidymodels (use when execution language is R).

references/mixture-models.md

View on GitHub
# Mixture Models

GaussianMixture and BayesianGaussianMixture for model-based clustering, soft assignments, and principled model selection with BIC/AIC. For methodology guidance (when to prefer GMM over K-means, the classify-analyze problem), see `exploratory-unsupervised.md` in the `data-scientist` skill; its References section gives full citations for works cited here.

## GaussianMixture

Models data as a mixture of k Gaussian distributions. Each observation gets a posterior probability of belonging to each component (soft assignment).

```python
from sklearn.mixture import GaussianMixture

# --- Fit GMM ---
gmm = GaussianMixture(
    n_components=4,          # Number of mixture components
    covariance_type="full",  # Shape constraint (see table below)
    n_init=5,                # Number of initializations
    max_iter=200,            # Maximum EM iterations
    random_state=42
)
gmm.fit(X_scaled)

# --- Hard assignments (most likely component) ---
labels = gmm.predict(X_scaled)

# --- Soft assignments (posterior probabilities) ---
probs = gmm.predict_proba(X_scaled)  # Shape: (n_samples, n_components)

# --- Model attributes ---
means = gmm.means_               # Component means: (n_components, n_features)
covariances = gmm.covariances_   # Component covariances
weights = gmm.weights_           # Mixing proportions (sum to 1)
```

## Covariance Types

| Type | Shape Constraint | Parameters per Component | When to Use |
|------|-----------------|--------------------------|-------------|
| `full` | Unconstrained (any elliptical shape) | d(d+1)/2 | Default; most flexible |
| `tied` | All components share same covariance | d(d+1)/2 total | Clusters differ in location only |
| `diag` | Axis-aligned ellipses only | d | Moderate flexibility; fewer params |
| `spherical` | Round clusters; each component has its own single variance | 1 | With equal variances across components, closely related to K-means |

Where d = number of features. Start with `full` and let BIC guide simplification.

## Model Selection with BIC/AIC

Fit models across a range of k values AND covariance types. Lower BIC (or AIC) is better.

```python
from sklearn.mixture import GaussianMixture

# --- Compare models ---
results = []
for n in range(2, 9):
    for cov_type in ["full", "tied", "diag", "spherical"]:
        gmm = GaussianMixture(
            n_components=n,
            covariance_type=cov_type,
            n_init=5,
            random_state=42
        )
        gmm.fit(X_scaled)
        results.append({
            "n_components": n,
            "covariance_type": cov_type,
            "bic": gmm.bic(X_scaled),
            "aic": gmm.aic(X_scaled)
        })

# --- Find best model ---
best = min(results, key=lambda r: r["bic"])
print(f"Best: k={best['n_components']}, cov={best['covariance_type']}, BIC={best['bic']:.1f}")
```

### BIC vs AIC

| Criterion | Penalty | Tendency | Recommendation |
|-----------|---------|----------|----------------|
| BIC | log(n) x params | More conservative; favors simpler models | Primary criterion |
| AIC | 2 x params | Less conservative; favors more complex models | Secondary / comparison |

## BayesianGaussianMixture

Bayesian variant that can automatically determine the number of components. Components with near-zero weight are effectively pruned.

```python
from sklearn.mixture import BayesianGaussianMixture

# --- Fit Bayesian GMM ---
bgmm = BayesianGaussianMixture(
    n_components=10,         # Upper bound on components (set high)
    covariance_type="full",
    weight_concentration_prior_type="dirichlet_process",
    weight_concentration_prior=0.1,  # Lower = fewer components favored
    n_init=3,
    random_state=42
)
bgmm.fit(X_scaled)

# --- Effective number of components ---
weights = bgmm.weights_
effective_k = (weights > 0.01).sum()  # Components with meaningful weight
print(f"Effective components: {effective_k}")
print(f"Weights: {weights.round(3)}")

# --- Predict (same interface as GaussianMixture) ---
labels = bgmm.predict(X_scaled)
probs = bgmm.predict_proba(X_scaled)
```

### Weight Concentration Prior

| Value | Effect |
|-------|--------|
| Large (>1) | Encourages many equal-weight components |
| Small (<1) | Encourages few dominant components (sparse) |
| 0.1 - 1.0 | Typical range for exploration |

## Convergence and Diagnostics

```python
# --- Check convergence ---
print(f"Converged: {gmm.converged_}")
print(f"Iterations: {gmm.n_iter_}")

# --- Log-likelihood (higher = better fit) ---
ll = gmm.score(X_scaled)  # Per-sample average log-likelihood
print(f"Avg log-likelihood: {ll:.3f}")

# --- Identify ambiguous assignments ---
probs = gmm.predict_proba(X_scaled)
max_prob = probs.max(axis=1)
n_ambiguous = (max_prob < 0.70).sum()
print(f"Ambiguous assignments (max prob < 0.70): {n_ambiguous}")
```

## GMM vs KMeans Quick Reference

| Feature | GaussianMixture | KMeans |
|---------|----------------|--------|
| Assignment | Probabilistic (posterior) | Hard (nearest centroid) |
| Cluster shape | Elliptical (any covariance) | Spherical only |
| Model selection | BIC/AIC (formal) | Silhouette/elbow (heuristic) |
| Outlier detection | Low fitted density (`gmm.score_samples`) flags potential outliers; low max-probability flags ambiguous assignment, not outlyingness | Forced into nearest cluster |
| Downstream use | Posterior probabilities quantify assignment uncertainty; used directly as weights they do not remove classify-analyze bias (Vermunt, 2010) | Hard labels; classification error is not quantified |
| Computation | Slower (EM + covariance) | Faster |
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall