Skip to content

alirezarezvani/claude-skills / engineering/deep-learning-book/skills/deep-learning-book

deep-learning-book

Verified

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

FileKindSize
SKILL.mdskill md10.8 KB
assets/chapter_worksheet.mdmarkdown2.3 KB
assets/example_layer_spec.jsonconfig1.2 KB
assets/study_log_template.mdmarkdown2.0 KB
chapters/ch01-introduction.mdmarkdown3.1 KB
chapters/ch02-linear-algebra.mdmarkdown3.1 KB
chapters/ch03-probability-information-theory.mdmarkdown3.6 KB
chapters/ch04-numerical-computation.mdmarkdown3.0 KB
chapters/ch05-machine-learning-basics.mdmarkdown3.5 KB
chapters/ch06-deep-feedforward-networks.mdmarkdown3.2 KB
chapters/ch07-regularization.mdmarkdown3.6 KB
chapters/ch08-optimization.mdmarkdown3.5 KB
chapters/ch09-convolutional-networks.mdmarkdown3.4 KB
chapters/ch10-sequence-modeling.mdmarkdown3.5 KB
chapters/ch11-practical-methodology.mdmarkdown3.2 KB
chapters/ch12-applications.mdmarkdown3.0 KB
chapters/ch13-linear-factor-models.mdmarkdown3.1 KB
chapters/ch14-autoencoders.mdmarkdown3.3 KB
chapters/ch15-representation-learning.mdmarkdown3.5 KB
chapters/ch16-structured-probabilistic-models.mdmarkdown3.2 KB
chapters/ch17-monte-carlo-methods.mdmarkdown3.0 KB
chapters/ch18-partition-function.mdmarkdown3.5 KB
chapters/ch19-approximate-inference.mdmarkdown3.3 KB
chapters/ch20-deep-generative-models.mdmarkdown3.7 KB
cheatsheet.mdmarkdown3.2 KB
glossary.mdmarkdown5.9 KB
patterns.mdmarkdown4.7 KB
references/book_to_2026_delta.mdmarkdown6.4 KB
references/prerequisite_map.mdmarkdown4.9 KB
references/rights_and_use.mdmarkdown4.4 KB
references/study_method_canon.mdmarkdown5.3 KB
scripts/capacity_planner.pyscript14.4 KB
scripts/model_arithmetic.pyscript16.9 KB
scripts/reading_path_planner.pyscript19.7 KB
scripts/training_diagnostics.pyscript14.1 KB

chapters/ch13-linear-factor-models.md

View on GitHub
# Chapter 13: Linear Factor Models

**Source chapter (free, official):** https://www.deeplearningbook.org/contents/linear_factors.html

## Core Idea

The simplest generative models: sample latent factors from a simple prior, map them linearly to
observations, add noise. Probabilistic PCA, factor analysis, ICA, sparse coding and slow feature
analysis are all this one template with different priors — and they are the scaffolding for
everything in Chapters 14–20.

## Key Concepts

- **The template**: h ~ p(h); x = Wh + b + noise. Change p(h) and the noise model, get a
  different named method.
- **Factor analysis**: Gaussian latent prior, diagonal (per-feature) observation noise.
- **Probabilistic PCA**: factor analysis with isotropic noise; recovers PCA as noise → 0.
- **Independent component analysis (ICA)**: non-Gaussian independent latents — this is what makes
  the factors identifiable, which Gaussian models cannot be (any rotation fits equally well).
- **Sparse coding**: a heavy-tailed (Laplace/Cauchy) prior; inference is an optimization, not a
  closed form, which makes encoding expensive.
- **Slow feature analysis**: a prior that useful factors change slowly over time.
- **Manifold interpretation of PCA**: the model concentrates probability near a linear subspace.

## Mental Models

- Read each model as **a prior choice**, and read the prior as the assumption that buys
  identifiability. Gaussian latents are rotation-invariant, so a Gaussian model cannot tell you
  *which* factors — only which subspace.
- Treat sparse coding as the moment where **inference becomes iterative**; that cost is precisely
  what autoencoders (Ch 14) amortize with a learned encoder.
- Use these as the **linear baseline** for any representation-learning claim: if a linear factor
  model matches your deep encoder, the depth is not earning its cost.

## Anti-patterns

- **Skipping this chapter and starting at Ch 20** — VAEs read as arbitrary machinery without the
  latent-variable template established here.
- **Expecting interpretable factors from a Gaussian-latent model.**

## What changed after 2016

The template survived; the emphasis moved. Nonlinear ICA identifiability results (Hyvärinen et
al., 2016–2020) clarified when latent factors are recoverable at all, and the disentanglement
literature — notably Locatello et al. (2019) — showed that unsupervised disentanglement is
impossible without inductive biases or supervision, which is a formal statement of this
chapter's identifiability point. Sparse coding returned as a tool for interpreting neural
networks (sparse autoencoders over LLM activations, 2023–2024). **Confidence: high.**

## Key Takeaways

1. Name the latent prior when you propose any generative model; it determines what is learnable.
2. Use a linear factor model as the baseline before claiming a deep representation helps.
3. Remember that identifiability, not fit quality, is what non-Gaussian priors buy.

## Connects To

- **Ch 14**: autoencoders amortize the inference these models do by optimization.
- **Ch 15**: what makes a representation good.
- **Ch 20**: the VAE is this template with a nonlinear decoder and amortized inference.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall