Skip to content

alirezarezvani/claude-skills / engineering/deep-learning-book/skills/deep-learning-book

deep-learning-book

Verified

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

FileKindSize
SKILL.mdskill md10.8 KB
assets/chapter_worksheet.mdmarkdown2.3 KB
assets/example_layer_spec.jsonconfig1.2 KB
assets/study_log_template.mdmarkdown2.0 KB
chapters/ch01-introduction.mdmarkdown3.1 KB
chapters/ch02-linear-algebra.mdmarkdown3.1 KB
chapters/ch03-probability-information-theory.mdmarkdown3.6 KB
chapters/ch04-numerical-computation.mdmarkdown3.0 KB
chapters/ch05-machine-learning-basics.mdmarkdown3.5 KB
chapters/ch06-deep-feedforward-networks.mdmarkdown3.2 KB
chapters/ch07-regularization.mdmarkdown3.6 KB
chapters/ch08-optimization.mdmarkdown3.5 KB
chapters/ch09-convolutional-networks.mdmarkdown3.4 KB
chapters/ch10-sequence-modeling.mdmarkdown3.5 KB
chapters/ch11-practical-methodology.mdmarkdown3.2 KB
chapters/ch12-applications.mdmarkdown3.0 KB
chapters/ch13-linear-factor-models.mdmarkdown3.1 KB
chapters/ch14-autoencoders.mdmarkdown3.3 KB
chapters/ch15-representation-learning.mdmarkdown3.5 KB
chapters/ch16-structured-probabilistic-models.mdmarkdown3.2 KB
chapters/ch17-monte-carlo-methods.mdmarkdown3.0 KB
chapters/ch18-partition-function.mdmarkdown3.5 KB
chapters/ch19-approximate-inference.mdmarkdown3.3 KB
chapters/ch20-deep-generative-models.mdmarkdown3.7 KB
cheatsheet.mdmarkdown3.2 KB
glossary.mdmarkdown5.9 KB
patterns.mdmarkdown4.7 KB
references/book_to_2026_delta.mdmarkdown6.4 KB
references/prerequisite_map.mdmarkdown4.9 KB
references/rights_and_use.mdmarkdown4.4 KB
references/study_method_canon.mdmarkdown5.3 KB
scripts/capacity_planner.pyscript14.4 KB
scripts/model_arithmetic.pyscript16.9 KB
scripts/reading_path_planner.pyscript19.7 KB
scripts/training_diagnostics.pyscript14.1 KB

glossary.md

View on GitHub
# Glossary

Terms as this skill uses them, with the chapter that develops each. Definitions are written for
lookup, not for teaching — read the chapter file for the reasoning.

- **Adam / AdamW** (ch08, ch07) — adaptive optimizer combining momentum with per-parameter scaling. AdamW decouples weight decay from the gradient, which Adam+L2 does not do correctly.
- **Amortized inference** (ch19, ch14) — training a network to predict a posterior's parameters instead of optimizing per example.
- **Ancestral sampling** (ch16) — sampling a directed model by drawing each variable after its parents.
- **Attention** (ch10) — a learned weighted read over a set of positions; introduced here as a fix for the seq2seq bottleneck, later the whole architecture.
- **Backpropagation** (ch06) — reverse-mode automatic differentiation over the computation graph.
- **Batch normalization** (ch08) — reparameterization normalizing layer inputs by batch statistics.
- **Bias–variance decomposition** (ch05) — expected error as bias² + variance + irreducible noise.
- **Capacity, effective vs representational** (ch05) — functions reachable by the optimizer vs functions expressible by the architecture.
- **Condition number** (ch02, ch04) — ratio of largest to smallest singular value; how amplified small perturbations become.
- **Contrastive divergence (CD-k)** (ch18) — biased approximation of the negative phase using a short chain started at the data.
- **Cross-entropy** (ch03) — H(p,q) = H(p) + D_KL(p‖q); minimizing it over q minimizes KL.
- **Curse of dimensionality** (ch05, ch01) — configurations grow exponentially with dimension, defeating local-interpolation methods.
- **Denoising autoencoder (DAE)** (ch14) — reconstructs clean input from a corrupted copy; its learned map estimates the score.
- **Disentangled representation** (ch15) — separate generative factors on separate directions.
- **Distributed representation** (ch15) — n features describing exponentially many regions, versus one-hot's n.
- **d-separation** (ch16) — reading conditional independence off a directed graph.
- **Double descent** (ch05) — test error falling again past the interpolation threshold; post-dates the book and qualifies its U-curve.
- **ELBO** (ch19) — evidence lower bound; log p(x) minus KL(q ‖ posterior).
- **Empirical risk minimization** (ch08) — minimizing training loss as a surrogate for unmeasurable true risk.
- **Energy-based model** (ch16) — p(x) ∝ exp(−E(x)).
- **Equivariance** (ch09) — a shift in input produces the corresponding shift in output; convolution's defining property.
- **Exposure bias** (ch10) — train/inference mismatch from teacher forcing.
- **Gradient clipping** (ch08, ch10) — rescaling gradients above a norm threshold to survive cliffs.
- **Ill-conditioning** (ch04, ch08) — wide Hessian eigenvalue spread; loss stalls while gradient norm stays large.
- **Importance sampling** (ch17) — sampling from q and reweighting by p/q.
- **KL divergence** (ch03) — asymmetric divergence between distributions; direction decides mode-covering vs mode-seeking.
- **Manifold hypothesis** (ch05, ch14) — real data concentrates near a low-dimensional manifold.
- **Maximum likelihood estimation (MLE)** (ch05) — parameters maximizing data probability; the justification for standard losses.
- **Mean field** (ch19) — factorized variational family; buys tractability with an independence assumption.
- **Mixing time / burn-in** (ch17) — steps an MCMC chain needs before its samples are usable.
- **Mode collapse** (ch20) — a generator covering only part of the data distribution.
- **Momentum / Nesterov momentum** (ch08) — velocity accumulation across gradient steps.
- **No Free Lunch theorem** (ch05) — averaged over all distributions, all learners tie; priors carry all leverage.
- **Noise-contrastive estimation (NCE)** (ch18) — density estimation recast as data-versus-noise classification.
- **Partition function (Z)** (ch16, ch18) — the normalizer of an undirected model; the central intractability of Part III.
- **Persistent contrastive divergence (PCD)** (ch18) — CD with a chain persisted across updates.
- **Pooling** (ch09) — local aggregation giving approximate invariance to small translations.
- **Positive / negative phase** (ch18) — the two terms of an undirected model's likelihood gradient.
- **Pseudolikelihood** (ch18) — product of conditionals, in which Z cancels.
- **Receptive field** (ch09) — the input region influencing one output unit.
- **Regularization** (ch07, ch05) — any change intended to reduce generalization error but not training error.
- **Reparameterization trick** (ch20) — expressing a sample as a differentiable function of noise, making the ELBO backprop-able.
- **Representation learning** (ch15, ch01) — learning features rather than hand-designing them.
- **Saddle point** (ch08) — critical point that is a minimum in some directions and a maximum in others; the dominant critical point in high dimensions.
- **Score matching** (ch18) — matching ∇_x log p instead of p, so Z vanishes; the basis of diffusion models.
- **SGD, minibatch** (ch08) — gradient descent on noisy gradients from data subsets.
- **Singular value decomposition (SVD)** (ch02) — A = U D Vᵀ for any matrix.
- **Sparse coding** (ch13) — heavy-tailed latent prior with inference by optimization.
- **Teacher forcing** (ch10) — training an autoregressive model on ground-truth history.
- **Universal approximation theorem** (ch06) — one wide hidden layer suffices in principle; silent on width, learnability and sample cost.
- **Vanishing / exploding gradients** (ch10, ch08) — repeated Jacobian multiplication shrinking or growing gradients through depth or time.
- **Variational autoencoder (VAE)** (ch20, ch19) — amortized variational inference plus the reparameterization trick.
- **Weight decay** (ch07) — pulling parameters toward zero; equivalent to L2 for SGD, not for Adam.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall