Skip to content

alirezarezvani/claude-skills / engineering/deep-learning-book/skills/deep-learning-book

deep-learning-book

Verified

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

FileKindSize
SKILL.mdskill md10.8 KB
assets/chapter_worksheet.mdmarkdown2.3 KB
assets/example_layer_spec.jsonconfig1.2 KB
assets/study_log_template.mdmarkdown2.0 KB
chapters/ch01-introduction.mdmarkdown3.1 KB
chapters/ch02-linear-algebra.mdmarkdown3.1 KB
chapters/ch03-probability-information-theory.mdmarkdown3.6 KB
chapters/ch04-numerical-computation.mdmarkdown3.0 KB
chapters/ch05-machine-learning-basics.mdmarkdown3.5 KB
chapters/ch06-deep-feedforward-networks.mdmarkdown3.2 KB
chapters/ch07-regularization.mdmarkdown3.6 KB
chapters/ch08-optimization.mdmarkdown3.5 KB
chapters/ch09-convolutional-networks.mdmarkdown3.4 KB
chapters/ch10-sequence-modeling.mdmarkdown3.5 KB
chapters/ch11-practical-methodology.mdmarkdown3.2 KB
chapters/ch12-applications.mdmarkdown3.0 KB
chapters/ch13-linear-factor-models.mdmarkdown3.1 KB
chapters/ch14-autoencoders.mdmarkdown3.3 KB
chapters/ch15-representation-learning.mdmarkdown3.5 KB
chapters/ch16-structured-probabilistic-models.mdmarkdown3.2 KB
chapters/ch17-monte-carlo-methods.mdmarkdown3.0 KB
chapters/ch18-partition-function.mdmarkdown3.5 KB
chapters/ch19-approximate-inference.mdmarkdown3.3 KB
chapters/ch20-deep-generative-models.mdmarkdown3.7 KB
cheatsheet.mdmarkdown3.2 KB
glossary.mdmarkdown5.9 KB
patterns.mdmarkdown4.7 KB
references/book_to_2026_delta.mdmarkdown6.4 KB
references/prerequisite_map.mdmarkdown4.9 KB
references/rights_and_use.mdmarkdown4.4 KB
references/study_method_canon.mdmarkdown5.3 KB
scripts/capacity_planner.pyscript14.4 KB
scripts/model_arithmetic.pyscript16.9 KB
scripts/reading_path_planner.pyscript19.7 KB
scripts/training_diagnostics.pyscript14.1 KB

chapters/ch02-linear-algebra.md

View on GitHub
# Chapter 2: Linear Algebra

**Source chapter (free, official):** https://www.deeplearningbook.org/contents/linear_algebra.html

## Core Idea

The minimum linear algebra a deep learning practitioner needs, chosen for what appears later:
tensors and broadcasting, norms, eigendecomposition, SVD, the pseudoinverse, and PCA derived
from scratch. This is a filter, not a course — it teaches what Chapters 4, 5, 8 and 13 will use.

## Key Concepts

- **Tensor / broadcasting**: an n-dimensional array plus the rules for combining shapes. Most
  real "model bugs" are shape bugs.
- **Norms**: L2 (Euclidean), L1 (sparsity-friendly, non-differentiable at 0), max-norm,
  Frobenius. The choice of norm *is* the choice of what "small" means in a regularizer.
- **Eigendecomposition**: A = V diag(λ) V⁻¹ for square matrices; the eigenvalues tell you the
  local curvature story later in Ch 4 and 8.
- **Singular value decomposition (SVD)**: A = U D Vᵀ, defined for *any* matrix. The general tool
  where eigendecomposition needs square and diagonalizable.
- **Moore–Penrose pseudoinverse**: the least-squares / minimum-norm solution when a system is
  over- or under-determined.
- **Condition number**: ratio of largest to smallest singular value; large means small input
  perturbations produce large output changes — the numerical fragility Ch 4 confronts.
- **PCA**: derived here as the linear encoder/decoder minimizing L2 reconstruction error, which
  is exactly the framing Ch 13–14 generalize.

## Mental Models

- Read a matrix as a **function on space**, and its singular values as how much it stretches
  each orthogonal direction. Condition number = worst stretch / least stretch.
- Use SVD as the **default** decomposition and reach for eigendecomposition only when symmetry
  buys you something (it does for Hessians).
- Treat a regularizer's norm as a **prior over parameter space**: L2 says "small and spread",
  L1 says "mostly zero."

## Anti-patterns

- **Skipping to Chapter 6.** Ch 4 and 8's discussion of ill-conditioning is unreadable without
  eigenvalues and condition number, and readers who skip typically bounce off Ch 8.
- **Memorizing decompositions as identities** instead of as geometry — the geometry is what
  transfers to optimization.

## What changed after 2016

Nothing in the mathematics. What changed is the practice: in half-precision training, condition
number stopped being a theoretical concern and became an operational one (loss scaling, bf16 over
fp16 precisely because of dynamic range). **Confidence: high.**

## Key Takeaways

1. Track shapes explicitly; treat a shape mismatch as a modelling error, not a typing error.
2. Pick your norm deliberately when you regularize — you are choosing the shape of the prior.
3. Learn PCA in this chapter's form (encoder/decoder minimizing reconstruction) so Ch 13–14 read
   as generalizations rather than new material.

## Connects To

- **Ch 4**: conditioning and numerical stability use exactly these quantities.
- **Ch 13**: PCA reappears as a linear factor model with an explicit probabilistic story.
- **Ch 8**: Hessian eigenvalues explain why gradient descent stalls.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall