Skip to content

alirezarezvani/claude-skills / engineering/deep-learning-book/skills/deep-learning-book

deep-learning-book

Verified

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

FileKindSize
SKILL.mdskill md10.8 KB
assets/chapter_worksheet.mdmarkdown2.3 KB
assets/example_layer_spec.jsonconfig1.2 KB
assets/study_log_template.mdmarkdown2.0 KB
chapters/ch01-introduction.mdmarkdown3.1 KB
chapters/ch02-linear-algebra.mdmarkdown3.1 KB
chapters/ch03-probability-information-theory.mdmarkdown3.6 KB
chapters/ch04-numerical-computation.mdmarkdown3.0 KB
chapters/ch05-machine-learning-basics.mdmarkdown3.5 KB
chapters/ch06-deep-feedforward-networks.mdmarkdown3.2 KB
chapters/ch07-regularization.mdmarkdown3.6 KB
chapters/ch08-optimization.mdmarkdown3.5 KB
chapters/ch09-convolutional-networks.mdmarkdown3.4 KB
chapters/ch10-sequence-modeling.mdmarkdown3.5 KB
chapters/ch11-practical-methodology.mdmarkdown3.2 KB
chapters/ch12-applications.mdmarkdown3.0 KB
chapters/ch13-linear-factor-models.mdmarkdown3.1 KB
chapters/ch14-autoencoders.mdmarkdown3.3 KB
chapters/ch15-representation-learning.mdmarkdown3.5 KB
chapters/ch16-structured-probabilistic-models.mdmarkdown3.2 KB
chapters/ch17-monte-carlo-methods.mdmarkdown3.0 KB
chapters/ch18-partition-function.mdmarkdown3.5 KB
chapters/ch19-approximate-inference.mdmarkdown3.3 KB
chapters/ch20-deep-generative-models.mdmarkdown3.7 KB
cheatsheet.mdmarkdown3.2 KB
glossary.mdmarkdown5.9 KB
patterns.mdmarkdown4.7 KB
references/book_to_2026_delta.mdmarkdown6.4 KB
references/prerequisite_map.mdmarkdown4.9 KB
references/rights_and_use.mdmarkdown4.4 KB
references/study_method_canon.mdmarkdown5.3 KB
scripts/capacity_planner.pyscript14.4 KB
scripts/model_arithmetic.pyscript16.9 KB
scripts/reading_path_planner.pyscript19.7 KB
scripts/training_diagnostics.pyscript14.1 KB

chapters/ch01-introduction.md

View on GitHub
# Chapter 1: Introduction

**Source chapter (free, official):** https://www.deeplearningbook.org/contents/intro.html

## Core Idea

Deep learning is representation learning stacked: instead of hand-designing the features a
model consumes, you let the model learn a hierarchy of representations, each layer expressed
in terms of the one below it. Depth is the mechanism that makes composition cheap.

## Key Concepts

- **Representation learning**: learning the features, not just the mapping from features to output.
- **Depth as composition**: a deep model reuses sub-computations, so a function needing
  exponentially many units at depth 1 may need polynomially many at depth k.
- **The curse of dimensionality**: as input dimension grows, the number of distinguishable
  configurations grows exponentially, so local-interpolation methods run out of neighbours.
- **Prior of compositionality**: deep learning bets that the data was generated by a composition
  of simple factors. This is a prior, not a theorem — where the bet is wrong, depth does not help.
- **AI winters and rebrands**: cybernetics → connectionism → deep learning; the ideas outlived
  three funding cycles because the constraint was data and compute, not concept.

## Mental Models

- Think of each layer as **re-coordinatizing** the input so the next layer's job gets easier —
  not as "extracting meaning."
- Prefer depth over width when the target function plausibly **factors**; prefer width when it
  does not. Depth is a structural prior, and priors have to be earned.
- Treat "the model learns features automatically" as **shifting where the human judgment goes**
  (architecture, objective, data curation), not eliminating it.

## Anti-patterns

- **Reading this chapter as history and skipping it**: the three growth curves it draws
  (dataset size, model size, benchmark accuracy) are the argument for why anything else works.
- **Assuming depth is always better**: without the compositional structure, extra depth buys
  optimization difficulty and nothing else.

## What changed after 2016

The chapter's central claim — capability tracks dataset and model scale — held, and was made
quantitative by neural scaling laws (Hestness et al. 2017; Kaplan et al. 2020; Hoffmann et al.
2022 "Chinchilla"), which the 2016 text predates. Its examples of "what deep learning can do"
are now badly dated; its explanation of *why* depth helps is not. **Confidence: high** — the
scaling-law literature is public and replicated.

## Key Takeaways

1. Before adding depth, state the compositional structure you believe the data has.
2. Use the curse of dimensionality as the test for whether a nearest-neighbour-style baseline
   is even viable on your input.
3. Read the growth curves as a resource argument: most 2016-era "deep learning doesn't work
   here" results were data or compute verdicts, not architecture verdicts.

## Connects To

- **Ch 5**: the formal learning-theory machinery behind the claims made informally here.
- **Ch 15**: representation learning, promised here and delivered there.
- **references/book_to_2026_delta.md**: the scaling-law layer this chapter now needs.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall