alirezarezvani/claude-skills / engineering/deep-learning-book/skills/deep-learning-book
deep-learning-book
VerifiedUse when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.
chapters/ch01-introduction.md
View on GitHub# Chapter 1: Introduction
**Source chapter (free, official):** https://www.deeplearningbook.org/contents/intro.html
## Core Idea
Deep learning is representation learning stacked: instead of hand-designing the features a
model consumes, you let the model learn a hierarchy of representations, each layer expressed
in terms of the one below it. Depth is the mechanism that makes composition cheap.
## Key Concepts
- **Representation learning**: learning the features, not just the mapping from features to output.
- **Depth as composition**: a deep model reuses sub-computations, so a function needing
exponentially many units at depth 1 may need polynomially many at depth k.
- **The curse of dimensionality**: as input dimension grows, the number of distinguishable
configurations grows exponentially, so local-interpolation methods run out of neighbours.
- **Prior of compositionality**: deep learning bets that the data was generated by a composition
of simple factors. This is a prior, not a theorem — where the bet is wrong, depth does not help.
- **AI winters and rebrands**: cybernetics → connectionism → deep learning; the ideas outlived
three funding cycles because the constraint was data and compute, not concept.
## Mental Models
- Think of each layer as **re-coordinatizing** the input so the next layer's job gets easier —
not as "extracting meaning."
- Prefer depth over width when the target function plausibly **factors**; prefer width when it
does not. Depth is a structural prior, and priors have to be earned.
- Treat "the model learns features automatically" as **shifting where the human judgment goes**
(architecture, objective, data curation), not eliminating it.
## Anti-patterns
- **Reading this chapter as history and skipping it**: the three growth curves it draws
(dataset size, model size, benchmark accuracy) are the argument for why anything else works.
- **Assuming depth is always better**: without the compositional structure, extra depth buys
optimization difficulty and nothing else.
## What changed after 2016
The chapter's central claim — capability tracks dataset and model scale — held, and was made
quantitative by neural scaling laws (Hestness et al. 2017; Kaplan et al. 2020; Hoffmann et al.
2022 "Chinchilla"), which the 2016 text predates. Its examples of "what deep learning can do"
are now badly dated; its explanation of *why* depth helps is not. **Confidence: high** — the
scaling-law literature is public and replicated.
## Key Takeaways
1. Before adding depth, state the compositional structure you believe the data has.
2. Use the curse of dimensionality as the test for whether a nearest-neighbour-style baseline
is even viable on your input.
3. Read the growth curves as a resource argument: most 2016-era "deep learning doesn't work
here" results were data or compute verdicts, not architecture verdicts.
## Connects To
- **Ch 5**: the formal learning-theory machinery behind the claims made informally here.
- **Ch 15**: representation learning, promised here and delivered there.
- **references/book_to_2026_delta.md**: the scaling-law layer this chapter now needs.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |