Skip to content

alirezarezvani/claude-skills / engineering/deep-learning-book/skills/deep-learning-book

deep-learning-book

Verified

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

FileKindSize
SKILL.mdskill md10.8 KB
assets/chapter_worksheet.mdmarkdown2.3 KB
assets/example_layer_spec.jsonconfig1.2 KB
assets/study_log_template.mdmarkdown2.0 KB
chapters/ch01-introduction.mdmarkdown3.1 KB
chapters/ch02-linear-algebra.mdmarkdown3.1 KB
chapters/ch03-probability-information-theory.mdmarkdown3.6 KB
chapters/ch04-numerical-computation.mdmarkdown3.0 KB
chapters/ch05-machine-learning-basics.mdmarkdown3.5 KB
chapters/ch06-deep-feedforward-networks.mdmarkdown3.2 KB
chapters/ch07-regularization.mdmarkdown3.6 KB
chapters/ch08-optimization.mdmarkdown3.5 KB
chapters/ch09-convolutional-networks.mdmarkdown3.4 KB
chapters/ch10-sequence-modeling.mdmarkdown3.5 KB
chapters/ch11-practical-methodology.mdmarkdown3.2 KB
chapters/ch12-applications.mdmarkdown3.0 KB
chapters/ch13-linear-factor-models.mdmarkdown3.1 KB
chapters/ch14-autoencoders.mdmarkdown3.3 KB
chapters/ch15-representation-learning.mdmarkdown3.5 KB
chapters/ch16-structured-probabilistic-models.mdmarkdown3.2 KB
chapters/ch17-monte-carlo-methods.mdmarkdown3.0 KB
chapters/ch18-partition-function.mdmarkdown3.5 KB
chapters/ch19-approximate-inference.mdmarkdown3.3 KB
chapters/ch20-deep-generative-models.mdmarkdown3.7 KB
cheatsheet.mdmarkdown3.2 KB
glossary.mdmarkdown5.9 KB
patterns.mdmarkdown4.7 KB
references/book_to_2026_delta.mdmarkdown6.4 KB
references/prerequisite_map.mdmarkdown4.9 KB
references/rights_and_use.mdmarkdown4.4 KB
references/study_method_canon.mdmarkdown5.3 KB
scripts/capacity_planner.pyscript14.4 KB
scripts/model_arithmetic.pyscript16.9 KB
scripts/reading_path_planner.pyscript19.7 KB
scripts/training_diagnostics.pyscript14.1 KB

chapters/ch09-convolutional-networks.md

View on GitHub
# Chapter 9: Convolutional Networks

**Source chapter (free, official):** https://www.deeplearningbook.org/contents/convnets.html

## Core Idea

Convolution is three ideas at once — sparse interactions, parameter sharing, and equivariance to
translation — and each is a prior about grid-structured data. Pooling adds approximate invariance
to small translations. The architecture is a statement about the data, not a trick.

## Frameworks Introduced

- **Sparse interactions**: each output depends on a small receptive field, so cost drops from
  O(m·n) to O(k·n).
- **Parameter sharing**: one kernel is reused at every position — the strongest regularizer in
  Ch 7's catalogue, applied structurally.
- **Equivariance to translation**: shift the input, the feature map shifts. Convolution is *not*
  equivariant to rotation or scale — a fact that motivates augmentation.
- **Pooling**: max/average pooling gives local invariance and downsampling; it is a prior that
  small position changes should not matter.
- **Variants**: valid/same/full padding, strided convolution, dilated convolution, tiled
  convolution, locally-connected layers (no sharing), transposed convolution for upsampling.
- **Convolution as an infinitely strong prior**: the chapter's sharpest framing — a conv layer is
  a fully-connected layer with hard constraints on its weights.
- **Structured outputs**: dense prediction (segmentation) rather than one label per image.
- **Efficiency**: FFT-based and separable convolutions.

## Mental Models

- Ask "**is the statistic I need position-invariant?**" If yes, convolve. If not (a face-aligned
  dataset, a tabular grid with meaningful coordinates), the prior is wrong and locally-connected
  or attention layers may fit better.
- Compute **receptive field** deliberately: it must cover the evidence needed for the decision.
  Depth, stride and dilation are three ways to buy it, with different costs.
- Treat pooling as **throwing away location on purpose**; when location is the answer
  (segmentation, detection), pool less and use dilation or skip connections.

## Anti-patterns

- **Using convolution on data with no spatial/temporal locality** (arbitrary tabular columns) —
  the sharing prior is simply false there.
- **Ignoring receptive field** and then adding parameters to fix an underfitting model.
- **Aggressive pooling in a dense-prediction task.**

## What changed after 2016

Residual networks (2015) are treated only briefly here but became the default. Since then:
depthwise-separable convolutions (MobileNet/Xception), EfficientNet-style compound scaling,
and — most importantly — **Vision Transformers** (Dosovitskiy et al. 2020), which discard the
convolutional prior in favour of data plus attention, and win at scale while ConvNets remain
competitive at smaller data sizes (ConvNeXt, 2022, closed much of the gap). The chapter's
argument that the conv prior is a *bet on the data* is exactly what ViT's data-hunger confirms.
**Confidence: high.**

## Key Takeaways

1. Justify convolution by the invariance you actually believe in.
2. Size the receptive field before adding depth for its own sake.
3. Expect the conv prior to pay off most when data is limited — this is the modern boundary
   between ConvNets and ViTs.

## Connects To

- **Ch 7**: parameter sharing as structural regularization.
- **Ch 12**: computer-vision applications built on this.
- **references/book_to_2026_delta.md**: the ViT/ConvNeXt line.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall