Skip to content

alirezarezvani/claude-skills / engineering/deep-learning-book/skills/deep-learning-book

deep-learning-book

Verified

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

FileKindSize
SKILL.mdskill md10.8 KB
assets/chapter_worksheet.mdmarkdown2.3 KB
assets/example_layer_spec.jsonconfig1.2 KB
assets/study_log_template.mdmarkdown2.0 KB
chapters/ch01-introduction.mdmarkdown3.1 KB
chapters/ch02-linear-algebra.mdmarkdown3.1 KB
chapters/ch03-probability-information-theory.mdmarkdown3.6 KB
chapters/ch04-numerical-computation.mdmarkdown3.0 KB
chapters/ch05-machine-learning-basics.mdmarkdown3.5 KB
chapters/ch06-deep-feedforward-networks.mdmarkdown3.2 KB
chapters/ch07-regularization.mdmarkdown3.6 KB
chapters/ch08-optimization.mdmarkdown3.5 KB
chapters/ch09-convolutional-networks.mdmarkdown3.4 KB
chapters/ch10-sequence-modeling.mdmarkdown3.5 KB
chapters/ch11-practical-methodology.mdmarkdown3.2 KB
chapters/ch12-applications.mdmarkdown3.0 KB
chapters/ch13-linear-factor-models.mdmarkdown3.1 KB
chapters/ch14-autoencoders.mdmarkdown3.3 KB
chapters/ch15-representation-learning.mdmarkdown3.5 KB
chapters/ch16-structured-probabilistic-models.mdmarkdown3.2 KB
chapters/ch17-monte-carlo-methods.mdmarkdown3.0 KB
chapters/ch18-partition-function.mdmarkdown3.5 KB
chapters/ch19-approximate-inference.mdmarkdown3.3 KB
chapters/ch20-deep-generative-models.mdmarkdown3.7 KB
cheatsheet.mdmarkdown3.2 KB
glossary.mdmarkdown5.9 KB
patterns.mdmarkdown4.7 KB
references/book_to_2026_delta.mdmarkdown6.4 KB
references/prerequisite_map.mdmarkdown4.9 KB
references/rights_and_use.mdmarkdown4.4 KB
references/study_method_canon.mdmarkdown5.3 KB
scripts/capacity_planner.pyscript14.4 KB
scripts/model_arithmetic.pyscript16.9 KB
scripts/reading_path_planner.pyscript19.7 KB
scripts/training_diagnostics.pyscript14.1 KB

chapters/ch12-applications.md

View on GitHub
# Chapter 12: Applications

**Source chapter (free, official):** https://www.deeplearningbook.org/contents/applications.html

## Core Idea

How deep learning was actually deployed circa 2016: large-scale implementation (GPUs,
distributed training, model compression), then computer vision, speech recognition, and NLP.
This is the most time-dated chapter in the book — read the *systems* half, discount the
*state-of-the-art* half.

## Frameworks Introduced

- **Large-scale implementation**: GPU data parallelism, model parallelism, asynchronous SGD
  (Hogwild-style), parameter servers.
- **Model compression**: distillation, quantization, pruning — for inference cost.
- **Dynamic structure / conditional computation**: cascades and gating so that not every input
  pays the full cost.
- **Preprocessing in vision**: contrast normalization, whitening, dataset augmentation.
- **Speech**: the acoustic-model pipeline and its shift from GMM-HMM to deep networks.
- **NLP**: n-gram models, word embeddings and the curse of dimensionality over vocabularies,
  hierarchical softmax and sampling-based approximations for large output vocabularies,
  neural machine translation.
- **Recommender systems** and the exploration/exploitation problem.
- **Knowledge representation and relational reasoning.**

## Mental Models

- Separate **capability claims** (perishable) from **systems constraints** (durable). Memory
  bandwidth, batch efficiency, and inference cost still shape architecture choices.
- Read the large-vocabulary softmax section as an instance of the general pattern: **when the
  normalizing sum is expensive, approximate it** — the same problem Ch 18 attacks head-on.
- Treat conditional computation as the ancestor of **Mixture-of-Experts** routing.

## Anti-patterns

- **Citing this chapter's benchmark numbers or SOTA claims** — they are a decade old.
- **Copying its NLP pipeline**: subword tokenization (BPE/SentencePiece) and pretrained
  transformers replaced nearly all of it.

## What changed after 2016

Almost everything at the application layer. Speech moved to end-to-end CTC/attention models and
then to large self-supervised encoders. NLP moved to pretrained transformers (BERT 2018, GPT
family), with subword tokenization and full-softmax over ~30k–200k subwords making hierarchical
softmax largely unnecessary. Vision moved to self-supervised pretraining and ViTs. Distillation,
quantization and pruning grew into a mature inference-optimization discipline. Conditional
computation matured into sparse MoE. **Confidence: high.**

## Key Takeaways

1. Read this chapter for the systems reasoning and the approximation techniques, not for what is
   state of the art.
2. When your output vocabulary or normalizing constant is huge, recognize it as the recurring
   partition-function problem.
3. Plan inference cost as an architecture constraint from the start.

## Connects To

- **Ch 18**: the partition-function problem in its general form.
- **Ch 9 / Ch 10**: the architectures being applied here.
- **references/book_to_2026_delta.md**: what replaced each application pipeline.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall