Skip to content

alirezarezvani/claude-skills / engineering/deep-learning-book/skills/deep-learning-book

deep-learning-book

Verified

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

FileKindSize
SKILL.mdskill md10.8 KB
assets/chapter_worksheet.mdmarkdown2.3 KB
assets/example_layer_spec.jsonconfig1.2 KB
assets/study_log_template.mdmarkdown2.0 KB
chapters/ch01-introduction.mdmarkdown3.1 KB
chapters/ch02-linear-algebra.mdmarkdown3.1 KB
chapters/ch03-probability-information-theory.mdmarkdown3.6 KB
chapters/ch04-numerical-computation.mdmarkdown3.0 KB
chapters/ch05-machine-learning-basics.mdmarkdown3.5 KB
chapters/ch06-deep-feedforward-networks.mdmarkdown3.2 KB
chapters/ch07-regularization.mdmarkdown3.6 KB
chapters/ch08-optimization.mdmarkdown3.5 KB
chapters/ch09-convolutional-networks.mdmarkdown3.4 KB
chapters/ch10-sequence-modeling.mdmarkdown3.5 KB
chapters/ch11-practical-methodology.mdmarkdown3.2 KB
chapters/ch12-applications.mdmarkdown3.0 KB
chapters/ch13-linear-factor-models.mdmarkdown3.1 KB
chapters/ch14-autoencoders.mdmarkdown3.3 KB
chapters/ch15-representation-learning.mdmarkdown3.5 KB
chapters/ch16-structured-probabilistic-models.mdmarkdown3.2 KB
chapters/ch17-monte-carlo-methods.mdmarkdown3.0 KB
chapters/ch18-partition-function.mdmarkdown3.5 KB
chapters/ch19-approximate-inference.mdmarkdown3.3 KB
chapters/ch20-deep-generative-models.mdmarkdown3.7 KB
cheatsheet.mdmarkdown3.2 KB
glossary.mdmarkdown5.9 KB
patterns.mdmarkdown4.7 KB
references/book_to_2026_delta.mdmarkdown6.4 KB
references/prerequisite_map.mdmarkdown4.9 KB
references/rights_and_use.mdmarkdown4.4 KB
references/study_method_canon.mdmarkdown5.3 KB
scripts/capacity_planner.pyscript14.4 KB
scripts/model_arithmetic.pyscript16.9 KB
scripts/reading_path_planner.pyscript19.7 KB
scripts/training_diagnostics.pyscript14.1 KB

chapters/ch11-practical-methodology.md

View on GitHub
# Chapter 11: Practical Methodology

**Source chapter (free, official):** https://www.deeplearningbook.org/contents/guidelines.html

## Core Idea

Knowing many algorithms matters less than knowing which one to reach for given a measurement.
This chapter is a workflow: pick a metric, build an end-to-end baseline fast, then let
instrumentation — not intuition — decide whether to add data, capacity, or regularization.

## Frameworks Introduced

- **The design process**: (1) determine goals — the error metric and the target value; (2) build
  a working end-to-end pipeline early; (3) instrument it to find bottlenecks; (4) change one
  thing at a time based on measurement.
- **Choose the metric before the model**: accuracy, precision/recall, F-score, PR/ROC curves,
  coverage. Name the target value and where it came from.
- **Sensible baselines**: pick the standard architecture and optimizer for the data type before
  inventing anything.
- **More data or a bigger model?** — the decision procedure: if training error is high, the model
  or the optimization is the bottleneck (data will not help). If training error is low and test
  error is high, gather more data or regularize.
- **Hyperparameter tuning**: manual (understand what each knob does to effective capacity),
  **grid search** (poor scaling in dimensions), **random search** (better — it does not waste
  trials on unimportant dimensions; Bergstra & Bengio 2012), and model-based/Bayesian.
- **Debugging strategies**: visualize the model's actual predictions and worst cases; fit a tiny
  subset to zero training error; compare backprop against numerical derivatives; monitor
  activation and gradient histograms.

## Mental Models

- Use the **train-error-first rule** as the single most valuable heuristic in the book: high
  training error means "do not collect data yet."
- Treat "**can it overfit 20 examples?**" as the smoke test that separates a bug from a modelling
  limitation. If it cannot, you have a bug.
- Prefer **random search over grid search** whenever the number of hyperparameters exceeds ~2.

## Anti-patterns

- **Tuning many things per experiment** — you learn nothing attributable.
- **Collecting more data to fix underfitting.**
- **Optimizing a proxy metric** whose relationship to the real objective was never checked.
- **Skipping the end-to-end pipeline** in favour of perfecting one component.

## What changed after 2016

The workflow is the most durable material in the book and is now the backbone of MLOps practice.
Additions since: experiment tracking as standard tooling, seeded reproducibility expectations,
Hyperband/ASHA for early-stopping-based search (post-2016), and — for large models — scaling laws
used to *predict* the return on more data or parameters instead of testing empirically at full
size. **Confidence: high.**

## Key Takeaways

1. Write the metric and its target number before writing model code.
2. Ask "is training error high?" before every data or capacity decision.
3. Change one thing per experiment and log it.

## Connects To

- **Ch 5**: the underfit/overfit framing this operationalizes.
- **Ch 7 / Ch 8**: the two toolboxes the workflow selects from.
- **scripts/training_diagnostics.py**: this decision tree, executable.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall