Skip to content

alirezarezvani/claude-skills / engineering/deep-learning-book/skills/deep-learning-book

deep-learning-book

Verified

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

FileKindSize
SKILL.mdskill md10.8 KB
assets/chapter_worksheet.mdmarkdown2.3 KB
assets/example_layer_spec.jsonconfig1.2 KB
assets/study_log_template.mdmarkdown2.0 KB
chapters/ch01-introduction.mdmarkdown3.1 KB
chapters/ch02-linear-algebra.mdmarkdown3.1 KB
chapters/ch03-probability-information-theory.mdmarkdown3.6 KB
chapters/ch04-numerical-computation.mdmarkdown3.0 KB
chapters/ch05-machine-learning-basics.mdmarkdown3.5 KB
chapters/ch06-deep-feedforward-networks.mdmarkdown3.2 KB
chapters/ch07-regularization.mdmarkdown3.6 KB
chapters/ch08-optimization.mdmarkdown3.5 KB
chapters/ch09-convolutional-networks.mdmarkdown3.4 KB
chapters/ch10-sequence-modeling.mdmarkdown3.5 KB
chapters/ch11-practical-methodology.mdmarkdown3.2 KB
chapters/ch12-applications.mdmarkdown3.0 KB
chapters/ch13-linear-factor-models.mdmarkdown3.1 KB
chapters/ch14-autoencoders.mdmarkdown3.3 KB
chapters/ch15-representation-learning.mdmarkdown3.5 KB
chapters/ch16-structured-probabilistic-models.mdmarkdown3.2 KB
chapters/ch17-monte-carlo-methods.mdmarkdown3.0 KB
chapters/ch18-partition-function.mdmarkdown3.5 KB
chapters/ch19-approximate-inference.mdmarkdown3.3 KB
chapters/ch20-deep-generative-models.mdmarkdown3.7 KB
cheatsheet.mdmarkdown3.2 KB
glossary.mdmarkdown5.9 KB
patterns.mdmarkdown4.7 KB
references/book_to_2026_delta.mdmarkdown6.4 KB
references/prerequisite_map.mdmarkdown4.9 KB
references/rights_and_use.mdmarkdown4.4 KB
references/study_method_canon.mdmarkdown5.3 KB
scripts/capacity_planner.pyscript14.4 KB
scripts/model_arithmetic.pyscript16.9 KB
scripts/reading_path_planner.pyscript19.7 KB
scripts/training_diagnostics.pyscript14.1 KB

chapters/ch17-monte-carlo-methods.md

View on GitHub
# Chapter 17: Monte Carlo Methods

**Source chapter (free, official):** https://www.deeplearningbook.org/contents/monte_carlo.html

## Core Idea

When a sum or integral is intractable, estimate it with samples. The estimator is unbiased and
its error shrinks as 1/√n regardless of dimension — which is why sampling survives where
quadrature does not. The difficulty moves from computing the integral to obtaining the samples.

## Key Concepts

- **Monte Carlo estimation**: approximate E_p[f(x)] by the sample mean; variance falls as 1/n,
  standard error as 1/√n.
- **Importance sampling**: sample from a tractable q and reweight by p/q. Variance depends
  entirely on how well q matches p·f — a bad proposal gives an estimator that is unbiased and
  useless.
- **Markov chain Monte Carlo (MCMC)**: build a chain whose stationary distribution is the target,
  then use its states as (correlated) samples.
- **Gibbs sampling**: resample each variable from its conditional given the rest; the natural
  MCMC scheme for graphical models.
- **Burn-in / mixing time**: the chain needs time to forget its initialization and to move
  between modes.
- **The mixing problem between separated modes**: with well-separated modes, transition
  probability between them is tiny and the chain reports a single mode as if it were everything.
- **Tempering / annealing** as remedies: flatten the distribution so the chain can travel.

## Mental Models

- Treat a Monte Carlo estimate as a **measurement with error bars**; report the standard error,
  because "the estimate is unbiased" says nothing about whether n was large enough.
- Diagnose a suspiciously confident sampler as a **mixing failure**, not a modelling success.
- Read importance sampling's variance condition as: **you must already know roughly where the
  mass is** — this is why proposals matter more than sample counts.

## Anti-patterns

- **Reporting MCMC samples without a mixing diagnostic.**
- **Trusting importance weights with huge dynamic range** — an effective sample size of ~1 is
  common and invisible unless measured.

## What changed after 2016

The chapter's methods remain textbook-correct, but their role in deep generative modelling
shrank: diffusion models replaced slow MCMC-based sampling with a fixed, finite denoising chain,
and modern generation is dominated by ancestral sampling from autoregressive models. Where MCMC
is still used — Bayesian deep learning, some energy-based models — HMC/NUTS variants dominate
over plain Gibbs. **Confidence: high.**

## Key Takeaways

1. Always pair a Monte Carlo estimate with its standard error.
2. Check mixing before believing anything an MCMC sampler tells you about multimodality.
3. Recognize the pattern "intractable expectation → sample it" — it recurs throughout Ch 18–20.

## Connects To

- **Ch 16**: the undirected models that need sampling in the first place.
- **Ch 18**: sampling as an ingredient in partition-function estimation.
- **Ch 20**: sampling as generation.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall