Skip to content

alirezarezvani/claude-skills / engineering/deep-learning-book/skills/deep-learning-book

deep-learning-book

Verified

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

FileKindSize
SKILL.mdskill md10.8 KB
assets/chapter_worksheet.mdmarkdown2.3 KB
assets/example_layer_spec.jsonconfig1.2 KB
assets/study_log_template.mdmarkdown2.0 KB
chapters/ch01-introduction.mdmarkdown3.1 KB
chapters/ch02-linear-algebra.mdmarkdown3.1 KB
chapters/ch03-probability-information-theory.mdmarkdown3.6 KB
chapters/ch04-numerical-computation.mdmarkdown3.0 KB
chapters/ch05-machine-learning-basics.mdmarkdown3.5 KB
chapters/ch06-deep-feedforward-networks.mdmarkdown3.2 KB
chapters/ch07-regularization.mdmarkdown3.6 KB
chapters/ch08-optimization.mdmarkdown3.5 KB
chapters/ch09-convolutional-networks.mdmarkdown3.4 KB
chapters/ch10-sequence-modeling.mdmarkdown3.5 KB
chapters/ch11-practical-methodology.mdmarkdown3.2 KB
chapters/ch12-applications.mdmarkdown3.0 KB
chapters/ch13-linear-factor-models.mdmarkdown3.1 KB
chapters/ch14-autoencoders.mdmarkdown3.3 KB
chapters/ch15-representation-learning.mdmarkdown3.5 KB
chapters/ch16-structured-probabilistic-models.mdmarkdown3.2 KB
chapters/ch17-monte-carlo-methods.mdmarkdown3.0 KB
chapters/ch18-partition-function.mdmarkdown3.5 KB
chapters/ch19-approximate-inference.mdmarkdown3.3 KB
chapters/ch20-deep-generative-models.mdmarkdown3.7 KB
cheatsheet.mdmarkdown3.2 KB
glossary.mdmarkdown5.9 KB
patterns.mdmarkdown4.7 KB
references/book_to_2026_delta.mdmarkdown6.4 KB
references/prerequisite_map.mdmarkdown4.9 KB
references/rights_and_use.mdmarkdown4.4 KB
references/study_method_canon.mdmarkdown5.3 KB
scripts/capacity_planner.pyscript14.4 KB
scripts/model_arithmetic.pyscript16.9 KB
scripts/reading_path_planner.pyscript19.7 KB
scripts/training_diagnostics.pyscript14.1 KB

chapters/ch16-structured-probabilistic-models.md

View on GitHub
# Chapter 16: Structured Probabilistic Models for Deep Learning

**Source chapter (free, official):** https://www.deeplearningbook.org/contents/graphical_models.html

## Core Idea

A joint distribution over n variables is exponentially large; a graph that encodes conditional
independence makes it tractable. Directed models factor into conditionals, undirected models into
unnormalized potentials divided by a partition function — and that partition function is the bill
Chapters 17–19 spend their pages paying.

## Frameworks Introduced

- **The challenge of unstructured modelling**: memory, statistical efficiency, and inference cost
  all scale with the number of parameters in a full joint.
- **Directed models (Bayesian networks)**: p(x) = ∏ p(xᵢ | parents(xᵢ)). Normalized by
  construction — no partition function.
- **Undirected models (Markov random fields)**: p(x) = (1/Z) ∏ φ_c(x_c) over cliques. Z is the
  partition function, and it is a sum over all configurations.
- **Energy-based models**: p(x) ∝ exp(−E(x)). Any positive distribution can be written this way.
- **Separation / d-separation**: reading conditional independence off the graph.
- **Converting between graph types; factor graphs** for disambiguating factorization.
- **Sampling from graphical models**: ancestral sampling (easy, directed) vs Gibbs sampling
  (needed for undirected).
- **Structure learning and latent variables**; the **restricted Boltzmann machine (RBM)** as the
  worked example.

## Mental Models

- Use the rule of thumb: **directed = easy sampling, harder inference with explaining-away;
  undirected = natural for mutual constraints, hard normalization.**
- Read an energy function as a **soft constraint set**: low energy where constraints are
  satisfied. Design E, and p follows.
- Remember that the partition function is not an inconvenience — it is **the** obstacle that
  organizes all of Part III.

## Anti-patterns

- **Writing an undirected model and ignoring Z** until training refuses to work.
- **Assuming a graph's missing edge means independence in the data** — it means the *model*
  asserts independence.

## What changed after 2016

Deep undirected graphical models (deep Boltzmann machines, RBM stacks) are now largely historical
as generative workhorses. But energy-based modelling did not die: it returned via score matching
and diffusion (which sidestep Z entirely by learning ∇ log p rather than p), and via
energy-based reinterpretations of contrastive learning. Autoregressive factorization — the
directed side of this chapter — became the dominant paradigm through transformer language models,
which are exactly chain-rule factorizations with a huge neural conditional. **Confidence: high.**

## Key Takeaways

1. Classify any generative proposal as directed or undirected first; it predicts which problems
   you will have.
2. When Z is intractable, look for a formulation that never needs it (score, ratio, or
   autoregressive).
3. Read modern LLMs as ancestral sampling from a directed chain — the framing here still applies.

## Connects To

- **Ch 17–19**: the three responses to intractability (sample it, approximate Z, approximate the
  posterior).
- **Ch 20**: the models built on these foundations.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall