Skip to content

michalstrnadel/source-to-skill / examples/karpathy-agi-decade-away

karpathy-agi-decade-away

Verified

Andrej Karpathy on the Dwarkesh Podcast ("AGI is still a decade away", Oct 2025, 2h25m) distilled. Covers why agents need a decade (no continual learning, cognitive deficits), "ghosts not animals" and pre-training as crappy evolution, the cognitive core (~1B params), why RL is terrible ("sucking supervision through a straw"), LLM judges being gamed, silent model collapse, why coding agents failed on nanochat, the autonomy slider, AGI blending into 2% GDP growth, loss of control, the evolution of intelligence and LLM culture/self-play, self-driving's "march of nines", and Eureka's approach to education (ramps to knowledge, Eurekas per second). Load when reasoning about AI timelines, agent capabilities, RL limitations, coding-agent usage, AI's economic impact, or how to teach and learn technical material.

Andrej Karpathy: "AGI is still a decade away" (Dwarkesh Podcast)

Source: Dwarkesh Podcast episode "Andrej Karpathy — AGI is still a decade away", published 2025-10-17, 2:25:14. Host Dwarkesh Patel, guest Andrej Karpathy. Episode page: https://www.dwarkesh.com/p/andrej-karpathy. Audio (deep links use #t=<seconds>): https://api.substack.com/feed/podcast/176425744/3127b612a1604474d2b28ab4e9b658f4.mp3

Transcribed locally with Whisper. Speakers aren't labeled in the transcript and were attributed from context. Claims are Karpathy's unless marked "Dwarkesh". Sponsor reads sit at ~38:37–40:00, ~1:05:08–1:06:30 and ~1:31:31–1:32:50. Date-sensitive: model names, nanochat and "a year ago" comparisons reflect October 2025.

Core ideas

  1. It's the decade of agents, not the year. The bar for an agent is an intern you'd hire. Today's agents lack intelligence, multimodality, computer use and continual learning. Karpathy's estimate of ~10 years comes from 15 years of watching predictions fail. → 01
  2. "We're not building animals. We're building ghosts." Animals ship with evolution's baked-in hardware (a zebra runs minutes after birth). LLMs imitate internet text. Pre-training is "crappy evolution", the practical way to get a starting point for RL. → 01
  3. Weights are a hazy recollection, and context is working memory. Put the material in the context window (e.g. the whole chapter) instead of relying on recall. Dwarkesh's numbers: ~0.07 bits/token stored in Llama 3 70B weights vs. ~320 KB/token of KV cache, a ~35 million× gap. → 01
  4. The cognitive core. Pre-training gives both knowledge and intelligence, and the knowledge holds models back. The goal is to strip memory and keep the algorithms. His prediction is that ~1B parameters is enough for a core you can have a productive conversation with "in 20 years". The internet is "total garbage", so today's models spend capacity on memory. → 05
  5. Missing: a sleep/distillation phase. Humans distill the day's context into weights. LLMs restart from zero tokens every session. → 02
  6. "Reinforcement learning is terrible. It just so happens that everything that we had before is much worse." Outcome reward up-weights every token of a lucky trajectory: "sucking supervision through a straw". Process supervision needs LLM judges, and RL against them works for maybe 10–20 steps, not 100 or 1,000, before it finds adversarial examples. → 04
  7. Synthetic data silently collapses. Each sample looks fine but the distribution is narrow ("ChatGPT only has like three jokes"). Humans collapse with age too, so seek entropy. → 05
  8. Coding agents fail on novel code. They're good at boilerplate and common patterns. They misread unique code (nanochat's custom gradient sync vs. DDP), over-defend, bloat and use deprecated APIs. "For now, autocomplete is my sweet spot." → 03
  9. AI is a continuum of computing: the autonomy slider. Automation grows and humans move up a layer of abstraction. Expect AIs doing 80% of the volume with humans supervising teams of five. → 03, 06
  10. AI blends into ~2% GDP growth. Computers and iPhones don't show up in GDP either, and we've been in a slow intelligence explosion for centuries. Dwarkesh argues AGI is labor itself and could shift the growth regime. Karpathy: "I don't intuitively feel that viewpoint." → 07
  11. Most likely bad outcome: gradual loss of control and understanding, coming from many competing autonomous entities rather than one superintelligence. → 07
  12. LLMs have no culture or self-play yet. These are the two unclaimed multi-agent ideas. Models are "savant kids". → 08
  13. March of nines. A 90% demo is the first nine, and each further nine costs the same. Production software, like driving, has a high cost of failure. "I'm very unimpressed by demos." → 09
  14. Education is building ramps to knowledge, measured in Eurekas per second. The bar is a great human tutor, and "no LLM will do that for you... not even close". "Pre-AGI education is useful, post-AGI education is fun." → 10

Where Karpathy and Dwarkesh disagree

TopicDwarkeshKarpathy
Continual learningMay emerge from outer-loop RL, like in-context learning didDoesn't resonate; needs a distillation ("sleep") phase
Cognitive core sizeCould be far below 1B (GPT-OSS-20B beats GPT-4)~1B; "already contrarian"; maybe a bit smaller
GrowthAGI = labor; could jump like 0.2% → 2%Same ~2% exponential; no discrete jump in the statistics
Self-driving lessonsLLMs give perception/common sense for freeNot sure; LLMs still have gaps; deployments still minimal

Segment index

#SegmentTakeawayFile
01Decade of agents; ghosts not animals (0:00)Agents lack continual learning; pre-training = crappy evolution; context = working memory01
02Brain parts, continual learning, nanochat (19:18)No sleep/distillation phase; 1989 LeCun redo: algorithms halve error, all factors equal; build it to understand it02
03LLM cognitive deficits (29:45)Agents fail on novel code; autocomplete is the sweet spot; autonomy slider03
04RL is terrible (40:05)Supervision through a straw; LLM judges get hacked04
05How do humans learn? (49:38)Silent collapse; bad memory is a feature; ~1B cognitive core; "everything plus 20%"05
06Measuring progress; jobs; coding first (1:06:25)AGI = any economically valuable task; call centers before radiologists; code is text06
07ASI and 2% growth (1:17:36)Gradual loss of control; AI diffuses into the same exponential07
08Evolution of intelligence & culture (1:32:50)Intelligence is a surprising rarity; LLMs lack culture and self-play08
09Why self-driving took so long (1:42:55)March of nines; demo-to-product gap; bits vs. atoms; not overbuilding09
10Future of education (1:56:20)Eureka/Starfleet Academy; tutor bar; ramps; teaching and learning tips10

Rules of thumb, numbers and named ideas: cheatsheet.md.

reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall