AI summary by TwinMind

Scaling laws are explained by memorization and not intelligence – Francois Chollet

Dwarkesh Patel

6m 9s

Definition of General Intelligence

  • General intelligence ≠ task-specific skill scaled up to many skills
  • Infinite space of possible skills exists
  • General intelligence = ability to approach any problem/skill and quickly master it using minimal data
  • Generality = ability to apply mind to arbitrary things, not specificity scaled up
  • Requires fundamentally: ability to adapt and learn on-the-fly efficiently

Scaling Laws and Performance Measurement

  • Maximalist argument relies on scaling laws
  • Scaling laws = empirical relationship between compute spent on training and benchmark performance
  • Key question: how do you measure performance and what improves with more compute/data?
  • Performance measurement narrows set of questions asked and answers sought
  • Measurement method is not technical detail/afterthought

Current LLM Benchmarks Analysis

  • All memorization-based benchmarks
  • Sometimes literally knowledge-based (school tests)
  • Even reasoning benchmarks: solvable by memorizing finite set of reasoning patterns
  • LLMs = static programs that memorize small static programs
  • Have "bank of solution programs" - fetch appropriate program when given new puzzle
  • Appears like reasoning but actually program fetching, not on-the-fly program synthesis

LLM Architecture and Scaling

  • Models = big parametric curves fitted to data distribution via gradient descent
  • Essentially big interpolative databases/memories
  • Scaling up database size + cramming more knowledge/patterns = increased memorization benchmark performance
  • Scaling increases system skill, usefulness, scope of applicability
  • Does NOT increase intelligence ("skill ≠ intelligence")
  • Fundamental confusion: people conflate skill and intelligence

GSM8K Benchmark Example

  • Great school math problems smart high schooler could solve
  • Models achieve 95% accuracy (excel at memorization)
  • Example problem: 30 students, 1/5 are 12yo, 1/3 are 13yo, 1/10 are 11yo - how many not 11, 12, or 13?
  • Not rocket science, solvable on paper by smart high school student

Two Definitions of Reasoning

Definition 1: Template-Based Reasoning

  • Have available set of program templates (puzzle structure + solution)
  • Identify right template from memory
  • Input new values into template, run program, get solution
  • Could be called reasoning

Definition 2: On-the-Fly Program Synthesis

  • When faced with puzzle without existing program in memory
  • Must synthesize new program on-the-fly based on pieces of existing programs
  • Dramatically harder than fetching/applying memorized program
  • True reasoning capability

Human Learning Comparison

  • Humans also not infinitely sample efficient
  • Require training/drilling in reasoning pathways for certain problems
  • Math example: can't show baby set theory axioms and expect them to know math
  • Requires years: pre-algebra → algebra → geometry → pre-calculus → calculus
  • Both humans and models need drilling with training data
  • For on-the-fly program synthesis, need building blocks (knowledge/memory)
  • Memory and reasoning not opposing forces - memory tremendously important for effective reasoning

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome