AI summary by TwinMind
Scaling laws are explained by memorization and not intelligence – Francois Chollet
Dwarkesh Patel
6m 9s
Definition of General Intelligence
- General intelligence ≠ task-specific skill scaled up to many skills
- Infinite space of possible skills exists
- General intelligence = ability to approach any problem/skill and quickly master it using minimal data
- Generality = ability to apply mind to arbitrary things, not specificity scaled up
- Requires fundamentally: ability to adapt and learn on-the-fly efficiently
Scaling Laws and Performance Measurement
- Maximalist argument relies on scaling laws
- Scaling laws = empirical relationship between compute spent on training and benchmark performance
- Key question: how do you measure performance and what improves with more compute/data?
- Performance measurement narrows set of questions asked and answers sought
- Measurement method is not technical detail/afterthought
Current LLM Benchmarks Analysis
- All memorization-based benchmarks
- Sometimes literally knowledge-based (school tests)
- Even reasoning benchmarks: solvable by memorizing finite set of reasoning patterns
- LLMs = static programs that memorize small static programs
- Have "bank of solution programs" - fetch appropriate program when given new puzzle
- Appears like reasoning but actually program fetching, not on-the-fly program synthesis
LLM Architecture and Scaling
- Models = big parametric curves fitted to data distribution via gradient descent
- Essentially big interpolative databases/memories
- Scaling up database size + cramming more knowledge/patterns = increased memorization benchmark performance
- Scaling increases system skill, usefulness, scope of applicability
- Does NOT increase intelligence ("skill ≠ intelligence")
- Fundamental confusion: people conflate skill and intelligence
GSM8K Benchmark Example
- Great school math problems smart high schooler could solve
- Models achieve 95% accuracy (excel at memorization)
- Example problem: 30 students, 1/5 are 12yo, 1/3 are 13yo, 1/10 are 11yo - how many not 11, 12, or 13?
- Not rocket science, solvable on paper by smart high school student
Two Definitions of Reasoning
Definition 1: Template-Based Reasoning
- Have available set of program templates (puzzle structure + solution)
- Identify right template from memory
- Input new values into template, run program, get solution
- Could be called reasoning
Definition 2: On-the-Fly Program Synthesis
- When faced with puzzle without existing program in memory
- Must synthesize new program on-the-fly based on pieces of existing programs
- Dramatically harder than fetching/applying memorized program
- True reasoning capability
Human Learning Comparison
- Humans also not infinitely sample efficient
- Require training/drilling in reasoning pathways for certain problems
- Math example: can't show baby set theory axioms and expect them to know math
- Requires years: pre-algebra → algebra → geometry → pre-calculus → calculus
- Both humans and models need drilling with training data
- For on-the-fly program synthesis, need building blocks (knowledge/memory)
- Memory and reasoning not opposing forces - memory tremendously important for effective reasoning
