AI summary by TwinMind
AI Can’t Reason. Should It Drive Cars?
Sabine Hossenfelder
7m 54s
AI Reasoning Debate Overview
- Computer scientists believe making AI models bigger will eventually enable learning everything including reason
- Current AI models are deep neural nets trained on large data, sometimes with reinforcement learning
- Models learn to recognize patterns and interpolate/extrapolate from them
- Unknown how much they can learn with more data
Emergent Abilities in Large Models
- Some scientists claim models acquire emergent abilities as they get bigger
- Examples of emergent abilities:
- Unscrambling words
- Understanding relative locations in space
- Geographic maps comprehension
- Mathematical abilities
- Performance on standard math tests dramatically improved as models got bigger
Skepticism About AI Reasoning
- Language not good for encoding logic/maths - too vague
- People make logical mistakes constantly
- Models can learn patterns in training data but this doesn't equal reasoning
- Concern about AI use in cases requiring solid reasoning:
- Autonomous vehicles
- Government/military advisory
- Critical decision-making systems
GPT Math Performance Example
- Asked GPT for smallest integer whose square is larger than 5 but smaller than 17
- GPT gave garbled answers claiming answer is 3
- Correct answer is -4
- GPT doesn't understand integers or order relations
- Throws together text and math symbols that look plausible
- Output semantically similar but mathematically inaccurate
DeepMind/Apple Study on AI Reasoning
- Researchers from DeepMind and Apple systematically studied reasoning capabilities
- Tested large language models including GPT and LLaMA
- Used GSM 8K test (grade school math with ~8K questions)
- Example question: "James writes 3-page letter to 2 different friends twice a week, how many pages does he write each year?"
- Correct answer: 624
Study Methodology and Results
- Altered math questions by:
- Changing names
- Changing numbers
- Adding unnecessary text
- Math performance significantly dropped for all models
- Suggests models don't actually reason - just draw language relations and output similar learned patterns
- Performance drop only few percentage points
- Simple math test, so results may be worse than they appear
Human vs AI Reasoning Comparison
- Parody paper appeared from "Chicken Little Center for Stopping the Sky from Falling"
- Fictional authors claim humans are "crazy monkeys who excel at creative storytelling rather than genuine logical analysis"
- Example: humans advocating communism despite evidence it hasn't worked
- Questions why humans learned logical reasoning to begin with
Proposed Solution: Physics Training
- Humans learned logical reasoning because nature obeys rules of logic
- More AIs learn about physical reality, more likely to understand reasoning
- Suggestion: teach AI physics to improve reasoning abilities
Ground News Sponsorship
- News platform collecting/summarizing news from worldwide sources
- Provides quick summaries and extra information not in standard media
- Shows coverage breakdown by right/center/left leaning outlets
- Includes factuality ratings and media ownership information
- Special US election coverage page
- Discount available via ground.news/sabina link for Vantage plan
