AI summary by TwinMind
Bombshell Paper Shows AI Has Thinking Collapse. Or Does It?
Sabine Hossenfelder
6m 53s
Apple Paper on AI Reasoning Collapse
- Apple researchers released paper claiming current AI reasoning models can't reason
- Study examined large reasoning models (LLMs with chain of thought) on thinking puzzles
- Chain of thought: extra instructions/training to break prompts into smaller steps
- Models work steps separately, analyze, then recombine results
- Can improve accuracy but doesn't mean conceptually new capabilities
- Used puzzles with deterministic algorithmic solutions, gradual complexity increase
- Definition of reasoning: models knowing when/how to use algorithms to solve puzzles
- Key finding: "frontier large reasoning models face complete accuracy collapse beyond certain complexities"
- Follow-up paper argued collapse due to token output limits (text length), not reasoning ability
- Preliminary tests supported this counter-argument
- Paper widely celebrated/criticized, sparked extensive discussion
Overlooked Human-Like AI Thinking Study
- Released almost simultaneously with Apple paper but received little attention
- Researchers studied how LLMs and visual models classify images
- Method: gave humans and AI sets of 3 images, asked to identify odd one out
- Repeated process to construct similarity measures
- Key findings:
- Models develop human-like conceptual representations of objects
- Strong alignment between model embeddings and neural activity patterns in brain regions
- Object representations in LLMs share fundamental similarities with human conceptual knowledge
- Provides evidence AIs learn to think like humans in some sense
Analysis and Interpretation
- Both algorithm execution and image classification are poor definitions of reasoning/thinking
- Random people on street likely couldn't run algorithms - doesn't mean they can't reason
- Current AIs think somewhat and reason a little, but not much and not generalizably
- Major issue: lack of generalizability, not complete absence of reasoning
- People expected models to acquire human reasoning capabilities automatically
- Reality: models gradually handle more tasks with more effort/training/money
- Models don't develop intelligence hallmarks:
- Deductive/inductive analysis
- Abstract thinking
- Building correct theories/models
- Quick learning
- Computer executing algorithms is useful but not new
- Current models aren't right approach for human-level AI
- AGI imminent claims could be called "complete accuracy collapse"
Broader Implications
- AI learning to code will become major internet safety problem
- Companies heavily invested in current models may eventually admit limitations
- Artificial intelligence everywhere but current approach may be fundamentally flawed
