AI summary by TwinMind

Bombshell Paper Shows AI Has Thinking Collapse. Or Does It?

Sabine Hossenfelder

6m 53s

Apple Paper on AI Reasoning Collapse

  • Apple researchers released paper claiming current AI reasoning models can't reason
  • Study examined large reasoning models (LLMs with chain of thought) on thinking puzzles
    • Chain of thought: extra instructions/training to break prompts into smaller steps
    • Models work steps separately, analyze, then recombine results
    • Can improve accuracy but doesn't mean conceptually new capabilities
  • Used puzzles with deterministic algorithmic solutions, gradual complexity increase
  • Definition of reasoning: models knowing when/how to use algorithms to solve puzzles
  • Key finding: "frontier large reasoning models face complete accuracy collapse beyond certain complexities"
  • Follow-up paper argued collapse due to token output limits (text length), not reasoning ability
    • Preliminary tests supported this counter-argument
  • Paper widely celebrated/criticized, sparked extensive discussion

Overlooked Human-Like AI Thinking Study

  • Released almost simultaneously with Apple paper but received little attention
  • Researchers studied how LLMs and visual models classify images
  • Method: gave humans and AI sets of 3 images, asked to identify odd one out
    • Repeated process to construct similarity measures
  • Key findings:
    • Models develop human-like conceptual representations of objects
    • Strong alignment between model embeddings and neural activity patterns in brain regions
    • Object representations in LLMs share fundamental similarities with human conceptual knowledge
  • Provides evidence AIs learn to think like humans in some sense

Analysis and Interpretation

  • Both algorithm execution and image classification are poor definitions of reasoning/thinking
  • Random people on street likely couldn't run algorithms - doesn't mean they can't reason
  • Current AIs think somewhat and reason a little, but not much and not generalizably
  • Major issue: lack of generalizability, not complete absence of reasoning
  • People expected models to acquire human reasoning capabilities automatically
  • Reality: models gradually handle more tasks with more effort/training/money
  • Models don't develop intelligence hallmarks:
    • Deductive/inductive analysis
    • Abstract thinking
    • Building correct theories/models
    • Quick learning
  • Computer executing algorithms is useful but not new
  • Current models aren't right approach for human-level AI
  • AGI imminent claims could be called "complete accuracy collapse"

Broader Implications

  • AI learning to code will become major internet safety problem
  • Companies heavily invested in current models may eventually admit limitations
  • Artificial intelligence everywhere but current approach may be fundamentally flawed

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome