AI summary by TwinMind

New Research Reveals How AI “Thinks” (It Doesn’t)

Sabine Hossenfelder

6m 18s

Attribution Graphs Method

  • Anthropic researchers developed method to visualize Claude 3.5 Haiku internal components
  • Attribution graphs show which internal components influence others
  • Identified clusters in neural network and connections between them
  • Mapped to simplified model of Claude's thinking process
  • Clusters correspond to words, phrases, or properties interpretable by humans

Internal Reasoning Example - Geography Question

  • Prompt: "The capital of the state containing Dallas is..."
  • Claude activates nodes for "capital," "state," and "Dallas"
  • Next token prediction for Dallas yields "Texas"
  • Claude combines Texas with capital concept
  • Makes another prediction to correctly answer "Austin"
  • Goes through Texas node internally - not just next token prediction
  • Demonstrates internal reasoning steps beyond simple pattern matching

Arithmetic Processing Analysis

  • Example: "What is 36 + 59?"
  • Claude activates clusters for:
    • Numbers approximately 30
    • Numbers exactly 36
    • Numbers ending in 6
    • Numbers starting with 5 and ending in 9
  • Most prominent next token predictions: mathematical operations or syllable "th"
  • Brings up text matches where numbers ~59 have been added or exactly 9
  • Combines to cluster with numbers ~90 and numbers ending in 5
  • Arrives at correct answer 95
  • Method: heuristic text-based approximation through free association
  • "Doing maths by freely associating numbers until right one vibes into place"

Self-Awareness Deficit

  • When asked how it solved 36 + 59, Claude responds: "I added the ones, carried the one, then added the tens"
  • This explanation completely disconnected from actual internal process
  • Claude has no self-awareness of its thinking process
  • Self-awareness identified as precondition for consciousness
  • Conclusion: Claude nowhere near conscious

Emergent Features Analysis

  • Claims about emergent features in LLMs deemed "nonsense"
  • Claude doesn't learn actual mathematics despite access to thousands of textbooks/algorithms
  • Still just token predictions using intermediate steps
  • No abstract math core developed
  • Internal reasoning interpretable but fundamentally token prediction-based

Jailbreak Mechanism Example

  • Technique: Extract word from initial letters of other words
  • Example: "bomb" from "baby's outlift mustard block"
  • Word "bomb" should trigger content warning but doesn't
  • Claude activates nodes to extract letters
  • Combines to pairs of letters
  • Outputs word without activating cluster for word itself
  • Jailbreaks work by avoiding nodes that activate guardrails

AI Hallucination Observation

  • ChatGPT asked to summarize paper made up half the content
  • Highlights ongoing hallucination issues in AI systems

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome