Summary by TwinMind · 7m 10s

Summary by TwinMind · 7m 10s

The AI Revolution Hiding in Obscure Research

The AI Revolution Hiding in Obscure Research

Sabine Hossenfelder

Sabine Hossenfelder

Current AI Model Stagnation

  • Recent AI updates described as incremental
  • OpenAI GPT 4.5 (February release)
    • Sam Altman described as "first model that feels like talking to a thoughtful person"
    • Users described as "shiny new coat of paint on same old car," "nothing burger," "disappointing and underwhelming"
  • OpenAI O3 model issues
    • Hallucinates more than 4.5
    • Still can't count letters in "strawberry"
    • Falls for same riddles as previous LLMs
    • Also described as disappointing
  • Other model disappointments
    • Anthropic's Claude 3.7: similar disappointing story
    • Gemini 2.0: similar issues
    • Meta Llama recent version: actually worse than previous version

Industry Founder Experiences

  • Dean Valentine essay findings
    • YC founders doing AI application startups report similar experiences
    • Pattern: announcement → good benchmarks → mediocre actual performance
    • Gains reported publicly not reflective of economic usefulness or generality
    • "Models haven't improved much since August"

Industry Expert Perspectives

  • Current large language models reached diminishing returns phase
  • Significant progress requires "something really new"
  • Satya (former OpenAI): "2010s were age of scaling. Now back in age of wonder and discovery. Everyone looking for next thing"
  • Google CEO Sundar Pichai: predicts slow year for AI in 2025
  • Yann LeCun (Meta): thinks few years needed for next big step
  • Focus shifting to "obscure academic papers" for future breakthroughs

Current Model Improvements

  • Companies working on incremental enhancements
    • More thinking time
    • Better memory
    • Integration with other software (math, figures)
  • Meta's December announcement: Large Concept Models
    • Combines different verbal expressions of same logical relations
  • Meta chains of thought
    • Lets models evaluate different reasoning lines
  • These help with logical relations but aren't the new paradigm

World Models - The Next Paradigm

  • Future models will learn by interacting with world
  • Continue learning after training
  • DeepMind's Genie 2 (December announcement)
    • Trained on large video dataset
    • Generates interactive 3D environments
    • AI agents placed in virtual worlds to "learn to learn"
  • Nvidia's Cosmos platform (January)
    • Generates 3D models with built-in physics laws
    • Creates videos where objects don't appear from nowhere
    • Maintains consistent perspective
    • Trains other models on how reality works

Human Intelligence Parallel

  • World models mirror human intelligence evolution
  • Humans interact with physical world, create mental models
  • DeepMind calls theirs "foundation world model"
  • World models will play big role in next AI revolution
  • Next steps: systematic reasoning upgrades, not just more training
  • "Future's here already, just stuck in obscure academic journals"