AI summary by TwinMind
The AI Revolution Hiding in Obscure Research
Sabine Hossenfelder
7m 10s
Current AI Model Stagnation
- Recent AI updates described as incremental
- OpenAI GPT 4.5 (February release)
- Sam Altman described as "first model that feels like talking to a thoughtful person"
- Users described as "shiny new coat of paint on same old car," "nothing burger," "disappointing and underwhelming"
- OpenAI O3 model issues
- Hallucinates more than 4.5
- Still can't count letters in "strawberry"
- Falls for same riddles as previous LLMs
- Also described as disappointing
- Other model disappointments
- Anthropic's Claude 3.7: similar disappointing story
- Gemini 2.0: similar issues
- Meta Llama recent version: actually worse than previous version
Industry Founder Experiences
- Dean Valentine essay findings
- YC founders doing AI application startups report similar experiences
- Pattern: announcement → good benchmarks → mediocre actual performance
- Gains reported publicly not reflective of economic usefulness or generality
- "Models haven't improved much since August"
Industry Expert Perspectives
- Current large language models reached diminishing returns phase
- Significant progress requires "something really new"
- Satya (former OpenAI): "2010s were age of scaling. Now back in age of wonder and discovery. Everyone looking for next thing"
- Google CEO Sundar Pichai: predicts slow year for AI in 2025
- Yann LeCun (Meta): thinks few years needed for next big step
- Focus shifting to "obscure academic papers" for future breakthroughs
Current Model Improvements
- Companies working on incremental enhancements
- More thinking time
- Better memory
- Integration with other software (math, figures)
- Meta's December announcement: Large Concept Models
- Combines different verbal expressions of same logical relations
- Meta chains of thought
- Lets models evaluate different reasoning lines
- These help with logical relations but aren't the new paradigm
World Models - The Next Paradigm
- Future models will learn by interacting with world
- Continue learning after training
- DeepMind's Genie 2 (December announcement)
- Trained on large video dataset
- Generates interactive 3D environments
- AI agents placed in virtual worlds to "learn to learn"
- Nvidia's Cosmos platform (January)
- Generates 3D models with built-in physics laws
- Creates videos where objects don't appear from nowhere
- Maintains consistent perspective
- Trains other models on how reality works
Human Intelligence Parallel
- World models mirror human intelligence evolution
- Humans interact with physical world, create mental models
- DeepMind calls theirs "foundation world model"
- World models will play big role in next AI revolution
- Next steps: systematic reasoning upgrades, not just more training
- "Future's here already, just stuck in obscure academic journals"
