AI summary by TwinMind
9 Years to AGI? OpenAI’s Dan Roberts Reasons About Emulating Einstein
Sequoia Capital
10m 10s
Background and Introduction
- Dan Roberts, former Sequoia team member, now at OpenAI
- Worked on reasoning models for 2.5-3 years
- Anecdote about accidental announcement of his OpenAI move at last year's AI Ascent
O1 Model Release and Test-Time Compute Breakthrough
- September release of O1 model by OpenAI
- Key innovation: performance improved with both train-time and test-time compute
- Left plot: traditional scaling with training compute
- Right plot: revolutionary scaling with test-time compute - model "thinks" longer and performs better
- Described as "totally new dimension for scaling" - put on company t-shirt
O3 Model Capabilities Demonstration
- O3 released last month - even better reasoning model
- Physics problem demonstration: quantum electrodynamics with Feynman diagrams
- Model solved complex physics problem in ~1 minute
- Roberts needed 3 hours to verify the calculation across 4 textbooks
- Shows current capability: reproduce textbook calculations and perturbations
Einstein Thought Experiment
- Hypothetical: asking Einstein in 1907 (pre-general relativity) a GR final exam question
- GPT-4.5 couldn't solve it, but O3 could
- Einstein took 8 years to develop general relativity theory
- Goal: models should make major contributions to human knowledge and science
Scaling Strategy and Compute Investment
- Focus on scaling reinforcement learning (RL) compute
- Evolution from GPT-4O (all pre-training compute) → O1 (added RL compute) → O3 (more RL compute)
- Future vision: RL compute will dominate over pre-training compute
- Contrarian view opposing Yann LeCun's 2019 slide showing RL as "cherry on top" of pre-training "cake"
- OpenAI wants to "invert the meme" - crush small pre-training cake with giant RL cherry
Infrastructure and Investment Plans
- $500 billion fundraising target
- Land acquisition in Abilene, Texas
- Building data centers with compute infrastructure
- Revenue generation to fund more expansion
Scaling Science Development
- Need to develop new scaling laws for test-time compute and RL training
- Reference to GPT-4 predictive scaling success - predicted final performance from smaller experiments
- Must "throw everything out the window" and reinvent scaling approaches for new paradigms
Current Limitations and Future Predictions
- Models currently feel like "idiot savants" - not discovering general relativity yet
- Possible issues:
- Asking wrong types of questions
- Training on too many competition math problems
- Jagged performance across different domains
- Solution: scale up further
Timeline Projections
- Current AI agents can handle ~1 hour tasks
- Task length doubling every 7 months
- Next year prediction: 2-2.5 hours
- Einstein's 8-year thinking timeline = 16 doubling periods
- Projection: 9 years until model discovers general relativity equivalent breakthroughs
