AI summary by TwinMind

9 Years to AGI? OpenAI’s Dan Roberts Reasons About Emulating Einstein

Sequoia Capital

10m 10s

Background and Introduction

  • Dan Roberts, former Sequoia team member, now at OpenAI
  • Worked on reasoning models for 2.5-3 years
  • Anecdote about accidental announcement of his OpenAI move at last year's AI Ascent

O1 Model Release and Test-Time Compute Breakthrough

  • September release of O1 model by OpenAI
  • Key innovation: performance improved with both train-time and test-time compute
  • Left plot: traditional scaling with training compute
  • Right plot: revolutionary scaling with test-time compute - model "thinks" longer and performs better
  • Described as "totally new dimension for scaling" - put on company t-shirt

O3 Model Capabilities Demonstration

  • O3 released last month - even better reasoning model
  • Physics problem demonstration: quantum electrodynamics with Feynman diagrams
  • Model solved complex physics problem in ~1 minute
  • Roberts needed 3 hours to verify the calculation across 4 textbooks
  • Shows current capability: reproduce textbook calculations and perturbations

Einstein Thought Experiment

  • Hypothetical: asking Einstein in 1907 (pre-general relativity) a GR final exam question
  • GPT-4.5 couldn't solve it, but O3 could
  • Einstein took 8 years to develop general relativity theory
  • Goal: models should make major contributions to human knowledge and science

Scaling Strategy and Compute Investment

  • Focus on scaling reinforcement learning (RL) compute
  • Evolution from GPT-4O (all pre-training compute) → O1 (added RL compute) → O3 (more RL compute)
  • Future vision: RL compute will dominate over pre-training compute
  • Contrarian view opposing Yann LeCun's 2019 slide showing RL as "cherry on top" of pre-training "cake"
  • OpenAI wants to "invert the meme" - crush small pre-training cake with giant RL cherry

Infrastructure and Investment Plans

  • $500 billion fundraising target
  • Land acquisition in Abilene, Texas
  • Building data centers with compute infrastructure
  • Revenue generation to fund more expansion

Scaling Science Development

  • Need to develop new scaling laws for test-time compute and RL training
  • Reference to GPT-4 predictive scaling success - predicted final performance from smaller experiments
  • Must "throw everything out the window" and reinvent scaling approaches for new paradigms

Current Limitations and Future Predictions

  • Models currently feel like "idiot savants" - not discovering general relativity yet
  • Possible issues:
    • Asking wrong types of questions
    • Training on too many competition math problems
    • Jagged performance across different domains
  • Solution: scale up further

Timeline Projections

  • Current AI agents can handle ~1 hour tasks
  • Task length doubling every 7 months
  • Next year prediction: 2-2.5 hours
  • Einstein's 8-year thinking timeline = 16 doubling periods
  • Projection: 9 years until model discovers general relativity equivalent breakthroughs

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome