AI summary by TwinMind

Bob McGrew: AI Agents And The Path To AGI

Y Combinator

30m 44s

AGI Definition and Current State

  • People traditionally defined AGI as model that passes Turing test, can look at things, write code, draw images
  • These capabilities have existed for years but haven't caused mass unemployment or "Game Over For Humanity"
  • Reaching bottleneck for pre-training and data, but new mechanism with reasoning and test-time compute emerging
  • Reasoning will unlock possibility of reliable agents to do actions on behalf of users

Bob McGrew's Background at OpenAI

  • Former Chief Research Officer at OpenAI
  • Originally intended to start robotics company after leaving Palantir in 2015
  • Learned deep learning wasn't ready for robotics startup in 2015-2016
  • Joined OpenAI as place to learn from smart people with big ambitions
  • Had management experience from Palantir but needed to become deep learning expert

Early OpenAI Projects and Culture

  • Goal was always to build AGI but early theory was through research and papers
  • Startup people (Sam, Greg, Bob) found academic approach painful but necessary
  • Early robotics project: taught humanoid robot hand to solve Rubik's Cube
    • Idea: complicated environments would force AI to generalize beyond narrow domains
  • Dota 2 project: solving video games as path to better AI
    • Insight: scale was path to improving AI
    • Could take huge amounts of experience, feed into neural network for learning/generalization
  • Alec Radford experimenting with language and GPT-1
    • Core idea: Transformer with simple objective of guessing next token would generate coherent text
    • No one thought this would work at the time
    • Alec persevered for years to make it work
  • Combined ideas from Dota/robot hand (scale + diverse data) with GPT-1 → GPT-2, GPT-3, GPT-4

OpenAI's Unique Culture vs Competitors

  • DeepMind: Demis had big plan, hired researchers to execute his vision
  • Google Brain: "rebuild Academia" - bring talented researchers, give resources, let them figure out what to do
  • OpenAI: startup approach with no centralized plan but had opinions about what needed to be done
  • Focus on showing scale as way to improve ideas
  • Leadership (Ilia, Dario) set opinions to avoid just throwing resources at everyone
  • Found happy medium between centralized planning and complete freedom

Research Collaboration and Credit

  • Academia has incentives preventing collaboration due to obsession with credit
  • OpenAI made decision to be "Catholic" in putting everyone's name on papers
  • Early robotics papers cited as "OpenAI" to avoid first author fights
  • Channeled credit desire into internal reputation rather than paper position
  • OpenAI had few titles early on but people knew who great researchers were

Scaling Laws Across Domains

  • Scaling laws exist across many AI domains, not just LLMs
  • First step is getting to scaling law (zero-to-one problem)
  • Example: Aditya Ramesh spent 18 months on DALL-E getting first version to work
    • Trying to make "pink panda skating on ice" (not in training set)
    • Just getting to plausibly working version is huge difficult problem
  • Once it works, scaling laws enable two approaches:
    • Pure scale itself (systems/data/algorithmic problem)
    • Change slope of scaling law (better architectures, optimization algorithms)

Data Wall and New Scaling Mechanisms

  • Definitely hitting data wall for traditional pre-training techniques
  • Shift from LLM pre-training scaling to reasoning techniques (O1, O3 models, Gemini Flash Thinking)
  • Moore's Law analogy: sum of S-curves, solve one bottleneck then attack next
  • Pre-training/data bottleneck being reached, but reasoning and test-time compute as new mechanism
  • Clear path now exists for scaling - no longer in zero-to-one phase for LLMs, now in pure scaling regime

Five Levels of AGI and Scientific Innovation

  • Reasoners are here, innovators coming next
  • O3 or O3 Pro could be hooked up to biolab for autonomous scientific exploration
  • Will likely have models that can explore scientific hypotheses and design experiments before having robots to run them
  • Back to robotics as next S-curve bottleneck

Reasoning Models and Agents

  • Reasoning = ability for model to have coherent chain of thought making progress over long periods
  • Techniques for thinking harder also apply to taking action in real/virtual world
  • Reasoning will unlock agents to do actions on user's behalf
  • Always been possible but never good enough - need high reliability
  • Rule of thumb: adding nine (90→99% or 99→99.9%) requires order of magnitude more compute
  • Historically achieved through bigger models, now through longer thinking time
  • Clear path to higher reliability levels unlocks many downstream applications

Distillation and Model Efficiency

  • Big Frontier labs figured out tricks to take big models and train much smaller, faster models for specific distributions
  • Trend across all labs: Sonnet vs Haiku, Gemini vs Gemini Flash, O1 vs O1 Mini, GPT-4 vs GPT-4 Mini
  • Distillation as a service emerging

Advice for AI Startup Founders

  • Start with very best model available
  • Startup success requires exploiting something on AI frontier
  • Get it working first, then use distillation/smaller models for cost optimization
  • Most important resource is time - don't spend 3 years getting to market like Palantir
  • Figure out value through user iteration, then optimize for cost

Personal AI and The Movie "Her"

  • Skeptical of deep emotional AI girlfriend connections
  • More promising: AI that shops for you, workplace assistant, personal AI bot representing you
  • Needs huge amount of context: Slack, Gmail, all productivity tools
  • Real market gap - can't purchase this today
  • Vision: super intelligent genie that knows your goals, proactively suggests actions, handles appointments

AI Adoption Mystery

  • By 2018, AI had capabilities people thought would automate all laptop jobs and end humanity
  • None of that happened - AI adoption much slower than expected
  • Effects visible in coding but not in productivity statistics (except data center construction)
  • Deep mystery why adoption so slow relative to 2018 expectations

Palantir Lessons for AI Adoption

  • Palantir discovered technology exists but not evenly distributed
  • Found government agencies making critical decisions with no software
  • Success came from reimagining workflows with technology, not just speeding up existing processes
  • Example: instead of faster database searches, check all databases at once
  • AI needs similar transformation - solve actual problems, not just automate existing workflows
  • Need forward deployed engineers: sit with customers, understand their needs, build perfect software for them
  • Initially skeptical of forward deployed model, now everyone building these functions

Parenting in AI Age

  • Struggle with teaching 8-year-old son coding when AI will be able to code
  • Using language models to create personalized coding lessons
  • Believes in "resistance of the medium" - value in having hands-on experience
  • Two future job roles: lone genius (like Alec Radford) and manager (CEO of mostly-AI firm)
  • Analogy to photography: cameras didn't eliminate artists, created more art appreciation
  • Historical precedent: 1880s farmers (90% of jobs) became 3% of population, new jobs emerged
  • Optimistic about humanity having important roles, but jobs of grandchildren will be unrecognizable

Robotics Future Outlook

  • Robotics companies now where LLM companies were 5 years ago
  • Expects ChatGPT moment for robotics within next 5 years
  • Harder to scale due to physical robot requirements
  • Companies like Skild AI, Physical Intelligence building foundation models for robots
  • Will transition from zero-to-one phase to scaling for reliability and market scope

Scientific Acceleration and Bottlenecks

  • May automate scientists/innovators before experiment executors
  • Potential for fast scientific advancement when combined with robotics
  • Expects to find new bottlenecks - science may only go 30% faster instead of 300x faster
  • Will need to figure out remaining constraints

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome