AI summary by TwinMind
Bob McGrew: AI Agents And The Path To AGI
Y Combinator
30m 44s
AGI Definition and Current State
- People traditionally defined AGI as model that passes Turing test, can look at things, write code, draw images
- These capabilities have existed for years but haven't caused mass unemployment or "Game Over For Humanity"
- Reaching bottleneck for pre-training and data, but new mechanism with reasoning and test-time compute emerging
- Reasoning will unlock possibility of reliable agents to do actions on behalf of users
Bob McGrew's Background at OpenAI
- Former Chief Research Officer at OpenAI
- Originally intended to start robotics company after leaving Palantir in 2015
- Learned deep learning wasn't ready for robotics startup in 2015-2016
- Joined OpenAI as place to learn from smart people with big ambitions
- Had management experience from Palantir but needed to become deep learning expert
Early OpenAI Projects and Culture
- Goal was always to build AGI but early theory was through research and papers
- Startup people (Sam, Greg, Bob) found academic approach painful but necessary
- Early robotics project: taught humanoid robot hand to solve Rubik's Cube
- Idea: complicated environments would force AI to generalize beyond narrow domains
- Dota 2 project: solving video games as path to better AI
- Insight: scale was path to improving AI
- Could take huge amounts of experience, feed into neural network for learning/generalization
- Alec Radford experimenting with language and GPT-1
- Core idea: Transformer with simple objective of guessing next token would generate coherent text
- No one thought this would work at the time
- Alec persevered for years to make it work
- Combined ideas from Dota/robot hand (scale + diverse data) with GPT-1 → GPT-2, GPT-3, GPT-4
OpenAI's Unique Culture vs Competitors
- DeepMind: Demis had big plan, hired researchers to execute his vision
- Google Brain: "rebuild Academia" - bring talented researchers, give resources, let them figure out what to do
- OpenAI: startup approach with no centralized plan but had opinions about what needed to be done
- Focus on showing scale as way to improve ideas
- Leadership (Ilia, Dario) set opinions to avoid just throwing resources at everyone
- Found happy medium between centralized planning and complete freedom
Research Collaboration and Credit
- Academia has incentives preventing collaboration due to obsession with credit
- OpenAI made decision to be "Catholic" in putting everyone's name on papers
- Early robotics papers cited as "OpenAI" to avoid first author fights
- Channeled credit desire into internal reputation rather than paper position
- OpenAI had few titles early on but people knew who great researchers were
Scaling Laws Across Domains
- Scaling laws exist across many AI domains, not just LLMs
- First step is getting to scaling law (zero-to-one problem)
- Example: Aditya Ramesh spent 18 months on DALL-E getting first version to work
- Trying to make "pink panda skating on ice" (not in training set)
- Just getting to plausibly working version is huge difficult problem
- Once it works, scaling laws enable two approaches:
- Pure scale itself (systems/data/algorithmic problem)
- Change slope of scaling law (better architectures, optimization algorithms)
Data Wall and New Scaling Mechanisms
- Definitely hitting data wall for traditional pre-training techniques
- Shift from LLM pre-training scaling to reasoning techniques (O1, O3 models, Gemini Flash Thinking)
- Moore's Law analogy: sum of S-curves, solve one bottleneck then attack next
- Pre-training/data bottleneck being reached, but reasoning and test-time compute as new mechanism
- Clear path now exists for scaling - no longer in zero-to-one phase for LLMs, now in pure scaling regime
Five Levels of AGI and Scientific Innovation
- Reasoners are here, innovators coming next
- O3 or O3 Pro could be hooked up to biolab for autonomous scientific exploration
- Will likely have models that can explore scientific hypotheses and design experiments before having robots to run them
- Back to robotics as next S-curve bottleneck
Reasoning Models and Agents
- Reasoning = ability for model to have coherent chain of thought making progress over long periods
- Techniques for thinking harder also apply to taking action in real/virtual world
- Reasoning will unlock agents to do actions on user's behalf
- Always been possible but never good enough - need high reliability
- Rule of thumb: adding nine (90→99% or 99→99.9%) requires order of magnitude more compute
- Historically achieved through bigger models, now through longer thinking time
- Clear path to higher reliability levels unlocks many downstream applications
Distillation and Model Efficiency
- Big Frontier labs figured out tricks to take big models and train much smaller, faster models for specific distributions
- Trend across all labs: Sonnet vs Haiku, Gemini vs Gemini Flash, O1 vs O1 Mini, GPT-4 vs GPT-4 Mini
- Distillation as a service emerging
Advice for AI Startup Founders
- Start with very best model available
- Startup success requires exploiting something on AI frontier
- Get it working first, then use distillation/smaller models for cost optimization
- Most important resource is time - don't spend 3 years getting to market like Palantir
- Figure out value through user iteration, then optimize for cost
Personal AI and The Movie "Her"
- Skeptical of deep emotional AI girlfriend connections
- More promising: AI that shops for you, workplace assistant, personal AI bot representing you
- Needs huge amount of context: Slack, Gmail, all productivity tools
- Real market gap - can't purchase this today
- Vision: super intelligent genie that knows your goals, proactively suggests actions, handles appointments
AI Adoption Mystery
- By 2018, AI had capabilities people thought would automate all laptop jobs and end humanity
- None of that happened - AI adoption much slower than expected
- Effects visible in coding but not in productivity statistics (except data center construction)
- Deep mystery why adoption so slow relative to 2018 expectations
Palantir Lessons for AI Adoption
- Palantir discovered technology exists but not evenly distributed
- Found government agencies making critical decisions with no software
- Success came from reimagining workflows with technology, not just speeding up existing processes
- Example: instead of faster database searches, check all databases at once
- AI needs similar transformation - solve actual problems, not just automate existing workflows
- Need forward deployed engineers: sit with customers, understand their needs, build perfect software for them
- Initially skeptical of forward deployed model, now everyone building these functions
Parenting in AI Age
- Struggle with teaching 8-year-old son coding when AI will be able to code
- Using language models to create personalized coding lessons
- Believes in "resistance of the medium" - value in having hands-on experience
- Two future job roles: lone genius (like Alec Radford) and manager (CEO of mostly-AI firm)
- Analogy to photography: cameras didn't eliminate artists, created more art appreciation
- Historical precedent: 1880s farmers (90% of jobs) became 3% of population, new jobs emerged
- Optimistic about humanity having important roles, but jobs of grandchildren will be unrecognizable
Robotics Future Outlook
- Robotics companies now where LLM companies were 5 years ago
- Expects ChatGPT moment for robotics within next 5 years
- Harder to scale due to physical robot requirements
- Companies like Skild AI, Physical Intelligence building foundation models for robots
- Will transition from zero-to-one phase to scaling for reliability and market scope
Scientific Acceleration and Bottlenecks
- May automate scientists/innovators before experiment executors
- Potential for fast scientific advancement when combined with robotics
- Expects to find new bottlenecks - science may only go 30% faster instead of 300x faster
- Will need to figure out remaining constraints
