AI summary by TwinMind
Jim Fan on Nvidia’s Embodied AI Lab and Jensen Huang’s Prediction that All Robots will be Autonomous
Sequoia Capital
49m 14s
Jim Fan's Background and Journey to Embodied AI
- First intern at OpenAI (summer 2016)
- Mentored by Andrej Karpathy and Ilya Sutskever
- Worked on "World of Bits" project - AI agent reading computer screens, controlling keyboard/mouse
- Part of larger OpenAI Universe initiative
- Used reinforcement learning (no Transformers existed in 2016)
- Challenge: worked on specific tasks but didn't generalize to arbitrary language instructions
- Stanford PhD (2016-2021) with Fei-Fei Li
- Witnessed transition from static computer vision to embodied computer vision
- Agents learning perception and taking actions in interactive environments (virtual simulation or physical world)
- Joined Nvidia after PhD graduation, continuing embodied AI work
Nvidia's Embodied AI Initiative - GEAR Team
- GEAR = Generalist Embodied Agent Research
- Mission: "generate actions" - build embodied AI agents taking actions in different worlds
- Virtual world actions = gaming AI and simulation
- Physical world actions = robotics
- Project Groot unveiled at GTC March 2024
- Nvidia's moonshot effort building foundation models for humanoid robotics
- Goal: AI brain for humanoid robots and beyond
Nvidia's Competitive Advantages in Robotics
- Compute resources for scaling foundation models
- Belief in scaling laws for embodied AI and robotics (yet to be studied)
- Simulation expertise
- Nvidia's graphics company heritage - years of physics simulation, rendering, real-time GPU acceleration
- Simulation heavily used in robotics approach
Three-Bucket Data Strategy for Robot Foundation Models
- Internet-scale data: Text and videos online
- Most diverse, encode common sense prior
- Human-centered content (selfies, instructional videos)
- Learn human-object interactions and object behaviors
- Weakness: no action/motor control signals available
- Simulation data: Nvidia simulation tools generating synthetic data
- Provides actions and consequences of actions
- Infinite data scaling with compute
- 10,000x real-time acceleration possible
- Weakness: sim-to-real gap, physics/visuals differences, limited diversity
- Real robot data: Teleoperated robot data collection
- No sim-to-real gap
- Weakness: expensive, limited by 24 hours/day, requires human operators
- Strategy: Combine complementary strengths, remove individual weaknesses
Vision for Robotics Breakthrough Timeline
- Research breakthrough in robot foundation models: 2-3 years ("GPT-3 moment for robotics")
- Mass market adoption: longer timeline due to affordability, mass production, safety, privacy, regulations
Defining the "GPT-3 Moment" for Robotics
- System 1 vs System 2 thinking (from "Thinking Fast and Slow")
- System 1: Low-level motor control, unconscious, fast (e.g., grasping cup)
- System 2: Slow, deliberate reasoning and planning
- GPT-3 moment will be System 1 breakthrough
- Example: Understanding verb "open" across different contexts
- Opening door vs window vs bottle vs phone - different motions, same concept
- Need model generalizing low-level motor control for abstract verbs
- System 2 already strong with existing LLMs (reasoning, planning, coding)
- Integration challenge: System 1 operates at 1000Hz, System 2 at 1Hz
- Open question: Monolithic model vs cascaded approach with communication between systems
Scaling Law Hypothesis for Embodied AI
- Haven't pushed Transformer limits yet for robotics
- Transformers fundamentally take tokens in, output tokens
- Quality of tokens determines model quality
- Complex robotics data strategy (internet + simulation + real robot data)
- Once high-quality action data tokenized and scaled, expect emerging properties
- Optimistic about Transformer-based scaling approach
Humanoid Robot Focus Rationale
- World design: Everything built for human form factor
- Restaurants, factories, hospitals, equipment, tools designed for human hands/body
- Sufficiently good humanoid hardware can support any reasonable human task
- Data availability: Abundant human-centered online content
- Videos of humans doing daily tasks, having fun
- Humanoid form factor closest to human - easier transfer learning
- Few videos of robot arms/grippers online vs many videos of five-finger hands
- Generality first: Train for full generality, then specialize to robot arms/specific forms
Current Robotics Landscape and "Why Now"
- Hardware improvements: Surge of new robot hardware since late 2023
- Tesla Optimus, Boston Dynamics, many startups
- Better dexterous hands, whole-body reliability
- Pricing revolution:
- 2001: NASA humanoid robot >$1.5M
- Recent: $30,000 full-fledged humanoids (comparable to car price)
- Raw materials only 4% of car cost - potential for exponential price decrease
- Foundation model advances:
- System 2 reasoning/planning addressed by frontier models (GPTs, Claude, Llama)
- Multimodal models improving computer vision/perception
- Coding abilities leveraged for robot solutions (Eureka project)
Eureka and Dr. Eureka Research Projects
- Eureka: LLM writes reward function code in Isaac Sim API
- Automates tedious manual reward function engineering
- Demonstrated with five-finger robot hand pen spinning (superhuman performance)
- General technique for complex task specification
- Dr. Eureka: Extension using domain randomization
- LLM writes code specifying simulation randomization parameters
- Trains across 10,000 different simulation configurations
- Zero-shot transfer to real world without fine-tuning
- Demo: Robot dog walking on yoga ball (better than real dogs)
Virtual Worlds and Gaming AI Research
- MineDojo: Platform for general-purpose Minecraft agents
- Collected internet data: gameplay videos, Wiki pages, Reddit discussions
- Multimodal datasets for training Minecraft-playing models
- Voyager: GPT-4 powered Minecraft agent using code as action
- Converts 3D Minecraft world to text representation
- Agent writes code using action APIs
- Self-reflection loop for error correction
- Skill library saves successful programs for reuse
- Automated curriculum - agent proposes next appropriate difficulty tasks
- Emerging property: discovers novel items/tools autonomously
Foundation Agent Vision
- Single model working across virtual and physical agents
- Three generalization axes:
- Skills: Different capabilities
- Embodiments: Different body forms/platforms
- Worlds: Different realities (virtual/physical)
- Physical world as "10,001st virtual simulation" in domain randomization approach
Gaming AI Future Vision
- Intelligent NPCs:
- Non-player characters with memory, personality, dynamic behavior
- Interactive conversations changing narrative/story
- Infinite replay value with unique experiences
- Generated game worlds:
- Text-to-3D asset generation
- Text-to-video models
- Language agents for storylines
- Real-time world generation during gameplay
- Current models (Llama-level) not quite sufficient - need more diverse, engaging behaviors
- Inference cost challenge for deployment (cloud hosting vs local device)
Alternative Architectures and Future Directions
- Haven't pushed Transformer limits for robotics yet
- Data remains primary bottleneck (action data not downloadable from internet)
- Interested in alternatives: Mamba, test-time training
- More efficient inference mechanisms
- Need scaling to frontier model sizes for proper comparison
- Video generation as world simulator
- Learning physics and rendering from data alone
- Data-driven simulation for embodied AI training
Long-term AI Vision (10+ years)
- Coding agents: Human-level software engineers accelerating development
- Humanoid robots: Reliability and agility matching/exceeding humans
- Economic impact: Robots in households (laundry, cooking, elderly care), restaurants, hospitals, factories
- Jensen's quote: "Everything that moves will eventually be autonomous"
- Vision: As many intelligent robots as iPhones within 10+ years
Research Philosophy and Advice
- Importance of identifying right problems vs solving problems
- Keep up with accessible research literature
- Try open-source tools and simulators
- Get hands dirty with practical experimentation
- Long-term vision thinking - work on future needs today
