AI summary by TwinMind
Making AI accessible with Andrej Karpathy and Stephanie Zhan
Sequoia Capital
36m 58s
Speaker Introduction and Background
- Andrej Karpathy introduced as final speaker at event
- Background: designed first deep learning class at Stanford, founding team at OpenAI, led computer vision team at Tesla, recently left OpenAI
- Event held at location across from OpenAI's original San Francisco office
- Fun fact: OpenAI's first office was above chocolate factory, team was 10-20 people
- Jensen Huang delivered first DGX to OpenAI at this location
Career Journey Overview
- Trained by Jeff Hinton and Fei-Fei Li
- Co-founded OpenAI in 2015, left in 2017 when poached by Elon Musk
- Elon had cycled through 6 different autopilot leaders, each lasting 6 months
- Returned to OpenAI for past year, now independent
Vision of AI Future: LLM OS Concept
- AGI seemed impossible 7 years ago, now seems within sight
- Everyone building what Karpathy calls "LLM OS" - operating system analogy
- Components:
- Peripherals: text, images, audio, all modalities
- CPU: LLM Transformer itself
- Connected to existing software 1.0 infrastructure
- Goal: customizable agents for different economic niches
- Future: multiple specialized agents for high-level tasks
OpenAI's Market Position and Opportunities
- OpenAI dominating ecosystem, founders worried about being displaced
- Operating system analogy: Windows comes with default apps (Edge browser) but allows third-party browsers
- OpenAI will have default apps but vibrant ecosystem of specialized applications expected
- Comparison to early iPhone app ecosystem - took time to develop, currently figuring out capabilities and limitations
- Need to understand psychology of AI systems, oversight, evaluation methods
AI Model Ecosystem Landscape
- Current race: OpenAI, Anthropic, Meta, Llama, Gemini, open source models
- Operating system analogy: oligopoly of proprietary systems (Windows, macOS) plus Linux with infinite distributions
- Important distinction in naming:
- Open weights models (Llama, Mistral) - like getting binary for OS, useful but limited
- Fully open source models (OLMo, LLM 360, Almo) - full infrastructure including training data
- Open weights models have fine-tuning limitations - can't add capabilities without regression on existing ones
- Need training loop and original dataset for proper capability addition
Scale and Other Success Factors
- Scale is number one factor (first principal component)
- Other critical ingredients:
- Data curation and cleaning (compute efficiency gains)
- Algorithms and model training expertise
- Infrastructure challenges: training at scale extremely difficult
- Distributed optimization problem across tens of thousands of GPUs
- GPUs failing randomly, instrumentation challenges
- Talent for large-scale training is scarce
- Scale alone insufficient - need expertise in infrastructure, algorithms, data
Current Research Challenges
- Algorithm side: unifying diffusion models and autoregressive models
- Both represent probability distributions
- Different modalities fit different approaches
- Opportunity for hybrid architecture
- Energy efficiency gap: massive disparity between brain (20 watts) and AI systems (megawatts)
- Off by factor of 1000 to 1 million in efficiency
- Three improvement levers:
- Precision reduction (64-bit to 4-6 bits, even 1.58 bits)
- Sparsity (brain not fully activated)
- Von Neumann architecture limitations (data movement between memory and compute)
- Computer architecture innovation needed
Elon Musk's Management Philosophy
- Unique company-running style, "biggest startups"
- Key principles:
- Small, strong, highly technical teams
- Force against growth - had to plead to hire people
- Default to removing low performers - had to fight to keep people
- No non-technical middle management
- Vibrant office culture - people pacing, working on exciting projects
- Encourages leaving meetings if not contributing or learning
- Less employee pampering, focus on best technical work
- Direct CEO connection to engineers:
- Talks directly to engineers, not just VPs/directors
- 50% time with engineers vs typical 99% with VPs
- Engineers have source of truth, not managers
- Large hammer approach:
- Removes bottlenecks directly
- Example: GPU cluster doubling after hearing engineer complaints
- Escalates to highest levels (calling Jensen Huang directly)
- Daily updates until problems resolved
- Advice: must have same DNA as founder, establish culture from start, be consistent
Personal Mission and Values
- Cares about ecosystem health over any single company
- Wants "coral reef" of startups in all economic niches
- Envisions "boiling soup of cool stuff"
- Concerned about 5 mega corps taking over with AGI as power magnifier
- Prefers vibrant startup ecosystem
Model Composability and Development
- Traditional code very composable, neural networks more fully connected
- Composability exists through pre-training components and fine-tuning
- Example: chat + images systems using pre-trained components
- Parameter efficient training methods available
- Scattered thoughts on current composability research
Path to Advanced AI Capabilities
- Current models not capable enough for physicist/Von Neumann level reasoning
- Two-step process needed (AlphaGo analogy):
- Imitation learning (current state)
- Reinforcement learning (missing step)
- Problems with current training:
- Human psychology differs from model psychology
- Human solutions may be trivial for model in some parts, impossible leaps in others
- Models need to practice solving problems themselves
- Need to discover own capabilities and limitations
- Reinforcement Learning from Human Feedback (RLHF) limitations:
- "Vibe check" rather than objective function
- Easy to overfit to reward model
- Unlike AlphaGo's clear objective function
- Need models to learn through self-practice, note-taking, knowledge manipulation
Practical AI Development Advice
- Start with most capable model regardless of cost (GPT-4, advanced prompting, RAG)
- Prioritize accuracy first, optimize cost later
- Try multiple approaches (10-20 prompts, pick best, debate mechanisms)
- Once working well, can distill to smaller, cheaper models
- Run expensive system on problem distribution to get labels, fine-tune cheaper model
Open Source vs Closed Source Future
- Capital-intensive nature of model training
- Meta's unique position: can afford training but not core business
- Incentive to release models to empower ecosystem and borrow ideas
- Currently only releasing open weights, should go further
- Data sourcing challenges may limit full open source releases
- Meta doing well with transparency (published logbooks) but could do more
Making AI Ecosystem More Vibrant
- Build the thing AND build the ramp
- More people building things, fewer building educational ramps
- Need more openness about what works/doesn't work
- Share training experiences, effective usage patterns
- Collaborative learning needed to understand effective AI usage
Future of Transformer Architecture
- Transformer remarkably successful and unexpected
- Expected neural network diversification, got convergence instead
- Unlikely to be final architecture given field history
- Opportunities for improvement:
- Autoregressive vs diffusion modeling
- Precision and sparsity optimizations
- Hardware co-design
- Transformer designed for GPU parallelization (major insight)
- Broke sequential dependencies of RNNs through attention
- Remarkably resilient over 6-7 years since original paper
- Current transformers not super different from original
Final Advice to Founders
- Care about individual startup success
- Also consider ecosystem health
- How can startups continue winning against big tech?
- Focus on making entire ecosystem healthier and more vibrant
