AI summary by TwinMind

Making AI accessible with Andrej Karpathy and Stephanie Zhan

Sequoia Capital

36m 58s

Speaker Introduction and Background

  • Andrej Karpathy introduced as final speaker at event
  • Background: designed first deep learning class at Stanford, founding team at OpenAI, led computer vision team at Tesla, recently left OpenAI
  • Event held at location across from OpenAI's original San Francisco office
  • Fun fact: OpenAI's first office was above chocolate factory, team was 10-20 people
  • Jensen Huang delivered first DGX to OpenAI at this location

Career Journey Overview

  • Trained by Jeff Hinton and Fei-Fei Li
  • Co-founded OpenAI in 2015, left in 2017 when poached by Elon Musk
  • Elon had cycled through 6 different autopilot leaders, each lasting 6 months
  • Returned to OpenAI for past year, now independent

Vision of AI Future: LLM OS Concept

  • AGI seemed impossible 7 years ago, now seems within sight
  • Everyone building what Karpathy calls "LLM OS" - operating system analogy
  • Components:
    • Peripherals: text, images, audio, all modalities
    • CPU: LLM Transformer itself
    • Connected to existing software 1.0 infrastructure
  • Goal: customizable agents for different economic niches
  • Future: multiple specialized agents for high-level tasks

OpenAI's Market Position and Opportunities

  • OpenAI dominating ecosystem, founders worried about being displaced
  • Operating system analogy: Windows comes with default apps (Edge browser) but allows third-party browsers
  • OpenAI will have default apps but vibrant ecosystem of specialized applications expected
  • Comparison to early iPhone app ecosystem - took time to develop, currently figuring out capabilities and limitations
  • Need to understand psychology of AI systems, oversight, evaluation methods

AI Model Ecosystem Landscape

  • Current race: OpenAI, Anthropic, Meta, Llama, Gemini, open source models
  • Operating system analogy: oligopoly of proprietary systems (Windows, macOS) plus Linux with infinite distributions
  • Important distinction in naming:
    • Open weights models (Llama, Mistral) - like getting binary for OS, useful but limited
    • Fully open source models (OLMo, LLM 360, Almo) - full infrastructure including training data
  • Open weights models have fine-tuning limitations - can't add capabilities without regression on existing ones
  • Need training loop and original dataset for proper capability addition

Scale and Other Success Factors

  • Scale is number one factor (first principal component)
  • Other critical ingredients:
    • Data curation and cleaning (compute efficiency gains)
    • Algorithms and model training expertise
    • Infrastructure challenges: training at scale extremely difficult
    • Distributed optimization problem across tens of thousands of GPUs
    • GPUs failing randomly, instrumentation challenges
    • Talent for large-scale training is scarce
  • Scale alone insufficient - need expertise in infrastructure, algorithms, data

Current Research Challenges

  • Algorithm side: unifying diffusion models and autoregressive models
    • Both represent probability distributions
    • Different modalities fit different approaches
    • Opportunity for hybrid architecture
  • Energy efficiency gap: massive disparity between brain (20 watts) and AI systems (megawatts)
    • Off by factor of 1000 to 1 million in efficiency
    • Three improvement levers:
      1. Precision reduction (64-bit to 4-6 bits, even 1.58 bits)
      2. Sparsity (brain not fully activated)
      3. Von Neumann architecture limitations (data movement between memory and compute)
  • Computer architecture innovation needed

Elon Musk's Management Philosophy

  • Unique company-running style, "biggest startups"
  • Key principles:
    • Small, strong, highly technical teams
    • Force against growth - had to plead to hire people
    • Default to removing low performers - had to fight to keep people
    • No non-technical middle management
    • Vibrant office culture - people pacing, working on exciting projects
    • Encourages leaving meetings if not contributing or learning
    • Less employee pampering, focus on best technical work
  • Direct CEO connection to engineers:
    • Talks directly to engineers, not just VPs/directors
    • 50% time with engineers vs typical 99% with VPs
    • Engineers have source of truth, not managers
  • Large hammer approach:
    • Removes bottlenecks directly
    • Example: GPU cluster doubling after hearing engineer complaints
    • Escalates to highest levels (calling Jensen Huang directly)
    • Daily updates until problems resolved
  • Advice: must have same DNA as founder, establish culture from start, be consistent

Personal Mission and Values

  • Cares about ecosystem health over any single company
  • Wants "coral reef" of startups in all economic niches
  • Envisions "boiling soup of cool stuff"
  • Concerned about 5 mega corps taking over with AGI as power magnifier
  • Prefers vibrant startup ecosystem

Model Composability and Development

  • Traditional code very composable, neural networks more fully connected
  • Composability exists through pre-training components and fine-tuning
  • Example: chat + images systems using pre-trained components
  • Parameter efficient training methods available
  • Scattered thoughts on current composability research

Path to Advanced AI Capabilities

  • Current models not capable enough for physicist/Von Neumann level reasoning
  • Two-step process needed (AlphaGo analogy):
    1. Imitation learning (current state)
    2. Reinforcement learning (missing step)
  • Problems with current training:
    • Human psychology differs from model psychology
    • Human solutions may be trivial for model in some parts, impossible leaps in others
    • Models need to practice solving problems themselves
    • Need to discover own capabilities and limitations
  • Reinforcement Learning from Human Feedback (RLHF) limitations:
    • "Vibe check" rather than objective function
    • Easy to overfit to reward model
    • Unlike AlphaGo's clear objective function
  • Need models to learn through self-practice, note-taking, knowledge manipulation

Practical AI Development Advice

  • Start with most capable model regardless of cost (GPT-4, advanced prompting, RAG)
  • Prioritize accuracy first, optimize cost later
  • Try multiple approaches (10-20 prompts, pick best, debate mechanisms)
  • Once working well, can distill to smaller, cheaper models
  • Run expensive system on problem distribution to get labels, fine-tune cheaper model

Open Source vs Closed Source Future

  • Capital-intensive nature of model training
  • Meta's unique position: can afford training but not core business
  • Incentive to release models to empower ecosystem and borrow ideas
  • Currently only releasing open weights, should go further
  • Data sourcing challenges may limit full open source releases
  • Meta doing well with transparency (published logbooks) but could do more

Making AI Ecosystem More Vibrant

  • Build the thing AND build the ramp
  • More people building things, fewer building educational ramps
  • Need more openness about what works/doesn't work
  • Share training experiences, effective usage patterns
  • Collaborative learning needed to understand effective AI usage

Future of Transformer Architecture

  • Transformer remarkably successful and unexpected
  • Expected neural network diversification, got convergence instead
  • Unlikely to be final architecture given field history
  • Opportunities for improvement:
    • Autoregressive vs diffusion modeling
    • Precision and sparsity optimizations
    • Hardware co-design
  • Transformer designed for GPU parallelization (major insight)
  • Broke sequential dependencies of RNNs through attention
  • Remarkably resilient over 6-7 years since original paper
  • Current transformers not super different from original

Final Advice to Founders

  • Care about individual startup success
  • Also consider ecosystem health
  • How can startups continue winning against big tech?
  • Focus on making entire ecosystem healthier and more vibrant

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome