AI summary by TwinMind

Founder Eric Steinberger on Magic’s Counterintuitive Approach to Pursuing AGI

Sequoia Capital

51m 15s

Eric's Background and Early AI Journey

  • Born in Vienna, experienced "midlife crisis" at age 14 searching for meaningful work
  • Explored physics, math, bio, medicine before discovering AI
  • Felt decision paralysis resolved when finding AI - could see next 30 years unfold
  • Started with no programming background, only wanted to solve AI
  • Math naturally attractive but "not useful" - 17-dimensional spheres not best career choice
  • AI seemed both achievable and "most important thing ever"

Path to Fair and Research Collaboration

  • At 14, didn't know how to program, only wanted to solve AI
  • After couple years warming up, reached out to David Silver's PhD student (Johannes)
  • Asked for year-long mentorship - every two weeks "bashing" his work
  • Sent giant multi-page email proposing to beat algorithm from Johannes's PhD
  • Johannes mentored him brutally for a year while in high school
  • Published "Single Deep Counterfactual Regret Minimization" that beat Noam Brown's similar work
  • Took Eric couple years vs Noam's 3 months, but Eric's version slightly better
  • Noam Brown reached out, led to 2-year collaboration at Fair
  • Worked part-time as Fair researcher while studying university

Climate Science NGO Experience

  • Started NGO while juggling Fair work and university - "too crazy"
  • Dropped out after 3 months doing all three things
  • NGO run like startup: clear objectives, software product, insane work hours
  • 1000+ volunteers, 20 hardcore engineers, no money in/out
  • Handed over leadership when realized AGI closer than expected
  • "Wouldn't ever have started anything else if I thought AI was so close"

Research Philosophy and Advice

  • Key traits: burning boats early, no Plan B mentality
  • Write detailed 5-page emails to top researchers showing concrete plans
  • Learn entire stack top-down: "what do I need, what do I need, what do I need"
  • Read every paper voraciously - builds database in head for synthesis
  • Bill Gates insight: "if you learned enough things they're all similar to each other"
  • Never stop grinding - formative moment when Johannes almost gave up
  • Be brutally honest about weaknesses, search for bad things
  • Bet on strengths only if necessary conditions not missing

Research Superpowers and Admired Researchers

  • Initially strong at synthesis: "this thing plus this thing plus that thing"
  • Trying to improve at "leaps" - totally alien ideas with no reference
  • Most good ideas are synthesis but best ideas have some leaps
  • Admires Noam Sher most: best at synthesis, sees future, very intuitive
  • Noam has larger database from being around longer, "literally knows everything"
  • Geoffrey Hinton deserves most credit for persistence through "AI winter"
  • Noam Brown: consistent 100x efficiency multipliers in RL, soloed problems that stumped 10-person lab teams

Current AI Challenges and AGI Timeline

  • Most problems now solved: multimodal, long context, smart models, cost efficiency
  • Last big unsolved problem: "General domain long horizon reliability"
  • Need inference time compute - spend "not 1x or 2x or 10x but million x resources on that token"
  • When proving theorems, writing software, essays - wouldn't write token by token
  • Need to think hard about some tokens, find productive ways to spend massive resources
  • Timeline: "very small number of years", "way less than 10", won't give specific number

Magic's Vision and Approach

  • Mission: automate software engineering
  • Subset goal: model that can build AGI (if great software engineer, can do everyone's job at Magic)
  • Recursive improvement: clone team into computer, press "money button" to run cluster
  • Two parallel tracks: ship good AI software engineer + automate themselves
  • Software engineering as "MVP of AGI" - lever to automate everything else
  • Don't want to train something like Sora - "not interesting from AGI perspective"

Why Train Custom Models vs APIs

  • "To build the best model you need to build the model"
  • Can't rely on API providers - if they solve it, why start company 3 years later?
  • Took 2 years to learn large model training (same as OpenAI GPT-3 to GPT-4)
  • Value will accrue at AGI level and hardware level, never application level
  • No incentive to offer API if it creates $100B company - will build internally
  • Much harder to build Netflix/Airbnb/Uber than chat interface
  • API provider can 2x price and launch competing product, undercut everyone

Competing Against Giants

  • Question: "how much does it cost to build AGI" not "how much money can you raise"
  • If you can build AGI with available funding, having more won't get you there substantially sooner
  • Need "sufficient hardware" not "more than sufficient"
  • Bet: AGI buildable for less than $100B, within range of achievable funding
  • Once have AGI, reasonably even playing field for additional revenue/compute

Team Scaling and Research Organization

  • Early weakness: scaling team effectively, very single-threaded on small number
  • Early estimate: 5 people, now closer to 20 (models + related work only)
  • Could grow to "tens of people" but not hundreds
  • Need maturity of codebase and research ideas to properly segment work
  • More focused approach - don't build video models, separate teams would handle that

Agent Vision and Product Philosophy

  • Three tiers: useless, micromanaged assistant, thing that manages you
  • Goal: colleague experience like talking to best engineer
  • User says what they want, agent clarifies, writes code, gets reviewed, works
  • "I want to talk to my computer, go have lunch, come back and it built AGI"
  • Need 99%+ reliability, not 95% - "I don't want to review code"
  • Difference between tiers where you review vs don't review code is fundamental

Long Context Evaluation (Hashes)

  • Frustrated by needle-in-haystack eval limitations
  • Needle-in-haystack: find weird thing in pool of normal stuff
  • Only need to "walk with backpack" from start to end, find weird thing
  • Don't actually need to remember whole context window
  • Hashes eval: replace everything with random noise (randomly generated letters)
  • No semantic information, forces model to know whole context
  • Add "hop" dimension: find one thing, then find another thing
  • Two key dimensions: context length and reasoning hops
  • Open sourced to let others compare architectures and results

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome