AI summary by TwinMind
Founder Eric Steinberger on Magic’s Counterintuitive Approach to Pursuing AGI
Sequoia Capital
51m 15s
Eric's Background and Early AI Journey
- Born in Vienna, experienced "midlife crisis" at age 14 searching for meaningful work
- Explored physics, math, bio, medicine before discovering AI
- Felt decision paralysis resolved when finding AI - could see next 30 years unfold
- Started with no programming background, only wanted to solve AI
- Math naturally attractive but "not useful" - 17-dimensional spheres not best career choice
- AI seemed both achievable and "most important thing ever"
Path to Fair and Research Collaboration
- At 14, didn't know how to program, only wanted to solve AI
- After couple years warming up, reached out to David Silver's PhD student (Johannes)
- Asked for year-long mentorship - every two weeks "bashing" his work
- Sent giant multi-page email proposing to beat algorithm from Johannes's PhD
- Johannes mentored him brutally for a year while in high school
- Published "Single Deep Counterfactual Regret Minimization" that beat Noam Brown's similar work
- Took Eric couple years vs Noam's 3 months, but Eric's version slightly better
- Noam Brown reached out, led to 2-year collaboration at Fair
- Worked part-time as Fair researcher while studying university
Climate Science NGO Experience
- Started NGO while juggling Fair work and university - "too crazy"
- Dropped out after 3 months doing all three things
- NGO run like startup: clear objectives, software product, insane work hours
- 1000+ volunteers, 20 hardcore engineers, no money in/out
- Handed over leadership when realized AGI closer than expected
- "Wouldn't ever have started anything else if I thought AI was so close"
Research Philosophy and Advice
- Key traits: burning boats early, no Plan B mentality
- Write detailed 5-page emails to top researchers showing concrete plans
- Learn entire stack top-down: "what do I need, what do I need, what do I need"
- Read every paper voraciously - builds database in head for synthesis
- Bill Gates insight: "if you learned enough things they're all similar to each other"
- Never stop grinding - formative moment when Johannes almost gave up
- Be brutally honest about weaknesses, search for bad things
- Bet on strengths only if necessary conditions not missing
Research Superpowers and Admired Researchers
- Initially strong at synthesis: "this thing plus this thing plus that thing"
- Trying to improve at "leaps" - totally alien ideas with no reference
- Most good ideas are synthesis but best ideas have some leaps
- Admires Noam Sher most: best at synthesis, sees future, very intuitive
- Noam has larger database from being around longer, "literally knows everything"
- Geoffrey Hinton deserves most credit for persistence through "AI winter"
- Noam Brown: consistent 100x efficiency multipliers in RL, soloed problems that stumped 10-person lab teams
Current AI Challenges and AGI Timeline
- Most problems now solved: multimodal, long context, smart models, cost efficiency
- Last big unsolved problem: "General domain long horizon reliability"
- Need inference time compute - spend "not 1x or 2x or 10x but million x resources on that token"
- When proving theorems, writing software, essays - wouldn't write token by token
- Need to think hard about some tokens, find productive ways to spend massive resources
- Timeline: "very small number of years", "way less than 10", won't give specific number
Magic's Vision and Approach
- Mission: automate software engineering
- Subset goal: model that can build AGI (if great software engineer, can do everyone's job at Magic)
- Recursive improvement: clone team into computer, press "money button" to run cluster
- Two parallel tracks: ship good AI software engineer + automate themselves
- Software engineering as "MVP of AGI" - lever to automate everything else
- Don't want to train something like Sora - "not interesting from AGI perspective"
Why Train Custom Models vs APIs
- "To build the best model you need to build the model"
- Can't rely on API providers - if they solve it, why start company 3 years later?
- Took 2 years to learn large model training (same as OpenAI GPT-3 to GPT-4)
- Value will accrue at AGI level and hardware level, never application level
- No incentive to offer API if it creates $100B company - will build internally
- Much harder to build Netflix/Airbnb/Uber than chat interface
- API provider can 2x price and launch competing product, undercut everyone
Competing Against Giants
- Question: "how much does it cost to build AGI" not "how much money can you raise"
- If you can build AGI with available funding, having more won't get you there substantially sooner
- Need "sufficient hardware" not "more than sufficient"
- Bet: AGI buildable for less than $100B, within range of achievable funding
- Once have AGI, reasonably even playing field for additional revenue/compute
Team Scaling and Research Organization
- Early weakness: scaling team effectively, very single-threaded on small number
- Early estimate: 5 people, now closer to 20 (models + related work only)
- Could grow to "tens of people" but not hundreds
- Need maturity of codebase and research ideas to properly segment work
- More focused approach - don't build video models, separate teams would handle that
Agent Vision and Product Philosophy
- Three tiers: useless, micromanaged assistant, thing that manages you
- Goal: colleague experience like talking to best engineer
- User says what they want, agent clarifies, writes code, gets reviewed, works
- "I want to talk to my computer, go have lunch, come back and it built AGI"
- Need 99%+ reliability, not 95% - "I don't want to review code"
- Difference between tiers where you review vs don't review code is fundamental
Long Context Evaluation (Hashes)
- Frustrated by needle-in-haystack eval limitations
- Needle-in-haystack: find weird thing in pool of normal stuff
- Only need to "walk with backpack" from start to end, find weird thing
- Don't actually need to remember whole context window
- Hashes eval: replace everything with random noise (randomly generated letters)
- No semantic information, forces model to know whole context
- Add "hop" dimension: find one thing, then find another thing
- Two key dimensions: context length and reasoning hops
- Open sourced to let others compare architectures and results
