AI summary by TwinMind

What's next for photonics-powered data centers and AI ft. Lightmatter's Nick Harris

Sequoia Capital

5m 56s

Introduction and Company Overview

  • Nick Harris, CEO of Lightmatter, introduced at SEOA event
  • Lightmatter building very large data centers with hundreds of thousands of GPUs, potentially millions of nodes
  • Focus on powering hyperscalers' next generation AI models
  • Company using light-based technology for data center scaling

AI Scaling Laws and Compute Requirements

  • AI progress driven by scaling laws and training large foundation models
  • Demonstrated with Sora example: scaling compute transformed "goofy munge of furry thing" to realistic physics of dog with hat, person with flowing hair
  • Amount of compute directly impacts AI model power and capabilities

Data Center Capital Expenditure Analysis

  • Supercomputer costs for AI training:
    • 4,000 GPUs: ~$150 million
    • 10,000 GPUs: ~$400 million
    • 60,000 GPUs: ~$4 billion
  • Future projections: $10-20 billion systems coming
  • Enormous pressure on companies to deliver ROI on astronomical investments
  • AGI potentially achievable with sufficient spending

End of Traditional Scaling

  • Traditional computer chip scaling is over
  • Jensen Huang's GTC announcement: chip twice as big for twice performance
  • Moore's Law and Dennard scaling ending
  • Core technology that democratized computing and enabled AGI hunt coming to end
  • Need for fundamentally new technologies

Current Data Center Limitations

  • Present supercomputers: separate racks for networking gear and computing gear
  • Strong interconnections within computing racks, but weak "spaghetti" connections between racks
  • Weak interconnectivity limits performance scaling
  • AI training workloads must be sliced/diced to fit tightly interconnected clusters
  • Difficulty achieving good unit performance scaling at 50,000+ GPUs
  • 1,000 GPUs performance depends heavily on wiring configuration

Lightmatter's Solution: Passage Technology

  • Vision: Delete all networking racks from data centers
  • Scale compute 100x larger
  • Replace spaghetti connections with all-to-all interconnect
  • Core technology called "Passage" - optical interconnect substrate
  • Partners: AMD, Intel, Nvidia, Qualcomm
  • Client chips placed on top of Lightmatter's optical interconnect substrate
  • Foundation for future AI computing progress

Benefits and Future Impact

  • Dramatically reduce energy consumption of clusters
  • Enable scaling to million nodes and beyond
  • Path to wafer-scale computing with biggest chips in the world
  • Critical technology for achieving AGI

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome