AI summary by TwinMind
What's next for photonics-powered data centers and AI ft. Lightmatter's Nick Harris
Sequoia Capital
5m 56s
Introduction and Company Overview
- Nick Harris, CEO of Lightmatter, introduced at SEOA event
- Lightmatter building very large data centers with hundreds of thousands of GPUs, potentially millions of nodes
- Focus on powering hyperscalers' next generation AI models
- Company using light-based technology for data center scaling
AI Scaling Laws and Compute Requirements
- AI progress driven by scaling laws and training large foundation models
- Demonstrated with Sora example: scaling compute transformed "goofy munge of furry thing" to realistic physics of dog with hat, person with flowing hair
- Amount of compute directly impacts AI model power and capabilities
Data Center Capital Expenditure Analysis
- Supercomputer costs for AI training:
- 4,000 GPUs: ~$150 million
- 10,000 GPUs: ~$400 million
- 60,000 GPUs: ~$4 billion
- Future projections: $10-20 billion systems coming
- Enormous pressure on companies to deliver ROI on astronomical investments
- AGI potentially achievable with sufficient spending
End of Traditional Scaling
- Traditional computer chip scaling is over
- Jensen Huang's GTC announcement: chip twice as big for twice performance
- Moore's Law and Dennard scaling ending
- Core technology that democratized computing and enabled AGI hunt coming to end
- Need for fundamentally new technologies
Current Data Center Limitations
- Present supercomputers: separate racks for networking gear and computing gear
- Strong interconnections within computing racks, but weak "spaghetti" connections between racks
- Weak interconnectivity limits performance scaling
- AI training workloads must be sliced/diced to fit tightly interconnected clusters
- Difficulty achieving good unit performance scaling at 50,000+ GPUs
- 1,000 GPUs performance depends heavily on wiring configuration
Lightmatter's Solution: Passage Technology
- Vision: Delete all networking racks from data centers
- Scale compute 100x larger
- Replace spaghetti connections with all-to-all interconnect
- Core technology called "Passage" - optical interconnect substrate
- Partners: AMD, Intel, Nvidia, Qualcomm
- Client chips placed on top of Lightmatter's optical interconnect substrate
- Foundation for future AI computing progress
Benefits and Future Impact
- Dramatically reduce energy consumption of clusters
- Enable scaling to million nodes and beyond
- Path to wafer-scale computing with biggest chips in the world
- Critical technology for achieving AGI
