Training Regime Parallelization Evolution

• Training compute shifting toward synthetic data generation and search processes
• Post-training techniques emerging to utilize available compute capacity
• Multi-data center training becoming viable solution for scale

Microsoft-OpenAI Infrastructure Expansion

• Microsoft signed $10+ billion deals with fiber companies (Lumen, Zayo, others)
• Permits filed for inter-data center connections
• 5 regions being connected comprising multiple data centers each
• Total power consumption: north of 1 gigawatt (close to 1 million GPUs)

GPU Power Consumption Scaling

• H100 GPU: 700W base, 1,200-1,400W total system power
• Next-gen Nvidia GPUs: 1,200W base, 2,000W total system power
• Power per GPU increasing with each generation

Current and Planned Cluster Deployments

• Existing 100K H100 clusters:
  • OpenAI in Arizona
  • xAI in Memphis
  • Multiple others under construction
• Planned GB200 clusters:
  • 5+ planned 100K GB200 clusters by Microsoft/OpenAI partners
  • Potential 500K GB200 cluster (1 gigawatt capacity)
  • Timeline: online next year

2025-2026 Scaling Projections

• 2025: 100K GPUs largest current cluster
• Next year: 300K-500K GPUs per cluster (up to 700K upper bound)
• GB200s are 2-3x faster than H100s
• H100-equivalent basis: 1 million chips next year in single cluster
• Multi-site efficiency loss: 20-50% (exact figure unknown)

Long-term Infrastructure Plans

• 2026: Single 1-gigawatt site planned
• Microsoft 5-gigawatt total capacity across multiple locations
• Each location: multiple sites, some single sites
• Total capacity: easily north of 2-3 gigawatts
• Integration of old chips with new chips under consideration

Financial Requirements

• OpenAI needs $50-100 billion funding for planned cluster scale
• Expected fundraising: end of 2024/early 2025
• Sam Altman's strengths: recruiting and fundraising capabilities

Competitive Landscape

• Anthropic has similar plans with Amazon
• Multiple providers involved: QTS, Cooper, others
• Self-built data centers also being constructed
• Old crypto mining sites being repurposed (CoreWeave Texas, Oracle Crusoe Texas)
• Locations: Wisconsin, Arizona, Texas, others