AI summary by TwinMind
Will Open Source AI Overtake Closed Models? Ft. Olama, Fireworks and Open Router
Sequoia Capital
19m 48s
Panel Introductions
- Jeff (Ollama): Founder, easiest way to run open-source LLMs
- Dimma (Fireworks): Co-founder, platform for fine-tuning and serving models at scale, big proponents of open source
- Alex (OpenRouter): CEO/co-founder, language model router and marketplace for best pricing/performance/uptime, works closely with Fireworks
Case for Open Source Models Against Regulation
- Jeff's argument: Human genius comes from anywhere globally, centralizing behind model labs is risky
- Open source allows unique intelligence to create on-demand services
- Currently open source is 20-30% of inference tokens vs closed source
- Often used as last-mile optimization or for customers refusing closed source
- Young developers will default to open source before closed source
- Alex's practical perspective:
- Models deployed on non-data center hardware, customers get access anyway
- Enterprises want to own entire fine-tuned models with their data, not just pieces
- Dimma's technology view: AI as general purpose technology like electricity
- Can regulate applications/distribution but shouldn't ban basic technology
- Customization critical for AI-first products
Current State: Open Source Behind Closed Source
- Alex's analysis: Open models currently losing to closed source
- Large labs building more general foundation models requiring less fine-tuning
- Predicts fine-tuning may have "weak moment" in next year as RL models scale
- Marginal fine-tuning benefits outweighed by obsolescence risk
- Eventually expects transparency/customizability demand like software evolution
Open Source Model Leadership Discussion
- Leadership changes: Meta (Llama), Mistral previously, now DeepSeek leading
- Common pattern: Companies solving problems beyond just selling models
- Meta: social network applications
- Chinese companies: next great consumer apps
- Open sourcing along path to other goals gets more feedback
- Magic of open source: Only need one player releasing good model to build upon
DeepSeek Success Analysis
- Technical factors: Really good engineering, small teams moving fast, integrated research/engineering
- Three key success elements:
- First good open source reasoning model
- UX innovation showing model thoughts (unlike OpenAI's o1)
- Couldn't handle inference demand, forced American companies to scale up
- Fireworks role: Captured most early inference, enabled American company adoption
- Infrastructure challenges: First successful open source mixture of experts model requiring complex multi-GPU/multi-server distributed inference
DeepSeek Model Optimization & Customization
- Scaling challenges: Mixture of experts models require sophisticated distributed inference
- Growth pattern: Crazy growth from Christmas day release, few weeks for market realization
- Fine-tuning applications: Perplexity and others fine-tuned for specific behaviors
- Business use cases: Sensitive elements don't apply to narrow business applications
Llama 4 Prospects
- Positive ingredients: High-level commitment, amazing people at Meta, lots of compute
- Current release issues: Benchmarking problems, missing smaller model variants
- Llama 3 releases: Behemoth, Scout, Maverick models, plus upcoming "Little Llama"
- DeepSeek advantage: Massive model + cheap hosting + distillations
- Learning rate: Llama 4 incorporating learnings, addressing reward hacking and leaderboard gaming
Future Predictions: 5-Year Inference Token Split
- Alex: 50/50 split
- Route LLM research: switching between open source and big models
- Stanford Minions project: using models in combination
- Routing layer developments
- Dimma: 50/50 split
- Closed source: few leading players, quarter through single company
- Open source: diverse families, variations, customizations
- Shift from pre-training to post-training/RL favors customization
- Less capital investment for good results with unique data
- Jeff: Depends on decentralized providers sustainability
- If decentralization fails: closed source majority
- If decentralization succeeds: changes dynamics completely
- Current decentralized players earning 360k/day in incentives, sustainability questioned
