AI summary by TwinMind
Fireworks Founder Lin Qiao on the Power of Small Models to Democratize AI Use Cases
Sequoia Capital
39m 18s
PyTorch Migration at Meta
- Meta's PyTorch framework migration initially estimated as 6-month project
- Actually took 5 years to support entire Meta AI workload
- Had to rebuild entire stack from scratch: data loading, distributed inference, training scaling
- At departure, Meta PyTorch stack sustained >5 trillion inference requests per day
- Fireworks mission: compress 5-year enterprise AI deployment timeline to 5 weeks or 5 days
Fireworks Company Overview
- Founded 2022 by Lin Qiao (former PyTorch lead at Meta) and team
- SaaS platform for generative AI inference and fine-tuning
- Focus on small model stack for low latency, low cost, automated customization
- Serves >100 models across text, image, audio, video, embedding, multimodal
PyTorch Strategy and Market Position
- Fireworks exclusively bets on PyTorch vs. framework-agnostic competitors
- PyTorch dominates research → flows downstream to production naturally
- Network effects: researchers prefer PyTorch simplicity → production teams follow
- Meta consolidated 3 frameworks (Caffe2 mobile, ONNX server, PyTorch research) into unified PyTorch stack
Customer Journey and Enterprise Adoption
- Pattern: customers start with OpenAI for experimentation/product-market fit
- Move to Fireworks when scaling for low latency + cost sustainability
- B2C applications require high responsiveness, can't wait 30 seconds for response
- Unexpected: traditional enterprises (banks, healthcare, insurance) adopting simultaneously with startups
- CTOs leading AI transformation initiatives vs. expected CIOs/CEOs
Open Source vs. Closed Source Models
- Prediction: open source and closed source model quality will converge at same model sizes (7B-70B parameters)
- Open source advantages: richer community, easier customization, better for fine-tuning
- Examples: Llama 3 excellent for instruction following, function calling
- Closed source providers focus on few models; open source enables "thousand flowers bloom"
Fine-Tuning and Customization Complexity
- Many domains work well with off-shelf models: coding, transcription, translation, OCR
- Business logic domains require fine-tuning: classification, summarization with specific templates
- Fine-tuning complexity: data collection, labeling, algorithm selection (supervised, DPO, preference-based)
- Parameter-efficient (LoRA) vs. full model fine-tuning decisions
- Fireworks automates technical complexity, leaves product design to customers
Function Calling and Knowledge Access Vision
- North star: simple API access to totality of knowledge
- Foundation models limited by training data cutoffs and internet-accessible information
- Function calling models route to appropriate APIs (public/private, model/non-model)
- Vision: mixture of experts architecture with hundreds of smaller, specialized experts
- Next-generation function calling model with "multiple layers of breakthroughs" coming soon
Hardware and Competition Landscape
- NVIDIA competition inevitable due to lucrative market economics
- Industry dislikes monopoly → pressure for alternatives
- AMD emerging in general-purpose GPU segment
- Custom silicon viable when AI model space stabilizes
- Competition timing: "coming soon"
Model Evolution and Scaling
- Returns to scale may plateau, but new model architectures will emerge
- Caution from Meta experience: seemingly stable model spaces can have breakthrough innovations
- Both stabilization and continued innovation will happen simultaneously
- Focus on customization and optimization while preparing for architectural shifts
OpenAI Competitive Positioning
- OpenAI focused on AGI mission vs. Fireworks' enterprise-specific problems
- Quality convergence prediction applies to OpenAI models too
- Real differentiation through automated customization for individual use cases
- OpenAI moving toward smaller, cheaper models aligns with Fireworks thesis
Rapid Fire Responses
- Favorite AI app: Video conferencing note-takers (Fathom)
- 2024 model predictions: Many high-performing models, continuous quality improvement
- Most admired: Meta's open source commitment with Llama series
- Agents outlook: Very bullish, expects significant growth
