AI summary by TwinMind

Fireworks Founder Lin Qiao on the Power of Small Models to Democratize AI Use Cases

Sequoia Capital

39m 18s

PyTorch Migration at Meta

  • Meta's PyTorch framework migration initially estimated as 6-month project
  • Actually took 5 years to support entire Meta AI workload
  • Had to rebuild entire stack from scratch: data loading, distributed inference, training scaling
  • At departure, Meta PyTorch stack sustained >5 trillion inference requests per day
  • Fireworks mission: compress 5-year enterprise AI deployment timeline to 5 weeks or 5 days

Fireworks Company Overview

  • Founded 2022 by Lin Qiao (former PyTorch lead at Meta) and team
  • SaaS platform for generative AI inference and fine-tuning
  • Focus on small model stack for low latency, low cost, automated customization
  • Serves >100 models across text, image, audio, video, embedding, multimodal

PyTorch Strategy and Market Position

  • Fireworks exclusively bets on PyTorch vs. framework-agnostic competitors
  • PyTorch dominates research → flows downstream to production naturally
  • Network effects: researchers prefer PyTorch simplicity → production teams follow
  • Meta consolidated 3 frameworks (Caffe2 mobile, ONNX server, PyTorch research) into unified PyTorch stack

Customer Journey and Enterprise Adoption

  • Pattern: customers start with OpenAI for experimentation/product-market fit
  • Move to Fireworks when scaling for low latency + cost sustainability
  • B2C applications require high responsiveness, can't wait 30 seconds for response
  • Unexpected: traditional enterprises (banks, healthcare, insurance) adopting simultaneously with startups
  • CTOs leading AI transformation initiatives vs. expected CIOs/CEOs

Open Source vs. Closed Source Models

  • Prediction: open source and closed source model quality will converge at same model sizes (7B-70B parameters)
  • Open source advantages: richer community, easier customization, better for fine-tuning
  • Examples: Llama 3 excellent for instruction following, function calling
  • Closed source providers focus on few models; open source enables "thousand flowers bloom"

Fine-Tuning and Customization Complexity

  • Many domains work well with off-shelf models: coding, transcription, translation, OCR
  • Business logic domains require fine-tuning: classification, summarization with specific templates
  • Fine-tuning complexity: data collection, labeling, algorithm selection (supervised, DPO, preference-based)
  • Parameter-efficient (LoRA) vs. full model fine-tuning decisions
  • Fireworks automates technical complexity, leaves product design to customers

Function Calling and Knowledge Access Vision

  • North star: simple API access to totality of knowledge
  • Foundation models limited by training data cutoffs and internet-accessible information
  • Function calling models route to appropriate APIs (public/private, model/non-model)
  • Vision: mixture of experts architecture with hundreds of smaller, specialized experts
  • Next-generation function calling model with "multiple layers of breakthroughs" coming soon

Hardware and Competition Landscape

  • NVIDIA competition inevitable due to lucrative market economics
  • Industry dislikes monopoly → pressure for alternatives
  • AMD emerging in general-purpose GPU segment
  • Custom silicon viable when AI model space stabilizes
  • Competition timing: "coming soon"

Model Evolution and Scaling

  • Returns to scale may plateau, but new model architectures will emerge
  • Caution from Meta experience: seemingly stable model spaces can have breakthrough innovations
  • Both stabilization and continued innovation will happen simultaneously
  • Focus on customization and optimization while preparing for architectural shifts

OpenAI Competitive Positioning

  • OpenAI focused on AGI mission vs. Fireworks' enterprise-specific problems
  • Quality convergence prediction applies to OpenAI models too
  • Real differentiation through automated customization for individual use cases
  • OpenAI moving toward smaller, cheaper models aligns with Fireworks thesis

Rapid Fire Responses

  • Favorite AI app: Video conferencing note-takers (Fathom)
  • 2024 model predictions: Many high-performing models, continuous quality improvement
  • Most admired: Meta's open source commitment with Llama series
  • Agents outlook: Very bullish, expects significant growth

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome