AI summary by TwinMind

Deepseek R1 Is Really, Really Good

Theo - t3․gg

34m 17s

DeepSeek R1 Introduction and Performance Claims

  • New open source reasoning model beating ChatGPT performance in many areas
  • 96% cheaper than OpenAI: $15/million tokens → $0.55-0.60/million input, $60/million → $2.19/million output
  • Available on GitHub as downloadable open model
  • First production-ready open-source reasoning model

Reasoning Models Explanation

  • Traditional AI models work as "fancy autocomplete" - predict next word based on previous context
  • Reasoning models add step: "how would you solve this" before generating response
  • ChatGPT O1 shows minimal thinking process, keeps reasoning hidden
  • DeepSeek R1 shows full reasoning chain of thought in plain text
  • Slower response time but more accurate, consistent answers

Live Testing Comparison

  • Advent of Code 2021 Day 12 Rust problem test
  • ChatGPT O1: minimal thinking shown, solved correctly
  • DeepSeek R1: extensive visible reasoning (291 seconds), initially incomplete but finished with help
  • GPT-4 Mini: failed to solve
  • Claude: failed to solve
  • Only reasoning models successfully solved complex problem

Open Source Advantages

  • Model downloadable and runnable (people running on phones)
  • Full transparency in reasoning process vs closed OpenAI models
  • Can see exactly how model thinks through problems
  • Enables better prompt engineering and debugging

Cost Comparison Analysis

  • OpenAI O1: $15/million input, $60/million output
  • DeepSeek R1: $0.55/million input, $2.19/million output
  • Even cheaper than Claude Sonnet ($3 input, $15 output)
  • Fundamentally changes economics of using smart models
  • T3 Chat offers on $8/month tier vs OpenAI's $200/month losing money

DeepSeek Architecture Details

  • Built 6 dense models distilled from DeepSeek V3
  • Based on Llama (Meta) and Qwen (Alibaba) models
  • Uses reasoning layer to provide more context and self-questioning
  • Achieves code forces score matching OpenAI's best models

Synthetic Data Training Revolution

  • DeepSeek trained primarily on AI-generated data, not scraped web data
  • Google DeepMind study confirms synthetic data effectiveness
  • Web scraping becoming harder: Reddit, Twitter, Stack Overflow restricting access
  • Stack Overflow usage plummeting since ChatGPT launch

Data Accessibility Crisis

  • Less web data accessible now than in 2020 when OpenAI/Anthropic trained
  • Paywalls, API restrictions, dead links reducing available training data
  • Companies like Reddit/Twitter monetizing data access
  • AI tools cannibalizing platforms they were trained on

Synthetic Data Generation Process

  • Use existing models (OpenAI) to generate training data for new models
  • Like compression: original data → compressed model → generate more data
  • AI models contain "distilled" version of original training data
  • Can generate unlimited questions/answers from compressed knowledge

Image Compression Analogy

  • Training AI like compressing video: massive data → smaller efficient model
  • Encoding (training) computationally expensive, decoding (inference) cheaper
  • Synthetic data generation like creating more content from compressed source
  • Better compression possible with structured, pattern-fitting data

DeepSeek V3 Pricing Impact

  • Current: $0.27/million input, $1.10/million output (after price hike)
  • Comparable to Claude 3.5 Sonnet quality
  • Represents massive cost reduction in AI access
  • Makes high-quality AI accessible at consumer prices

Potential Bias Concerns

  • 100% synthetic data allows heavy filtering/manipulation
  • System prompts can inject biases during data generation
  • Example: could filter out specific topics, characters, viewpoints
  • Can favor certain technologies/approaches over others
  • Biases embedded in training data, not just system prompts
  • Concern about Chinese company controlling widely-used model

Performance Benchmarks

  • Artificial Analysis shows DeepSeek performing near top across metrics
  • Neck-and-neck with Claude for most tasks
  • Some weaknesses compared to reasoning models like O1
  • Speed issues: started at 90 TPS, now down to 17 TPS due to traffic

Speed and Infrastructure Challenges

  • DeepSeek API speed degraded from 90 to 40 TPS
  • Third-party providers (Together, Fireworks) even slower (10-30 TPS)
  • T3 Chat uses Azure for OpenAI models: 180-200 TPS vs 77 TPS direct
  • Azure partnership provides faster, more reliable access

Future Implications

  • Race to bottom on price, race to top on quality
  • Reasoning models running on phones now possible
  • Need for diverse open source models from different countries/perspectives
  • Potential for sub-models handling different reasoning steps
  • 2024 will be "crazy year for AI"

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome