AI summary by TwinMind
Deepseek R1 Is Really, Really Good
Theo - t3․gg
34m 17s
DeepSeek R1 Introduction and Performance Claims
- New open source reasoning model beating ChatGPT performance in many areas
- 96% cheaper than OpenAI: $15/million tokens → $0.55-0.60/million input, $60/million → $2.19/million output
- Available on GitHub as downloadable open model
- First production-ready open-source reasoning model
Reasoning Models Explanation
- Traditional AI models work as "fancy autocomplete" - predict next word based on previous context
- Reasoning models add step: "how would you solve this" before generating response
- ChatGPT O1 shows minimal thinking process, keeps reasoning hidden
- DeepSeek R1 shows full reasoning chain of thought in plain text
- Slower response time but more accurate, consistent answers
Live Testing Comparison
- Advent of Code 2021 Day 12 Rust problem test
- ChatGPT O1: minimal thinking shown, solved correctly
- DeepSeek R1: extensive visible reasoning (291 seconds), initially incomplete but finished with help
- GPT-4 Mini: failed to solve
- Claude: failed to solve
- Only reasoning models successfully solved complex problem
Open Source Advantages
- Model downloadable and runnable (people running on phones)
- Full transparency in reasoning process vs closed OpenAI models
- Can see exactly how model thinks through problems
- Enables better prompt engineering and debugging
Cost Comparison Analysis
- OpenAI O1: $15/million input, $60/million output
- DeepSeek R1: $0.55/million input, $2.19/million output
- Even cheaper than Claude Sonnet ($3 input, $15 output)
- Fundamentally changes economics of using smart models
- T3 Chat offers on $8/month tier vs OpenAI's $200/month losing money
DeepSeek Architecture Details
- Built 6 dense models distilled from DeepSeek V3
- Based on Llama (Meta) and Qwen (Alibaba) models
- Uses reasoning layer to provide more context and self-questioning
- Achieves code forces score matching OpenAI's best models
Synthetic Data Training Revolution
- DeepSeek trained primarily on AI-generated data, not scraped web data
- Google DeepMind study confirms synthetic data effectiveness
- Web scraping becoming harder: Reddit, Twitter, Stack Overflow restricting access
- Stack Overflow usage plummeting since ChatGPT launch
Data Accessibility Crisis
- Less web data accessible now than in 2020 when OpenAI/Anthropic trained
- Paywalls, API restrictions, dead links reducing available training data
- Companies like Reddit/Twitter monetizing data access
- AI tools cannibalizing platforms they were trained on
Synthetic Data Generation Process
- Use existing models (OpenAI) to generate training data for new models
- Like compression: original data → compressed model → generate more data
- AI models contain "distilled" version of original training data
- Can generate unlimited questions/answers from compressed knowledge
Image Compression Analogy
- Training AI like compressing video: massive data → smaller efficient model
- Encoding (training) computationally expensive, decoding (inference) cheaper
- Synthetic data generation like creating more content from compressed source
- Better compression possible with structured, pattern-fitting data
DeepSeek V3 Pricing Impact
- Current: $0.27/million input, $1.10/million output (after price hike)
- Comparable to Claude 3.5 Sonnet quality
- Represents massive cost reduction in AI access
- Makes high-quality AI accessible at consumer prices
Potential Bias Concerns
- 100% synthetic data allows heavy filtering/manipulation
- System prompts can inject biases during data generation
- Example: could filter out specific topics, characters, viewpoints
- Can favor certain technologies/approaches over others
- Biases embedded in training data, not just system prompts
- Concern about Chinese company controlling widely-used model
Performance Benchmarks
- Artificial Analysis shows DeepSeek performing near top across metrics
- Neck-and-neck with Claude for most tasks
- Some weaknesses compared to reasoning models like O1
- Speed issues: started at 90 TPS, now down to 17 TPS due to traffic
Speed and Infrastructure Challenges
- DeepSeek API speed degraded from 90 to 40 TPS
- Third-party providers (Together, Fireworks) even slower (10-30 TPS)
- T3 Chat uses Azure for OpenAI models: 180-200 TPS vs 77 TPS direct
- Azure partnership provides faster, more reliable access
Future Implications
- Race to bottom on price, race to top on quality
- Reasoning models running on phones now possible
- Need for diverse open source models from different countries/perspectives
- Potential for sub-models handling different reasoning steps
- 2024 will be "crazy year for AI"
