AI summary by TwinMind
Google won. (Gemini 2.5 Pro is INSANE)
Theo - t3․gg
32m 46s
Gemini 2.5 Pro Overview
- New thinking model designed for complex problems
- Leads common benchmarks by meaningful margins
- Strong reasoning and code capabilities
- Immediately topped LM Arena leaderboard, beating GPT-4.5 and DeepSeek
- Pricing not yet announced but confirmed for paid tiers soon
- Added to T3 Chat with high rate limits through Google contacts
Sponsor Segment - Augment Code
- AI tool for understanding large codebases
- Example: Rocky Corp monorepo with 200,000 lines TypeScript code
- Indexes quickly in VS Code, answers questions instantaneously
- T3 Chat codebase comparison: 17,000 lines vs 200,000 lines (10x larger)
- Free trial available at soyv.link/ugmentcode
Model Performance Benchmarks
- Crushing OpenAI O3 Mini High on humanity last exam
- Particularly strong at math (previous Gemini Flash had poor math performance)
- Still not strongest at coding compared to Claude
- Thinking models create suboptimal coding editor experience due to response delays
- Ball-in-hexagon test results: performs well initially but balls fall out at corners
Google Model Features
- 1 million token input context window (vs 100k-200k for competitors)
- Grounding feature (search integration)
- Image parsing capabilities
- Recently added image editing
- Native PDF support with image, graph, chart extraction
- 2 million context window coming soon
Real-world Use Case Example
- Build Different blog HTML parsing task
- 71,000 tokens, 222,000 characters of HTML data
- Successfully extracted 61 iframes (60 with YouTube/SoundCloud sources)
- Demonstrates practical utility of large context windows
Google's Competitive Advantages
Three Key Success Factors
- Data: Infinite data from decades of running internet
- Science: Deep history in ML/LLM research, Google Translate as early LLM example
- Hardware: Custom Google chips (TPUs), not dependent on NVIDIA
Hardware Advantage
- Custom chips built specifically for inference (similar to crypto ASIC revolution)
- Gemini Flash significantly faster than competitors
- Only model in "green section" of intelligence vs speed graph
- Tight integration between science and hardware teams
Pricing Advantage
- Gemini Flash Light: 25x cheaper than GPT-4 for similar performance
- 1 million T3 Chat messages cost $1,200 vs $31,000 for Claude (60x difference)
- Gemini 2.0 Flash: insanely cheap despite high performance
Competitive Landscape Analysis
- Most companies focus on single advantage (data, science, or hardware)
- Science-focused: OpenAI, DeepSeek, Anthropic
- Data-focused: Scale AI, Data Curve
- Hardware-focused: Groq, Cerebras, Samba Nova
- Google: Only company meaningfully doing all three
Apple's Position
- Lacks data due to privacy focus
- Underinvested in AI science historically
- Hardware optimized for power efficiency, not inference performance
- Siri was acquired, not built internally
- Position described as "embarrassing" and "screwed"
Google Cloud Platform Integration
- AI Studio much better than rest of Google Cloud
- Logan (Google AI team) actively engaging community
- Improved communication and awareness compared to historical GCP experience
- VP of Research using hexagon demo shows community engagement
API Limitations
- Thinking capabilities visible in AI Studio but not available via API
- T3 Chat users don't see reasoning process, only final answers
- Request for Google to expose thinking data through API
