AI summary by TwinMind
AI code is here. We need to be responsible with it.
Theo - t3․gg
40m 44s
Devon AI Coding Mistake
- Devon AI added event on banner component mount
- Generated 6.6 million PostHog events in one week
- Cost $733 ($500 Devon + $733 events = $1,233 total)
- Lesson: review AI-generated code multiple times
- PostHog offered refund for honest mistakes
Code Review Culture Problems
- AI generates more code, but reviewing effort hasn't decreased
- People's tolerance for tedious tasks drops as tools improve
- Writing code now easier than reviewing it thoroughly
- Nobody liked code reviews before, tolerance going down further
- Teams responding "Claude generated it" when questioned about code
Code Review Best Practices
- Start/end each day with code review
- Clear all assigned PRs before opening editor
- Reserve 1 hour end-of-day for review updates
- Make code review part of routine, not requested task
- Code review is everyone's responsibility, not just on-call
- Ask questions in PRs - "dumb questions" prevent outages
- Commit good PR questions as code comments
- New hires required to ask 1 dumb question daily
Self-Review Practices
- Use
git add -pfor mini code reviews - Stage code piece by piece before committing
- Review AI-generated code thoroughly before pushing
- Create PRs for personal review even without team review
Usage-Based Pricing Challenges
- Enables scaling without fixed costs
- Allows preview builds and dev environments
- Risk of unexpectedly high bills from mistakes
- PostHog: $0 under 1M events/month, usage-based above
- Mix Panel comparison: terrible guess-and-check pricing model
Solutions for Usage-Based Billing
- Spend limits (trade-off: service might go down)
- Alerts for usage spikes
- Platform responsibility: better notifications when crossing thresholds
- Platforms should alert aggressively before big bills
Safety Nets and Rollback Culture
- 70% of Google outages from changes to live systems
- One-click rollback essential for usage-based platforms
- People more hesitant to rollback than deploy (counterintuitive)
- Rollbacks should be aggressive, not shameful
- Progressive rollouts ideal, but rollbacks critical fallback
Platform Responsibilities
- Better alerts when users cross spending thresholds
- Warn about large file uploads (>0.5MB)
- Automatic blocking of clearly malicious traffic
- Refund policy for honest mistakes vs. viral moments
- "Can't give back viral moment, can always refund bill"
AI Code Integration Strategy
- More code being generated, less time spent reviewing
- Need stronger review culture as AI adoption increases
- Tools like Code Rabbit helpful but don't replace human familiarity
- Must understand code well enough to theorize what could go wrong
