AI summary by TwinMind

AI code is here. We need to be responsible with it.

Theo - t3․gg

40m 44s

Devon AI Coding Mistake

  • Devon AI added event on banner component mount
  • Generated 6.6 million PostHog events in one week
  • Cost $733 ($500 Devon + $733 events = $1,233 total)
  • Lesson: review AI-generated code multiple times
  • PostHog offered refund for honest mistakes

Code Review Culture Problems

  • AI generates more code, but reviewing effort hasn't decreased
  • People's tolerance for tedious tasks drops as tools improve
  • Writing code now easier than reviewing it thoroughly
  • Nobody liked code reviews before, tolerance going down further
  • Teams responding "Claude generated it" when questioned about code

Code Review Best Practices

  • Start/end each day with code review
  • Clear all assigned PRs before opening editor
  • Reserve 1 hour end-of-day for review updates
  • Make code review part of routine, not requested task
  • Code review is everyone's responsibility, not just on-call
  • Ask questions in PRs - "dumb questions" prevent outages
  • Commit good PR questions as code comments
  • New hires required to ask 1 dumb question daily

Self-Review Practices

  • Use git add -p for mini code reviews
  • Stage code piece by piece before committing
  • Review AI-generated code thoroughly before pushing
  • Create PRs for personal review even without team review

Usage-Based Pricing Challenges

  • Enables scaling without fixed costs
  • Allows preview builds and dev environments
  • Risk of unexpectedly high bills from mistakes
  • PostHog: $0 under 1M events/month, usage-based above
  • Mix Panel comparison: terrible guess-and-check pricing model

Solutions for Usage-Based Billing

  • Spend limits (trade-off: service might go down)
  • Alerts for usage spikes
  • Platform responsibility: better notifications when crossing thresholds
  • Platforms should alert aggressively before big bills

Safety Nets and Rollback Culture

  • 70% of Google outages from changes to live systems
  • One-click rollback essential for usage-based platforms
  • People more hesitant to rollback than deploy (counterintuitive)
  • Rollbacks should be aggressive, not shameful
  • Progressive rollouts ideal, but rollbacks critical fallback

Platform Responsibilities

  • Better alerts when users cross spending thresholds
  • Warn about large file uploads (>0.5MB)
  • Automatic blocking of clearly malicious traffic
  • Refund policy for honest mistakes vs. viral moments
  • "Can't give back viral moment, can always refund bill"

AI Code Integration Strategy

  • More code being generated, less time spent reviewing
  • Need stronger review culture as AI adoption increases
  • Tools like Code Rabbit helpful but don't replace human familiarity
  • Must understand code well enough to theorize what could go wrong

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome