AI summary by TwinMind

How I cut our cache by 98.741% (real screenshot btw)

Theo - t3․gg

12m 48s

Problem Overview

  • Caching is one of 2-3 hard problems in computer science
  • Upload Thing had caching issue causing billing problems (not user-facing)
  • Issue: caching what we got vs. caching what we needed
  • 4GB bandwidth usage for SL upload thing registry, 2.5GB for upload thing latest in 7-day window

Original Implementation

  • Feature: warning alert for old Upload Thing SDK versions
  • Shows current version, key info, prompts to update to latest
  • Used npm Registry API to get version data
  • Cached using Vercel fetch cache with next revalidation tags
  • 24-hour cache expiration using revalidation seconds
  • Two main functions:
    • listUploadThingVersions() - gets all versions from npm API
    • getMinimumVersion() - filters to find lowest supported version
  • Skipped canary and deprecated versions
  • Problem: caching entire npm API response (over 1MB of text data)

Root Cause Analysis

  • Fetch cache stores everything returned from API calls by default
  • npm API response contains massive amounts of unnecessary data
  • Each page request pulled full cached response from cache
  • Thousands of users = thousands of cache reads of oversized data
  • Made things slower, increased bandwidth usage, increased storage costs
  • Bandwidth usage was orders of magnitude higher than needed

Solution: Unstable Cache Implementation

  • Replaced fetch-level caching with unstable_cache
  • Allows granular control over what gets cached
  • Removed all next revalidation tags from fetch calls
  • Created internal functions for API calls (not exposed)
  • Wrapped with cached functions using unstable_cache
  • Key change: cache function return values, not fetch responses
  • getLatestUTVersion() - caches only npmResponse.version
  • getMinimumUploadThingVersion() - caches only version + reason
  • Custom cache keys and revalidate times still supported

Results and Performance Impact

  • Bandwidth usage plummeted dramatically
  • Cache reads went from ~1GB to 5MB in same timeframe
  • 90% cache hit rate maintained
  • Cache writes increased slightly due to reduced specificity
  • Response times improved due to less data transfer
  • Orders of magnitude reduction in bandwidth costs
  • Vercel UI shows "undefined" for unstable cache (vs URL-based caching)

Caching Philosophy and Best Practices

  • Don't cache everything automatically from fetch requests
  • Cache with intent based on what you actually need
  • Cache as close to final result as possible, not intermediate steps
  • Think about cache input/output boundaries logically
  • Consider where cache starts and ends in your data pipeline
  • Fetch caching by default might not be the right approach
  • Pipeline visualization: User Request → npm API → Filter → Render → Page Load
  • Better to cache processed results than raw API responses

Future Considerations

  • unstable_cache will eventually become stable with different API
  • Granular caching control will remain important
  • Component-level caching possible with Next.js for even better performance
  • Consider caching rendered components for maximum optimization

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome