Problem Overview

• Caching is one of 2-3 hard problems in computer science
• Upload Thing had caching issue causing billing problems (not user-facing)
• Issue: caching what we got vs. caching what we needed
• 4GB bandwidth usage for SL upload thing registry, 2.5GB for upload thing latest in 7-day window

Original Implementation

• Feature: warning alert for old Upload Thing SDK versions
• Shows current version, key info, prompts to update to latest
• Used npm Registry API to get version data
• Cached using Vercel fetch cache with next revalidation tags
• 24-hour cache expiration using revalidation seconds
• Two main functions:
  • listUploadThingVersions() - gets all versions from npm API
  • getMinimumVersion() - filters to find lowest supported version
• Skipped canary and deprecated versions
• Problem: caching entire npm API response (over 1MB of text data)

Root Cause Analysis

• Fetch cache stores everything returned from API calls by default
• npm API response contains massive amounts of unnecessary data
• Each page request pulled full cached response from cache
• Thousands of users = thousands of cache reads of oversized data
• Made things slower, increased bandwidth usage, increased storage costs
• Bandwidth usage was orders of magnitude higher than needed

Solution: Unstable Cache Implementation

• Replaced fetch-level caching with unstable_cache
• Allows granular control over what gets cached
• Removed all next revalidation tags from fetch calls
• Created internal functions for API calls (not exposed)
• Wrapped with cached functions using unstable_cache
• Key change: cache function return values, not fetch responses
• getLatestUTVersion() - caches only npmResponse.version
• getMinimumUploadThingVersion() - caches only version + reason
• Custom cache keys and revalidate times still supported

Results and Performance Impact

• Bandwidth usage plummeted dramatically
• Cache reads went from ~1GB to 5MB in same timeframe
• 90% cache hit rate maintained
• Cache writes increased slightly due to reduced specificity
• Response times improved due to less data transfer
• Orders of magnitude reduction in bandwidth costs
• Vercel UI shows "undefined" for unstable cache (vs URL-based caching)

Caching Philosophy and Best Practices

• Don't cache everything automatically from fetch requests
• Cache with intent based on what you actually need
• Cache as close to final result as possible, not intermediate steps
• Think about cache input/output boundaries logically
• Consider where cache starts and ends in your data pipeline
• Fetch caching by default might not be the right approach
• Pipeline visualization: User Request → npm API → Filter → Render → Page Load
• Better to cache processed results than raw API responses

Future Considerations

• unstable_cache will eventually become stable with different API
• Granular caching control will remain important
• Component-level caching possible with Next.js for even better performance
• Consider caching rendered components for maximum optimization