AI summary by TwinMind

Was Porting To Rust A Mistake?

Theo - t3․gg

25m 41s

Migration Overview

  • Turbo repo completed migration from Go to Rust after 15 months
  • Ported 70,000 lines of Go code to 151,000 lines of Rust code
  • JavaScript/TypeScript build system focused on performance and stability
  • Used innovative "rust-go-rust sandwich" architecture with Zig compiler assistance

Technical Architecture & Challenges

  • Go Sandwich Strategy: Rust binary calls Go binary which contains statically linked Rust libraries
  • Cross-platform Compilation: Required building for 6 different platforms
  • Windows ARM Issues: Go's MW conflicted with Rust's MSVC, forced to use x86-64 emulation
  • Zig CC Compiler: Used for cross-compilation support, bundled libraries for all target platforms
  • Incremental Porting: Sandwich allowed gradual migration while maintaining functionality

Key Components Migrated

  • File Hashing: Mechanism for creating content fingerprints for cache comparisons
  • Lock File Analysis: Dependency checking in repositories
  • Cache Signature Verification: Ensuring cache integrity with private keys
  • Package Graph: Graph data structure of packages and dependencies
  • Prune Command: Removes everything except single package and dependencies

Performance Impact & Caching

  • 2 million hours saved in developer compute time (comparable to author's 2.6M YouTube watch hours)
  • Cache system uses tar files stored locally and in Vercel remote cache
  • Hashing includes source code, environment variables, dependencies
  • Global hash for repo-wide changes, task-specific hashes for individual components
  • Environment variables compiled into code affect cache validity

Hash Stability & Testing

  • Used Cap'n Proto serialization format for cross-language compatibility
  • CI runs tasks on both Go and Rust code paths, comparing hashes
  • Any hash divergence causes CI failure
  • Found bugs in null value handling, package change detection, filter flag processing
  • Ensured perfect matching between Go and Rust behavior

Release Process

  • Used feature flags for local development without shipping to users
  • Built "run outline" - all-Rust version with stubbed functionality
  • Dog-fooded internally at Vercel for 72 hours before v1.11 release
  • Integration tests matched Go version "log line for log line"

Lessons Learned

  • Should have invested in single serialization format from start
  • Needed faster shipping of porting strategies
  • Code quality improvement before porting would have accelerated process
  • Should have fully specified core behaviors (globbing, file watching, hashing) earlier
  • Many undocumented behaviors discovered during port

Performance Concerns & Criticisms

  • Repo Size: 839MB before installation, massive codebase
  • Feature Stagnation: Minimal new features shipped during 15-month migration
  • Rust Compilation: Slower compile times compared to Go alternatives
  • Development Speed: Contrast with esbuild's 4-month Go development cycle
  • Industry Trend: Concern about over-adoption of Rust for JavaScript tooling

Future Plans

  • Integration with Vercel ecosystem
  • Conformance and code owners features
  • Improved stability and CI performance
  • Deeper turbo pack integration

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome