AI summary by TwinMind
(Actually) Benchmarking TypeScript
Theo - t3․gg
7m 31s
Tool Introduction & Problem Statement
- First benchmarking tool for TypeScript type performance (not compilation speed)
- Addresses IDE/type-checking performance issues in large codebases
- Libraries like tRPC, Drizzle, Zod cause TypeScript server crashes in complex projects
- Current TypeScript team recommendations don't consider performance
- No existing ways to identify type performance bottlenecks
Archetype's New Testing Tool
- Archetype team released type-level benchmarking tool
- Alpha version includes type performance measurement
- Shows exact impact of changes on type performance
- Addresses gap in existing type testing (limited by static analysis)
Real-World Application Examples
- UploadThing experiencing increasingly complex dynamic builder patterns
- Previously relied on gut feel testing (manual editor testing)
- High-end M1-M3 MacBooks ($10,000s) show different performance than user machines
- Need objective ways to identify when types become too heavy
Tool Features & Capabilities
- Type assertions and testing built into framework
- Benchmarks run separately from tests, no special setup required
- Example usage:
tsx benches.tsorts-node benches.ts - Measures type instantiation counts (not just timing)
- Deterministic results across platforms and TypeScript versions
- Can set thresholds for maximum allowed instantiations before errors
Technical Implementation Details
- Uses TypeScript to track type definition operations
- Example: complex type with recursive calls shows 69 vs 349 instantiations
- Instantiation count correlates strongly with actual performance
- Same numerical output guaranteed across different machines/CI environments
- Some instantiations cheaper than others but correlation very strong
Archetype Project Context
- Zod-like validation library focused on speed optimization
- Optimized for both runtime validation and type definition performance
- Growing community adoption and hype
- Team focused on concrete solutions for TypeScript performance issues
Adoption Plans & Impact
- Expected adoption by library developers
- Speaker planning to test with UploadThing this weekend
- Tool essential for library maintainers (even if not all developers need it)
- Faster ecosystem improvement through better tooling for library authors
