Summary by TwinMind · 10m 45s
Summary by TwinMind · 10m 45s
How Node.js v22.5.0 Broke (Almost) Every Package
How Node.js v22.5.0 Broke (Almost) Every Package
Theo - t3․gg
Theo - t3․gg
Bug Overview
- Node 22.5.0 broke most applications due to V8 fast API implementation for fs.closeSync
- Performance improvement attempt by Yagiz caused unexpected breaking changes
- Issue affected applications using npm and other tools relying on fs.close
- Bug has since been patched with lessons learned about production software
Root Cause Analysis
- Primary Issue: Adding V8's fast API to fs.closeSync broke almost all applications
- Goal: Significantly improve performance of fs operations for applications like npm
- Unintended Consequence: fs.close and fs.closeSync shared same underlying C++ code path
- Fast API: V8 mechanism to access native C++ code without multiple abstraction layers
- Built into V8 JavaScript engine (used in Chrome and Node)
- Particularly useful for file system and IO operations
- Bypasses JavaScript layer parsing for better performance
Technical Implementation Problems
- First Bug: Crash with error "V8 object get creation context checked"
- Caused by destructuring results of internal binding FS
- Lost creation context of function when using destructuring syntax
- Fixed by accessing whole object instead of destructuring specific keys
- Second Bug: Fast API call triggered for both fs.close and fs.closeSync
- fs.close hitting new path meant only for closeSync
- Sync implementation didn't call callbacks expected by async fs.close
- fs.close callback never executed, causing npm failures
Code Review Analysis
- Original PR received thorough code review with multiple approvals
- No reviewer caught that close and closeSync hit same code path
- Large +/- diff made comprehensive review challenging
- Comment "I think I got it before merging is a little scary" noted
- Non-intuitive that close and closeSync shared underlying implementation
Internal Architecture Issues
- Shared Implementation: closeSync internally just called binding.close
- Binding Confusion: Set fast method bound to 'close' affected both functions
- Callback Handling: Fast API implementation lacked callback support for async fs.close
- Tracing Difficulty: Hard to trace C++ bindings back to JavaScript consumption points
- Type Safety: No type checker coverage from low-level C++ to JavaScript layer
Testing Infrastructure Failures
- Canary Process: Node's safety net for catching ecosystem-breaking bugs failed
- CI Issues: Jobs showed successful even when failures occurred
- Investigation Ongoing: Team investigating why safety mechanisms didn't catch issue
- Gold Mine: Project designed to catch unintended bugs affecting ecosystem
Resolution Implementation
- Separate Functions: Split close and closeSync into two different C++ functions
- Individual Binding: Each function now has separate binding mechanism
- Proper Callback Handling: Async version maintains callback functionality
- Fast API Isolation: Fast API only applied to sync version as intended
Lessons Learned
- Internal Function Complexity: Difficult to know all places where functions are called
- Code Path Sharing: Non-obvious shared implementations create unexpected dependencies
- Review Limitations: Even thorough code review can miss architectural implications
- CI Critical Importance: Final safety net must be reliable for production software
- Multi-Language Challenges: Tracing across C++/JavaScript boundaries is non-trivial
