AI summary by TwinMind
How Minecraft AI ACTUALLY works
Theo - t3․gg
40m 56s
Oasis AI Model Introduction
- New AI model called "Oasis" - not official name, generates Minecraft-like gameplay
- Generates video at 25 FPS vs less than 1 FPS from existing models
- Real-time AI-generated game that creates frames on-the-fly based on user inputs
- No actual game engine - purely foundation model generating pixels
Live Gameplay Demonstration
- Starts with Minecraft screenshot, generates new frames based on mouse/keyboard inputs
- No object permanence - looking away and back changes environment completely
- Trees/objects disappear and reappear as different things when view changes
- Only knows current frame, no memory of previous states
- Breaking blocks causes hallucinated inventory items and UI elements
- Opening/closing inventory changes surrounding environment
- System frequently crashes or freezes during use
Technical Architecture Comparison
- Traditional game engines: Store world data, render only visible objects using occlusion culling
- AI model: Only generates what's currently visible, deletes everything else
- Uses frame-by-frame generation like video compression (motion data between frames)
- Similar to video encoding where only pixel changes are stored, not full frames
- Model trained on subset of OpenAI's Minecraft video data
Video Compression Parallels
- Video compression works by encoding differences between frames, not full images
- Snow/confetti destroy video quality due to many changing pixels
- Static content (like text editors) easy to encode, movement increases CPU usage dramatically
- 1080p video with 300 frames can be 1/10th size of single 1080p PNG due to compression magic
Speedrunning Attempts
- Community already speedrunning AI Minecraft - goal is reaching the Nether
- Strategy: Look at orange/red objects to trick model into generating lava/Nether environment
- World record: 1 minute 47 seconds
- Extremely difficult due to unpredictable hallucinations and lack of object persistence
Gaming Industry Concerns
- Worried about impact on children's games - potential for AI-generated shovelware
- Historical parallel to Atari era: No quality control led to bins of terrible games
- ET game so bad they buried hundreds of thousands of copies in New Mexico landfill
- Nintendo's innovation: Seal of approval system to prevent shovelware
- Fear AI will push gaming back to pre-Nintendo quality control era
Technical Specifications
- Model uses diffusion forcing with independent per-token noise levels
- Requires $25,000 H100 GPU per user session
- Custom ASIC planned for 4K generation
- Dynamic noising system resets accumulated errors to prevent "noise hell"
- Intentionally adds and resets noise on schedule to maintain stability
Industry Analysis and Critique
- Infinite worlds don't equal better games - quantity vs quality problem
- Starfield example: Massive empty worlds with poor execution vs focused experiences
- Mass Effect comparison showing importance of crafted, intentional design
- No Man's Sky lesson: Exploration magic comes from quality of discoveries, not quantity
- Outer Wilds as counterexample: Small but densely crafted world
Future Implications
- Developers claim next goal is prompt-based world creation
- Potential for much higher quality visuals than traditional rendering
- Could enable games impossible to run on consumer hardware
- Risk of automating creation of terrible games rather than solving real problems
- Concern about brain rot content for children marketed as innovation
