AI summary by TwinMind
AI’s “Intelligence Explosion” Is Coming. Here’s What That Means.
Sabine Hossenfelder
7m 34s
Current State of AI Progress
- AI appears to progress slowly but represents "calm before the storm"
- Researchers expect intelligence explosion when systems learn self-improvement
- Progress accelerating toward recursive self-improvement capabilities
Terminology Evolution
- "Singularity" term falling out of favor
- Associated with black holes, destruction, time ending
- Poor marketing/selling point
- Now called "intelligence explosion"
- Driven by "recursive self-improvement"
Industry Timeline Predictions
- Google CEO Eric Schmidt statement:
- ~5 years until systems write own code
- Systems will take their code and improve it recursively
- Represents fundamental "change in slope" in progress curve
Recent Research Developments
DeepMind's Alpha Evolve (May)
- AI improves code through evolution-like process
- Uses random mutations, keeps most successful versions
- Evolves for accuracy/efficiency rather than biological traits
- Key achievement: Invented new matrix multiplication method
- 1% reduction in Gemini training time
- Matrix multiplication fundamental to LLM operations
- Limitation: Doesn't edit own code directly
Two Labs Self-Improvement Mechanism (March)
- LLM self-improvement through question simplification
- Process:
- Model simplifies questions
- Checks answer correctness
- Builds to increasing complexity
- Example: Learning increasingly complex integrals
- Result: Dramatic improvement in success rates
- Limitation: Not true self-rewriting
Hyperparameter Self-Tuning Research
- LLM adjusts own hyperparameters (normally tuned externally)
- Process:
- Creates fake training questions with known answers
- Trains on self-invented questions
- Tunes hyperparameters for best results
- Success: Significant performance increase on Llama-based model benchmarks
- Problem: Catastrophic forgetting observed
- Performance on earlier tasks declines with more edits
- "More you let model edit itself, less it remembers earlier training"
Darwin Google Machine (Recent Preprint)
- AI edits own Python code directly
- Evolutionary approach: mutations evaluated on benchmarks, best kept
- Results: Significant improvement on various benchmarks
- Potential: Could theoretically discard language model parts, build any program
- Current limitations:
- Each mutant must run/score within minutes
- Limited by affordable compiler resources
- Changes necessarily small
- Future concern: What happens with supercomputer resources and removed constraints?
Implementation Outlook
- Current work represents small research projects
- Near future: Ideas likely implemented in existing systems
- Timeline acceleration expected
