AI summary by TwinMind
How to Keep AI Under Control | Max Tegmark | TED
TED
12m 10s
Opening Reflection on Previous Predictions
- 5 years ago warned about superintelligence dangers on TED stage
- Admits being wrong - situation worse than predicted
- Governments allowed AI companies to progress without meaningful regulation
- AI progress exceeded predictions
AI Progress Visualization and Current State
- Abstract landscape metaphor: elevation = task difficulty for AI to reach human level
- Sea level = AI capabilities then vs now
- "Sea rising fast" - many tasks already submerged
- Water on track to submerge all land = matching human intelligence at all cognitive tasks
- This defines AGI (Artificial General Intelligence)
AGI Timeline Acceleration
- Companies pursuing AGI: OpenAI, Google DeepMind, Anthropic
- Also building superintelligence beyond human intelligence
- Few years gap predicted from AGI to superintelligence
- Timeline shift:
- Previously: AI researchers thought AGI decades away
- Microsoft: "almost here" - seeing "sparks of AGI" in ChatGPT-4
- Metaculus betting site: timeline dropped from 20 years to 3 years in last 18 months
- Industry leaders predict 2-3 years until "we get outsmarted"
Recent AI Progress Examples
- Robotics: from clunky movement to dancing
- Image generation: Midjourney dramatic improvement year-over-year with same prompt
- Deepfakes increasingly convincing (Tom Cruise example)
- Yoshua Bengio: Large language models mastered language/knowledge, pass Turing test
- Counter to "stochastic parrots" criticism: AIs have world representations
- Llama-2 contains literal world map
- AI builds geometric representations of abstract concepts (true/false)
Existential Risk Warnings
- Alan Turing prediction: "machines take control" as default outcome
- Superintelligence = new species, not like electricity
- Characteristics: super capable, amoral psychopaths, don't sleep, think faster, self-replicate, nothing human
- Industry leader warnings:
- Sam Altman (OpenAI): "lights out for all of us"
- Dario Amodei (Anthropic): 10-25% extinction risk
- May: AGI CEOs and AI researchers warned about human extinction
- EU leadership warned about AI extinction risk
Current AI Safety Limitations
- Lack convincing AI safety plan
- Evals (evaluations) look for risky behavior - necessary but insufficient
- Training AI "not to say bad things" vs "not do bad things"
- Evals can prove presence of risk, not absence of risk
- Guardrails try to physically limit harm but inadequate against superintelligence
Provably Safe AI Vision
- Need provably safe systems where harm is impossible
- "Provable" in strong physics sense, not just convincing judges
- Even superintelligent AI can't violate physics laws
- Co-authored paper with Steve Omohundro on this approach
Technical Implementation Approach
- Formal verification: proves properties about code
- AI will revolutionize automatic proving and program synthesis
- Process:
- Human writes specification AI tool must obey
- Examples: impossible laptop login without password, DNA printer can't synthesize dangerous viruses
- Powerful AI creates both AI tool and proof it meets spec
- Machine learning learns algorithms, then re-implement in verifiable architecture
- Key insight: easier to verify proof than discover it
- Only need to understand/trust proof-checking code (few hundred lines)
- Envision proof checkers built into compute hardware
Alternative: AI Neuroscientist Approach
- If AI can't write tool directly, train AI to learn desired behavior
- Use different AI to extract learned algorithm/knowledge
- Related to mechanistic interpretability field (rapid progress)
Practical Example: Addition Algorithm
- Demonstrate with binary addition (like counting on 2 fingers vs 10)
- Train recurrent neural network on addition task
- Algorithm in black box defined by parameter tables
- AI tool automatically distills learned algorithm into Python program
- Dafny formal verification tool proves program correctly adds any numbers (not just training data)
Conclusion and Call to Action
- Provably safe AI possible but requires time and work
- Most AI benefits don't require superintelligence
- Don't pause AI - pause reckless race to superintelligence
- Stop training ever-larger models we don't understand
- Heed ancient Greek hubris warning (Icarus story)
- AI gives "incredible intellectual wings" for amazing achievements if we avoid "flying to the sun"
