AI summary by TwinMind

How to Keep AI Under Control | Max Tegmark | TED

TED

12m 10s

Opening Reflection on Previous Predictions

  • 5 years ago warned about superintelligence dangers on TED stage
  • Admits being wrong - situation worse than predicted
  • Governments allowed AI companies to progress without meaningful regulation
  • AI progress exceeded predictions

AI Progress Visualization and Current State

  • Abstract landscape metaphor: elevation = task difficulty for AI to reach human level
  • Sea level = AI capabilities then vs now
  • "Sea rising fast" - many tasks already submerged
  • Water on track to submerge all land = matching human intelligence at all cognitive tasks
  • This defines AGI (Artificial General Intelligence)

AGI Timeline Acceleration

  • Companies pursuing AGI: OpenAI, Google DeepMind, Anthropic
  • Also building superintelligence beyond human intelligence
  • Few years gap predicted from AGI to superintelligence
  • Timeline shift:
    • Previously: AI researchers thought AGI decades away
    • Microsoft: "almost here" - seeing "sparks of AGI" in ChatGPT-4
    • Metaculus betting site: timeline dropped from 20 years to 3 years in last 18 months
    • Industry leaders predict 2-3 years until "we get outsmarted"

Recent AI Progress Examples

  • Robotics: from clunky movement to dancing
  • Image generation: Midjourney dramatic improvement year-over-year with same prompt
  • Deepfakes increasingly convincing (Tom Cruise example)
  • Yoshua Bengio: Large language models mastered language/knowledge, pass Turing test
  • Counter to "stochastic parrots" criticism: AIs have world representations
  • Llama-2 contains literal world map
  • AI builds geometric representations of abstract concepts (true/false)

Existential Risk Warnings

  • Alan Turing prediction: "machines take control" as default outcome
  • Superintelligence = new species, not like electricity
  • Characteristics: super capable, amoral psychopaths, don't sleep, think faster, self-replicate, nothing human
  • Industry leader warnings:
    • Sam Altman (OpenAI): "lights out for all of us"
    • Dario Amodei (Anthropic): 10-25% extinction risk
    • May: AGI CEOs and AI researchers warned about human extinction
    • EU leadership warned about AI extinction risk

Current AI Safety Limitations

  • Lack convincing AI safety plan
  • Evals (evaluations) look for risky behavior - necessary but insufficient
  • Training AI "not to say bad things" vs "not do bad things"
  • Evals can prove presence of risk, not absence of risk
  • Guardrails try to physically limit harm but inadequate against superintelligence

Provably Safe AI Vision

  • Need provably safe systems where harm is impossible
  • "Provable" in strong physics sense, not just convincing judges
  • Even superintelligent AI can't violate physics laws
  • Co-authored paper with Steve Omohundro on this approach

Technical Implementation Approach

  • Formal verification: proves properties about code
  • AI will revolutionize automatic proving and program synthesis
  • Process:
    • Human writes specification AI tool must obey
    • Examples: impossible laptop login without password, DNA printer can't synthesize dangerous viruses
    • Powerful AI creates both AI tool and proof it meets spec
    • Machine learning learns algorithms, then re-implement in verifiable architecture
  • Key insight: easier to verify proof than discover it
  • Only need to understand/trust proof-checking code (few hundred lines)
  • Envision proof checkers built into compute hardware

Alternative: AI Neuroscientist Approach

  • If AI can't write tool directly, train AI to learn desired behavior
  • Use different AI to extract learned algorithm/knowledge
  • Related to mechanistic interpretability field (rapid progress)

Practical Example: Addition Algorithm

  • Demonstrate with binary addition (like counting on 2 fingers vs 10)
  • Train recurrent neural network on addition task
  • Algorithm in black box defined by parameter tables
  • AI tool automatically distills learned algorithm into Python program
  • Dafny formal verification tool proves program correctly adds any numbers (not just training data)

Conclusion and Call to Action

  • Provably safe AI possible but requires time and work
  • Most AI benefits don't require superintelligence
  • Don't pause AI - pause reckless race to superintelligence
  • Stop training ever-larger models we don't understand
  • Heed ancient Greek hubris warning (Icarus story)
  • AI gives "incredible intellectual wings" for amazing achievements if we avoid "flying to the sun"

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome