AI summary by TwinMind

Institute for Extinction Risk Shuts Down: What We Know

Sabine Hossenfelder

7m 26s

Institute Background and Founding

  • Future of Humanity Institute shut down earlier this year, announced last week
  • Founded 2005 by Nick Bostrom at University of Oxford, UK
  • One of few places worldwide studying human extinction risk
  • Bostrom believes we live in computer simulation

Major Work and AI Focus

  • Most impactful work: putting AI on agenda as existential threat to human species
  • Achievement occurred ~10 years ago
  • Positioned AI as potentially species-ending threat

Effective Altruism Connection

  • Institute became strongly linked to effective altruism movement ~10 years ago
  • Key philosophers: Hillary Greaves, William MacAskill (both Oxford-based)
  • Effective altruism: research area using economic reasoning to find most effective ways of improving world

Long-termism Philosophy

  • Controversial idea arising from effective altruism, strongly represented at Institute
  • Core premise: focus on benefiting long-term human development
  • Rationale: many more humans will live in future than present
  • Population illustration: each grain of sand = 10 million people
    • Green grains: ~8 billion alive today
    • Red grains: ~110 billion lived in past
    • Vast majority: future lives yet to come
  • MacAskill 2019 paper: "can often simply ignore all effects contained in first 100 or even 1,000 years, focusing primarily on further future effects"
  • Current lives only matter for producing sufficient offspring to avoid extinction
  • With 8 billion people, "could spare a few billion"

Criticism and Reception

  • Gives rich people excuse to pursue future visions while ignoring current suffering
  • Elon Musk found long-termism compatible with his thinking
  • Critics called it anti-democratic and anti-humanistic
  • Many effective altruism supporters distanced themselves from long-termism

Administrative Problems and Closure

  • Major issue: administrative problems with host institution
  • Final report cited "gradual suffocation by faculty bureaucracy"
  • Institute's "flexible fast moving approach" incompatible with "rigid rules and slow decision-making"
  • "Several serious missteps in communications with other parts of University"
  • Misunderstood how messages would be received
  • Specific issue: Institute members not allowed to nap in offices (per Anders Sandberg)
  • 2020: faculty imposed freeze on fundraising and hiring
  • Freeze "basically spelled death of Institute"

Nick Bostrom Controversy

  • 2023: Bostrom drew unwanted attention over 26-year-old email text
  • Text expressed opinion that black people are intellectually inferior
  • Apology mainly elaborated on how his views are "so much more intelligent now"
  • Episode likely killed any remaining hopes for Institute survival
  • Apology reflected same spirit as final report: "we are so advanced that these intellectually inferior normal people just can't comprehend our greatness"

Related Organizations

  • Future of Humanity Institute ≠ Future of Life Institute
  • Future of Life Institute: Max Tegmark's organization
  • Published open letter requesting AI development pause (not implemented)
  • Continues AI extinction warnings despite lack of concrete action

Broader Implications

  • Lesson: even if humanity doesn't have future, life will continue
  • Science remains crucial for understanding current situation and solving created problems

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome