AI summary by TwinMind
Institute for Extinction Risk Shuts Down: What We Know
Sabine Hossenfelder
7m 26s
Institute Background and Founding
- Future of Humanity Institute shut down earlier this year, announced last week
- Founded 2005 by Nick Bostrom at University of Oxford, UK
- One of few places worldwide studying human extinction risk
- Bostrom believes we live in computer simulation
Major Work and AI Focus
- Most impactful work: putting AI on agenda as existential threat to human species
- Achievement occurred ~10 years ago
- Positioned AI as potentially species-ending threat
Effective Altruism Connection
- Institute became strongly linked to effective altruism movement ~10 years ago
- Key philosophers: Hillary Greaves, William MacAskill (both Oxford-based)
- Effective altruism: research area using economic reasoning to find most effective ways of improving world
Long-termism Philosophy
- Controversial idea arising from effective altruism, strongly represented at Institute
- Core premise: focus on benefiting long-term human development
- Rationale: many more humans will live in future than present
- Population illustration: each grain of sand = 10 million people
- Green grains: ~8 billion alive today
- Red grains: ~110 billion lived in past
- Vast majority: future lives yet to come
- MacAskill 2019 paper: "can often simply ignore all effects contained in first 100 or even 1,000 years, focusing primarily on further future effects"
- Current lives only matter for producing sufficient offspring to avoid extinction
- With 8 billion people, "could spare a few billion"
Criticism and Reception
- Gives rich people excuse to pursue future visions while ignoring current suffering
- Elon Musk found long-termism compatible with his thinking
- Critics called it anti-democratic and anti-humanistic
- Many effective altruism supporters distanced themselves from long-termism
Administrative Problems and Closure
- Major issue: administrative problems with host institution
- Final report cited "gradual suffocation by faculty bureaucracy"
- Institute's "flexible fast moving approach" incompatible with "rigid rules and slow decision-making"
- "Several serious missteps in communications with other parts of University"
- Misunderstood how messages would be received
- Specific issue: Institute members not allowed to nap in offices (per Anders Sandberg)
- 2020: faculty imposed freeze on fundraising and hiring
- Freeze "basically spelled death of Institute"
Nick Bostrom Controversy
- 2023: Bostrom drew unwanted attention over 26-year-old email text
- Text expressed opinion that black people are intellectually inferior
- Apology mainly elaborated on how his views are "so much more intelligent now"
- Episode likely killed any remaining hopes for Institute survival
- Apology reflected same spirit as final report: "we are so advanced that these intellectually inferior normal people just can't comprehend our greatness"
Related Organizations
- Future of Humanity Institute ≠ Future of Life Institute
- Future of Life Institute: Max Tegmark's organization
- Published open letter requesting AI development pause (not implemented)
- Continues AI extinction warnings despite lack of concrete action
Broader Implications
- Lesson: even if humanity doesn't have future, life will continue
- Science remains crucial for understanding current situation and solving created problems
