AI summary by TwinMind

The Dark Side of Competition in AI | Liv Boeree | TED

TED

12m 3s

Competition and Human Nature

  • Competition fundamental part of human nature
  • Speaker: professional poker player for 10 years, witnessed good/bad manifestations
  • Healthy competition drives incredible feats in sports and innovation
    • Car companies competing for safest cars
    • Most efficient solar panels
    • Creates win-win outcomes where everyone benefits long-term
  • Unhealthy competition creates lose-lose outcomes where everyone worse off

AI Beauty Filters Example

  • Impressive technology can salvage almost any picture
  • Can make Angelina Jolie and Margot Robbie "more beautiful"
  • Handy for influencers - transform into Hollywood versions at click of button
  • Problems:
    • Train users to hate natural face
    • Growing evidence of body dysmorphia, especially young people
    • Endemic to social media due to game nature
  • Platform incentives: hotter pictures = hijacked limbic systems = more scrolling = more ad revenue
  • User incentives: hotter pictures = more followers
  • Trap: hard to go back once started, no competitive advantage since everyone using them
  • Influencers stuck using with downsides, little upside

News Media Race to Bottom

  • Expected internet competition would create positive spiral toward nuanced, impartial, accurate journalism
  • Instead: race to bottom of clickbait and polarization
  • Even respectable papers leaning into low-brow partisan tactics
  • Root cause: crappy incentives
    • News now interactive (sharing, commenting)
    • Headlines triggering fear/anger far more likely to go viral than neutral/positive
  • News editors in similar trap as influencers
    • More competitors use clickbait tactics, more they must too
    • Otherwise stories get lost in noise
  • Results: media gets less public trust, harder to discern truth from falsehood
  • Big problem for democracy

Broader Pattern of Competition Gone Wrong

  • Driving force behind major issues:
    • Plastic pollution
    • Deforestation
    • Antibiotic overuse in farming
    • Arms races
    • Greenhouse gas emissions
  • All result from crappy incentives of poorly designed games
  • Push players (people, companies, governments) into strategies deferring costs/harms to future
  • Most actors don't want to be doing this
    • Packaging companies don't want oceans filled with plastic
    • Farmers don't want antibiotic resistance
  • Stuck in dilemma: "If I don't use this tactic, I'll get outcompeted by others who do"

Root Cause Analysis

  • Not capitalism (can cause problems but also solve them, fantastic in general)
  • Something deeper: misaligned incentives of Game Theory itself
  • Speaker retired from poker to understand mechanism better
  • Takes many forms, many names (abstract and clunky terms)
  • Need visceral way of recognizing given how insidious and connected problems are

Moloch Metaphor

  • Biblical story from Canaanite days
  • Cult wanted money and power badly enough to sacrifice literal children
  • Burned children alive in effigy of god Moloch, believing would reward ultimate sacrifice
  • Apt metaphor: getting lost in winning immediate game, losing sight of bigger picture
  • Sacrificing too much in pursuit of victory
  • Examples of modern sacrifices:
    • Influencers: happiness for likes
    • News editors: integrity for clicks
    • Polluters: biosphere for profit
  • Short-term game incentives push players to sacrifice more of future
  • Traps in death spiral where all lose in end
  • Moloch's trap = mechanism of unhealthy competition

AI Industry Moloch Trap

  • Race heating up between companies over:
    • Most compute
    • Biggest funding rounds
    • Top talent
  • More companies enter race = greater pressure to go fast as possible
  • Sacrifice other important stuff like safety testing
  • Classic Moloch trap hallmarks
  • CEO dilemma: believe team best to safely build powerful AI
    • Go too slowly = risk less cautious teams getting there first
    • Pushes to be more reckless
  • Many experts/researchers (within companies and independent) warning about extreme risks of rushed AI
  • Approach "absolutely mad"
  • AI companies beholden to satisfying investors (short-term incentive)
  • Will inevitably conflict with benevolent mission over time
  • AI/AGI = bigger paradigm shift than agricultural/industrial revolutions
  • Moment deserving reverence and reflection, not corporate rat race for daily active users
  • Don't know right trade-off between acceleration and safety
  • Will never find out if let Moloch dictate

Solutions and Coordination Examples

  • Have managed to escape some Moloch traps before:
    • Saved ozone layer from CFCs with Montreal Protocol
    • Reduced nuclear weapons on Earth by 80% with Strategic Arms Reduction Treaty 1991
  • Smart regulation may help with AI
  • Players within game have most influence
  • Need AI leaders to show awareness of:
    • Risks their technologies pose
    • Destructive nature of current incentives
  • As technological capabilities reach power of gods, need godlike wisdom to wield them
  • Discouraging when major company CEO says "I want people to know we made our competitor dance"
  • Not the mindset needed
  • Need leaders willing to flip Moloch's playbook
  • Sacrifice individual chance of winning for good of whole

Current AI Lab Initiatives

  • Three leading labs showing some signs:
    • Anthropic: responsible scaling policy - only increase capabilities once security criteria met
    • OpenAI: pledged 20% of compute purely to research
    • DeepMind: decade-long focus science ahead of commerce (AlphaFold given free to science community)
  • Steps in right direction but nowhere close to enough
  • Most currently just words, not proven actions
  • Need clear way to turn AI race into definitive race to top
  • Companies could compete over:
    • Best security criteria
    • Most compute dedicated to alignment
  • Would truly flip middle finger to Moloch

Stakes and Conclusion

  • Competition amazing tool if wielded wisely
  • Stakes astronomical:
    • Get AI/AGI wrong = unimaginable catastrophe
    • Get it right = path out of many Moloch traps
  • As things get crazier in coming years, important to remember:
    • Moloch is real enemy, not individual CEOs/companies
    • Not one another
  • "Don't hate the players, change the game"

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome