AI summary by TwinMind

AI’s Real Threat: Mass Manipulation

Sabine Hossenfelder

7m 34s

AI's Real Threat: Mass Manipulation Over Mass Unemployment

  • AI's primary threat = mass manipulation, not mass unemployment
  • Current AIs ideal for scammers due to personalization capabilities
  • Used for political influence, product sales, decision manipulation
  • Prevention difficult, but awareness helps avoid falling victim

Data Collection and Personal Profiling

  • Data brokers collect extensive personal information
  • Example broker claims: "most accurate identity" targeting real people vs devices/cookies
  • US data: 7,000 individual attributes per person, 75% of purchase behavior tracked
  • "Lola" example profile includes:
    • Basic identity, viewing/reading habits, household members
    • Social media follows, online/offline purchases
    • Purchase timing, location, motivation ("why she buys")

Scientific Research on AI Persuasion

  • University of Zurich study: 13 AI bots on Reddit's "change my view" forum
    • 1,700+ comments posted
    • Persuasion rate 6x higher than humans
  • GPT-4 debate study: humans vs AI on topics (school uniforms, sugar taxes)
    • With demographic information: AI won 64% of debates
    • Term: "microtargeting" - future of marketing/political lobbying
  • "Choice engineering with AI" research: using AI to engineer human decision-making

Real-World Implementation

  • Google AI Max for search (May 2025)
    • Automatically adjusts ads to match search queries
    • Increases purchase rate by 14%

AI Scam Epidemic

  • Celebrity impersonation scams:
    • French woman lost €800,000 to fake Brad Pitt (voice cloning, fake photos)
    • Similar Martin Henderson scam
  • Dating profile scams: hyperrealistic profiles, personalized "lovebombing"
  • US Federal Trade Commission (2024): 800,000+ impostor fraud reports, $3 billion losses
  • Phishing evolution: AI-generated emails 54% click-through rate (matches human experts)
  • Crypto scams: AI builds trust before theft, fake expert advice
    • Elon Musk frequently deepfaked for crypto offers
    • FBI (2024): 150,000 crypto complaints, $9+ billion losses
    • Preceded Google V3 rollout

Reality Questioning and Trust Erosion

  • Increasing skepticism about content authenticity
  • Questions about posts, images, videos being real
  • Personal note: speaker addresses avatar speculation due to lip issues

Data Protection Solutions

  • Laws exist for data protection, removal requests possible
  • Manual removal extremely time-consuming
  • Incogn service automation:
    • Contacts data brokers for removal requests
    • Continuous monitoring and removal
    • Progress updates provided
    • 60% discount with code "Zabina"

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome