AI summary by TwinMind

619. How to Poison the A.I. Machine | Freakonomics Radio

Freakonomics Radio Network

52m 5s

Introduction and Context Setting

  • Old saying: "imitation is sincerest form of flattery" but can tip into forgery
  • Historical example: Dutch painter Han Von Meeran forged Johan Vermeer works, sold to Nazis in 20th century
  • New kind of art forgery happening with machines as perpetrators
  • Host recently returned from San Francisco AI boom epicenter
  • American Economic Association conference filled with AI sessions
  • Economist Eric Brynjolfsson cited book by Michael Waldridge "A Brief History of Artificial Intelligence" (2021)
  • AI solved problems thought impossible: story understanding, translation, photo interpretation
  • Congressional Budget Office estimates 1.5% annual economic growth current decade
  • Brynjolfsson thinks AI could double that growth rate
  • Key insight: machines should complement humans, not just imitate/replace

Ben Xiao Background and Lab Setup

  • Professor of computer science at University of Chicago
  • Runs SAND Lab (Security Algorithms Networking and Data) with wife Heather Jang
  • Mission: use technology for good, limit harms/abuses, protect human beings
  • Lab setup: anticlimactic cubicles with standing desks, 7-8 PhD students
  • Xiao sits among students, not in separate office
  • Works on adversarial machine learning, tools to mitigate AI harms against human creatives
  • Focus shifted 2+ years ago to protecting artists/writers from AI misuse

Previous Tools: Bracelet of Silence and Fawkes

  • Bracelet of Silence: cyberpunk-style bracelet with 24/12 transducers
    • Radiates ultrasonic pulses to jam commodity microphones
    • Makes microphones vibrate at audible frequencies
    • Popular with privacy advocates, public figures
  • Fawkes (2019): facial recognition protection tool
    • Alters selfies to look different to AI models while appearing normal to humans
    • Developed before deep fakes, focused on surveillance concerns
    • Over 1 million downloads
    • Clearview AI story broke week of paper submission, validating concerns
    • Clearview now only works with government/law enforcement

Artist Outreach and Problem Identification

  • Belgian artist Kim Van Dun reached out via Fawkes mailing list
  • Invited to town hall meeting by Concept Art Association (Los Angeles)
  • Featured US copyright office representatives
  • Artists noticed AI knockoffs appearing in search results
  • Original images scraped from internet to train AI models
  • Text-to-image models: Dolly (OpenAI), Imagen (Google), Image Playground (Apple), Stable Diffusion (Stability AI), Midjourney
  • Companies run scrapers to collect high-quality imagery online
  • Opt-out programs from companies like OpenAI are "toothless"
    • No way to prove compliance
    • Billions at stake, little incentive to honor opt-outs

Impact on Artists

  • Identity theft: decades to develop artistic style, taken without permission
  • Job displacement: studios laying off artist teams
  • Remaining artists told to use AI tools like Midjourney
  • Core issue: right of publicity, ownership of identity/skills/style

Power Dynamics and Corporate Leverage

  • AI companies have trillions in market cap
  • Richest companies on planet with unlimited resources
  • Comparison to early 2000s music piracy
    • Then: copyright holders had power/lawyers
    • Now: "bad guys" have all the lawyers
  • Silicon Valley "ask forgiveness not permission" approach
  • Travis Kalanick/Uber model: establish first, deal with lawsuits later
  • Legal costs are "drop in bucket" for these companies

Market Context

  • "Magnificent Seven" stocks (Alphabet, Amazon, Apple, Meta, Microsoft, Nvidia, Tesla) rose 60%+ in 2024
  • Represent 33% of S&P 500 value
  • AI data centers need 60 gigawatts energy capacity
  • Enough to power 1/3 of US homes
  • Companies investing in nuclear power: OpenAI, Amazon, Google, Meta, Microsoft
  • Microsoft plans to revive Three Mile Island

Glaze and Nightshade Tools

Glaze

  • Protects individual artists from mimicry
  • Prevents third parties from fine-tuning local models on artist's work
  • Targets individual users, not training companies

Nightshade

  • More far-reaching tool
  • Makes training on scraped data expensive
  • Goal: push AI companies toward legitimate licensing deals
  • Takes images, alters them invisibly to humans
  • AI models see entirely different visual features
  • Example: cow image teaches AI that cows have "four wheels, big hood, fender, trunk"
  • Nightshade images are "potent" - few hundred can override hundreds of thousands of normal images
  • Result: AI generates cars when asked for cows

Technical Implementation

  • Nightshade chooses targets (cow → pickup truck vs cactus)
  • Requires consistency across all users globally
  • Prevents user choice to maintain poison strength
  • Simple for artists: set intensity parameters, hit go
  • Output looks nearly identical with tiny artifacts
  • Over 2 million downloads of Nightshade

AI Industry Response and Effectiveness

  • OpenAI spokesperson called data poison "type of abuse"
  • Effects likely detected in model training, causing delays/costs
  • Not about detecting individual images but billions of images quickly
  • Goal: make scraped data training more expensive than legitimate licensing
  • Won't see obvious failures in wild due to professional testing
  • Will cause rollbacks, delays, increased costs

AI Bubble and Limitations

  • Xiao believes AI bubble "bursting right now"
  • Models hitting training data limits
  • Need double amount of data ever created by humanity
  • Recent reports: Google, OpenAI struggling with next-generation models
  • Plateauing effect hitting AI growth
  • Sam Altman tweeted "there is no wall" in response
  • Few legitimate revenue-generating applications
  • Investment doesn't match potential returns
  • "Great interpolation/mimicry machines" with limited applications
  • Won't produce full movies/TV shows/books of human-desired value

Legal Landscape

  • Multiple lawsuits in progress:
    • OpenAI and Microsoft sued by New York Times
    • Anthropic fighting Universal Music claims
    • Artists suing Stability AI, Midjourney, DeviantArt
  • Art argument beyond intellectual property
  • Art needs intention, meaning, context
  • AI replication loses the point
  • Not automation analogy (horse/buggy → automobile)
  • AI takes past samples, "shakes in kaleidoscope"

Personal Motivation and Academic Position

  • Xiao's privilege as senior tenured professor
  • No pressure for reputation/establishment
  • No commercial interest in tools
  • All grants public, regular funding levels
  • Products have no money attached
  • Academic model: develop ideas, others commercialize
  • Considering nonprofit organization structure
  • Goal: be first to walk path, encourage others to follow

Future Vision

  • AI and human creativity need complementary relationship
  • AI needs human work product to survive
  • Humans need compensation for their work
  • If human artistry dies, AI becomes stale
  • Fair system: AI stays out of creative domain
  • Humans do imaginative work, AI handles conversational chatbots
  • Creativity is "best of humanity"

Student Conversations and Advice

  • Regular office hours discussions with CS majors
  • Students torn between harm awareness and job opportunities
  • Xiao's advice: don't get swept by hype
  • When tide turns, clear-headed students will be better positioned
  • Humans will be shown better than AI pretends to be
  • Bubble limitations becoming apparent
  • Truth emerging about realistic value generation

Whistleblower Context

  • Suchir Balaji (26) former OpenAI researcher died by suicide
  • Publicly charged OpenAI with copyright violations
  • Left company over ethical concerns
  • Enormous courage required for whistleblowing
  • Tremendous risk against powerful employers
  • Absence of courage in business world
  • Ethics courses taught because "things already in tough shape"

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome