AI summary by TwinMind

What If Someone Steals GPT-4?

Asianometry

18m 24s

LLM Architecture and Economic Value

  • LLMs consist of two files: ~500 lines of C language code + hundreds of billions/trillions of parameters
  • Parameters are seemingly random numbers but contain the "magic"
  • More parameters + more training tokens = more capability
  • Models are economically valuable, carry proprietary trade secrets
  • When separated from safety systems, can exhibit malicious capabilities
  • Training data also valuable, produced at considerable cost by educated workers
  • Commercial incentive for state actors, unethical AI labs, activists to steal from leaders

Model and Data Sizes

  • GPT-2: 1.5 billion parameters, 5GB
  • GPT-3: 175 billion parameters, 800GB
  • GPT-4: probably >1 trillion parameters, 1-2 terabytes
  • Common Crawl training dataset: 45 terabytes before compression
  • Size defines cybersecurity profile

Traditional Exfiltration Techniques

  • Encoding stolen data within images/videos using steganography
    • Hide 50,000 line CSV in 5MB PNG file (adds ~21MB)
    • Zip and email attachment method
  • Encoding within email headers or TCP/IP protocol option fields
    • Max 40 bytes can be encoded in TCP headers
  • Companies use heuristic scanning to detect suspicious traffic
  • Gigabytes/terabytes have been exfiltrated before
  • Traditional methods may be too slow for GPT-3/4 scale

Wide Distribution Challenge

  • LLMs must be embedded in everyday lives for success
  • Microsoft leading with GPT-4 powered products: Co-pilot, Co-pilot Studio
  • All run on same core GPT-4 foundation model per CEO Satya Nadella
  • Models too large for single GPU, split across ~24 GPUs
  • Many copies floating around data centers
  • Creates multiple attack vectors

Data Security States

Data at Rest

  • Stored on disk, doing nothing
  • Can encrypt with strong algorithms
  • Split across isolated storage spaces
  • Well-understood security practices

Data in Transit

  • Traveling from disk to another location
  • Higher security risks (man-in-the-middle attacks)
  • SSH (Secure Shell) standard for remote connections
  • Automatic key pairs encrypt connections
  • Data can be encrypted before transmission
  • Widely accepted protection methods exist

Data in Use

  • Least explored security area
  • During inference: model runs repeatedly, generating/appending tokens
  • Data primarily in GPU memory, moving in/out of registers
  • Vulnerable to physical access attacks

Physical Access Attacks

  • Memory bus monitoring: intercept data traveling between GPU memory and processing cores
  • Memory probe attacks: retrieve model from system RAM
  • Previously not major concern due to physical access requirement
  • Cloud computing changed threat landscape

Cloud Computing Vulnerabilities

  • Model providers often rent compute time from AWS, Microsoft, Oracle
  • Only way for smaller companies to access expensive AI accelerators
  • Attackers can breach third-party cloud centers instead of labs
  • Large cloud providers have better security practices but are big organizations
  • Low-level employees/temporary workers can gain unsupervised access
  • Particularly vulnerable during server installation/decommissioning

Performance vs Security Trade-offs

  • Most obvious protections negatively affect performance
  • iPhone's Secure Enclave creates trusted environment but isolated
  • Isolation creates bottlenecks, compounds Von Neumann bottleneck
  • Encrypting GPU memory can hurt latency by up to 50%
  • ChatGPT already considered too slow by users
  • Companies unlikely to accept significant performance degradation

Confidential Computing Consortium

  • Launched 2019, sits within Linux Foundation
  • Founding members: Alibaba, ARM, Huawei, Intel, Microsoft, Red Hat
  • Later joined by Google, Nvidia, Samsung, Meta, others
  • Develops hardware-based solutions for data in use protection

Trusted Execution Environment (TEE)

  • Core solution from Confidential Computing Consortium
  • Two main functions:
    1. Host applications in isolated environment (like Apple's Secure Enclave)
    2. Issue verifiable attestations about running programs
  • Protects from external access even by high-privilege users
  • Attestations act like letters of credit for trust verification
  • Built on "root of trust" - standalone module with cryptographic keys

CPU and GPU Implementation

  • First TEEs for CPUs from Intel, AMD, later ARM
  • Confidential Computing CPUs separate entire virtual machines
  • Memory management units configured with encryption engines
  • Isolate certain memory pages, cause page faults if accessed incorrectly
  • GPU adoption came later
  • A100 had some confidential compute aspects
  • H100 first full confidential compute GPU solution

Nvidia H100 Confidential Computing

  • Memory split into protected/unprotected parts
  • In confidential mode: nothing enters protected memory
  • Programs in TEE cannot easily write outside protected area
  • Requires compatible CPU cooperation for verification
  • All GPU-CPU communications encrypted
  • Minimal performance loss compared to A100
  • Massive step change in performance from A100

Remote Model Extraction Attacks

  • Goal: extract parameters to recreate target LLM
  • Train "student model" that learns from "master" through Q&A
  • Use commercial API endpoints to fire inputs and collect outputs
  • May get cut off before complete extraction
  • Focused attacks require less data for decent approximation

Model Leeching Case Study

  • Lancaster University research on specific topic extraction
  • Used 100M parameter model over 48 hours
  • 83,535 API requests costing $50
  • 76,000 valid responses generated model with 75-87% similarity to GPT-3.5

Membership Inference Attacks

  • Determine if certain dataset present in training data
  • Exploit LLM tendency to memorize and regurgitate training data
  • Knowing training data composition can enable model replication
  • Privacy concerns: reveal if person's data included in training
  • Copyright issues: expose illegally acquired copyrighted works in training sets
  • Laws around such data use remain murky, lawsuits filed

Insider Attack Threats

  • Concern for assets of interest to nation states
  • Nation states likely to recruit individuals with leverage
  • Historical examples: FBI 2019 case of Saudi recruitment of Twitter employees
  • Government distrust of Chinese Americans in academia/government
  • 83% of data breaches instigated by outsiders (Verizon 2023 report)
  • Mitigation through vetting, encryption, good cybersecurity practices
  • America strengthened by immigrant contributions, many AI leaders born in China

Inevitability of Breaches

  • Cybersecurity best practices for LLMs still being developed
  • Major AI labs (Anthropic, OpenAI, others) will eventually get hacked
  • All cybersecurity elites have been hacked: CIA, Israeli Army, US government
  • China and Iran likely suffered attacks but don't discuss
  • Nothing special about AI data protection
  • LLMs might enable greater attacker productivity (enhanced phishing, etc.)
  • Should consider ramifications of GPT-4 theft/open release
  • Timeline probably sooner than expected

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome