AI summary by TwinMind
What If Someone Steals GPT-4?
Asianometry
18m 24s
LLM Architecture and Economic Value
- LLMs consist of two files: ~500 lines of C language code + hundreds of billions/trillions of parameters
- Parameters are seemingly random numbers but contain the "magic"
- More parameters + more training tokens = more capability
- Models are economically valuable, carry proprietary trade secrets
- When separated from safety systems, can exhibit malicious capabilities
- Training data also valuable, produced at considerable cost by educated workers
- Commercial incentive for state actors, unethical AI labs, activists to steal from leaders
Model and Data Sizes
- GPT-2: 1.5 billion parameters, 5GB
- GPT-3: 175 billion parameters, 800GB
- GPT-4: probably >1 trillion parameters, 1-2 terabytes
- Common Crawl training dataset: 45 terabytes before compression
- Size defines cybersecurity profile
Traditional Exfiltration Techniques
- Encoding stolen data within images/videos using steganography
- Hide 50,000 line CSV in 5MB PNG file (adds ~21MB)
- Zip and email attachment method
- Encoding within email headers or TCP/IP protocol option fields
- Max 40 bytes can be encoded in TCP headers
- Companies use heuristic scanning to detect suspicious traffic
- Gigabytes/terabytes have been exfiltrated before
- Traditional methods may be too slow for GPT-3/4 scale
Wide Distribution Challenge
- LLMs must be embedded in everyday lives for success
- Microsoft leading with GPT-4 powered products: Co-pilot, Co-pilot Studio
- All run on same core GPT-4 foundation model per CEO Satya Nadella
- Models too large for single GPU, split across ~24 GPUs
- Many copies floating around data centers
- Creates multiple attack vectors
Data Security States
Data at Rest
- Stored on disk, doing nothing
- Can encrypt with strong algorithms
- Split across isolated storage spaces
- Well-understood security practices
Data in Transit
- Traveling from disk to another location
- Higher security risks (man-in-the-middle attacks)
- SSH (Secure Shell) standard for remote connections
- Automatic key pairs encrypt connections
- Data can be encrypted before transmission
- Widely accepted protection methods exist
Data in Use
- Least explored security area
- During inference: model runs repeatedly, generating/appending tokens
- Data primarily in GPU memory, moving in/out of registers
- Vulnerable to physical access attacks
Physical Access Attacks
- Memory bus monitoring: intercept data traveling between GPU memory and processing cores
- Memory probe attacks: retrieve model from system RAM
- Previously not major concern due to physical access requirement
- Cloud computing changed threat landscape
Cloud Computing Vulnerabilities
- Model providers often rent compute time from AWS, Microsoft, Oracle
- Only way for smaller companies to access expensive AI accelerators
- Attackers can breach third-party cloud centers instead of labs
- Large cloud providers have better security practices but are big organizations
- Low-level employees/temporary workers can gain unsupervised access
- Particularly vulnerable during server installation/decommissioning
Performance vs Security Trade-offs
- Most obvious protections negatively affect performance
- iPhone's Secure Enclave creates trusted environment but isolated
- Isolation creates bottlenecks, compounds Von Neumann bottleneck
- Encrypting GPU memory can hurt latency by up to 50%
- ChatGPT already considered too slow by users
- Companies unlikely to accept significant performance degradation
Confidential Computing Consortium
- Launched 2019, sits within Linux Foundation
- Founding members: Alibaba, ARM, Huawei, Intel, Microsoft, Red Hat
- Later joined by Google, Nvidia, Samsung, Meta, others
- Develops hardware-based solutions for data in use protection
Trusted Execution Environment (TEE)
- Core solution from Confidential Computing Consortium
- Two main functions:
- Host applications in isolated environment (like Apple's Secure Enclave)
- Issue verifiable attestations about running programs
- Protects from external access even by high-privilege users
- Attestations act like letters of credit for trust verification
- Built on "root of trust" - standalone module with cryptographic keys
CPU and GPU Implementation
- First TEEs for CPUs from Intel, AMD, later ARM
- Confidential Computing CPUs separate entire virtual machines
- Memory management units configured with encryption engines
- Isolate certain memory pages, cause page faults if accessed incorrectly
- GPU adoption came later
- A100 had some confidential compute aspects
- H100 first full confidential compute GPU solution
Nvidia H100 Confidential Computing
- Memory split into protected/unprotected parts
- In confidential mode: nothing enters protected memory
- Programs in TEE cannot easily write outside protected area
- Requires compatible CPU cooperation for verification
- All GPU-CPU communications encrypted
- Minimal performance loss compared to A100
- Massive step change in performance from A100
Remote Model Extraction Attacks
- Goal: extract parameters to recreate target LLM
- Train "student model" that learns from "master" through Q&A
- Use commercial API endpoints to fire inputs and collect outputs
- May get cut off before complete extraction
- Focused attacks require less data for decent approximation
Model Leeching Case Study
- Lancaster University research on specific topic extraction
- Used 100M parameter model over 48 hours
- 83,535 API requests costing $50
- 76,000 valid responses generated model with 75-87% similarity to GPT-3.5
Membership Inference Attacks
- Determine if certain dataset present in training data
- Exploit LLM tendency to memorize and regurgitate training data
- Knowing training data composition can enable model replication
- Privacy concerns: reveal if person's data included in training
- Copyright issues: expose illegally acquired copyrighted works in training sets
- Laws around such data use remain murky, lawsuits filed
Insider Attack Threats
- Concern for assets of interest to nation states
- Nation states likely to recruit individuals with leverage
- Historical examples: FBI 2019 case of Saudi recruitment of Twitter employees
- Government distrust of Chinese Americans in academia/government
- 83% of data breaches instigated by outsiders (Verizon 2023 report)
- Mitigation through vetting, encryption, good cybersecurity practices
- America strengthened by immigrant contributions, many AI leaders born in China
Inevitability of Breaches
- Cybersecurity best practices for LLMs still being developed
- Major AI labs (Anthropic, OpenAI, others) will eventually get hacked
- All cybersecurity elites have been hacked: CIA, Israeli Army, US government
- China and Iran likely suffered attacks but don't discuss
- Nothing special about AI data protection
- LLMs might enable greater attacker productivity (enhanced phishing, etc.)
- Should consider ramifications of GPT-4 theft/open release
- Timeline probably sooner than expected
