AI summary by TwinMind
Why AI art struggles with hands
Vox
9m 56s
AI Art Hand Generation Problem Overview
- AI can generate post-apocalyptic giraffe astronauts, Genghis Khan playing guitar, pixel art
- Consistently fails at basic hand generation regardless of AI art model used
- Can create Abraham Lincoln as Glam David Bowie but struggles with woman holding cell phone
- Hand generation issues reveal broader AI art limitations
Human vs AI Learning Comparison
- Artist consulted: teaches thousands how to draw hands from imagination
- Human learning: pattern recognition from living in world, seeing hands constantly
- AI learning: like being trapped in museum from birth
- Only learns from pictures and text descriptions ("Apple - red apple on brown table")
- Cannot physically interact with objects like humans can
- Must find multiple museum pictures to understand objects
Artist Training vs AI Processing
- Human artists simplify complex forms into basic shapes
- Hand = blocky palm (front, back, thickness) + square with thickness
- Add style, texture, detail after establishing structure
- AI focuses on pixel patterns without understanding underlying structure
- Excels at light and texture rendering
- Creates bizarre shapes because doesn't understand how fingers bend
- Guesses where "hand-like pixels" should be without knowing hand mechanics
Expert Interviews and Three Core Problems
- Ilundu: grad student in robotics, pulled into AI art due to popularity
- Roy Shellcrott: teaching generative art since 2018
- Three main reasons hands are difficult for AI:
- Data size and quality issues
- How hands behave/function
- Low margin for error tolerance
Data Size Problem
- Museum analogy: many rooms for faces, few for hands
- Face datasets: Flickr HQ has 70,000 faces
- Celebrity face dataset: 200,000 pics with detailed annotations (eyeglasses, pointy noses)
- Hand datasets exist (11,000 hands) but may not be used for art AI training
- Insufficient training data compared to faces
Data Quality and Complexity Issues
- Hand data lacks detailed annotations like face data
- Typical annotation: "person in image holding umbrella"
- Missing detailed descriptions: thumb position, finger curl patterns, spatial relationships
- Hands perform many more functions than faces
- Portrait faces have consistent structure (eyes always centered)
- Hands highly variable: different positions, visible finger counts (2-5), orientations
Variable Visibility Problem
- Hand photos show 2-3 fingers sometimes, 4-5 other times
- AI doesn't learn there should always be 5 fingers total
- Similar issues with horse legs (3, 5, 6 legs generated)
- Too much diversity without consistent bias correction
Low Margin for Error
- Humans have high standards for hand accuracy
- AI bias: doesn't prioritize hand perfection like humans do
- Example: "person with exactly 5 freckles" test
- Dall-E 2, Stable Diffusion, Midjourney all failed exact count
- Generated freckly faces (acceptable) but not precise count
- Hand-like isn't good enough; must be anatomically correct
- Other imperfections acceptable: stripe-like buttons, apple variations
Current Progress and Solutions
- Midjourney v5 shows hand improvement but not fully fixed
- Don't ask AI to hold umbrellas (still problematic)
- AI companies working on natural scenery, other improvements users notice
Two Main Solution Approaches
- More Training Data
- Requires massive computing power increases
- Training beyond handful of images needs tremendous resources
- Scaling challenge for model retraining
- Human Feedback Integration
- ChatGPT model: used human feedback for sentence rating
- Fine-tuned model to generate human-convincing content
- Potential solution: people rating generated images
- Engineering challenge: labeling massive datasets
- Training models to match human preferences
Broader Pattern Recognition
- Not just hands: teeth, abs affected
- Any pattern with "large amount of something" problematic
- AI doesn't learn rules like "there are this many [items]"
- Trained on different examples without consistent counting rules
