AI summary by TwinMind

Why AI art struggles with hands

Vox

9m 56s

AI Art Hand Generation Problem Overview

  • AI can generate post-apocalyptic giraffe astronauts, Genghis Khan playing guitar, pixel art
  • Consistently fails at basic hand generation regardless of AI art model used
  • Can create Abraham Lincoln as Glam David Bowie but struggles with woman holding cell phone
  • Hand generation issues reveal broader AI art limitations

Human vs AI Learning Comparison

  • Artist consulted: teaches thousands how to draw hands from imagination
  • Human learning: pattern recognition from living in world, seeing hands constantly
  • AI learning: like being trapped in museum from birth
    • Only learns from pictures and text descriptions ("Apple - red apple on brown table")
    • Cannot physically interact with objects like humans can
    • Must find multiple museum pictures to understand objects

Artist Training vs AI Processing

  • Human artists simplify complex forms into basic shapes
    • Hand = blocky palm (front, back, thickness) + square with thickness
    • Add style, texture, detail after establishing structure
  • AI focuses on pixel patterns without understanding underlying structure
    • Excels at light and texture rendering
    • Creates bizarre shapes because doesn't understand how fingers bend
    • Guesses where "hand-like pixels" should be without knowing hand mechanics

Expert Interviews and Three Core Problems

  • Ilundu: grad student in robotics, pulled into AI art due to popularity
  • Roy Shellcrott: teaching generative art since 2018
  • Three main reasons hands are difficult for AI:
    1. Data size and quality issues
    2. How hands behave/function
    3. Low margin for error tolerance

Data Size Problem

  • Museum analogy: many rooms for faces, few for hands
  • Face datasets: Flickr HQ has 70,000 faces
  • Celebrity face dataset: 200,000 pics with detailed annotations (eyeglasses, pointy noses)
  • Hand datasets exist (11,000 hands) but may not be used for art AI training
  • Insufficient training data compared to faces

Data Quality and Complexity Issues

  • Hand data lacks detailed annotations like face data
  • Typical annotation: "person in image holding umbrella"
  • Missing detailed descriptions: thumb position, finger curl patterns, spatial relationships
  • Hands perform many more functions than faces
  • Portrait faces have consistent structure (eyes always centered)
  • Hands highly variable: different positions, visible finger counts (2-5), orientations

Variable Visibility Problem

  • Hand photos show 2-3 fingers sometimes, 4-5 other times
  • AI doesn't learn there should always be 5 fingers total
  • Similar issues with horse legs (3, 5, 6 legs generated)
  • Too much diversity without consistent bias correction

Low Margin for Error

  • Humans have high standards for hand accuracy
  • AI bias: doesn't prioritize hand perfection like humans do
  • Example: "person with exactly 5 freckles" test
    • Dall-E 2, Stable Diffusion, Midjourney all failed exact count
    • Generated freckly faces (acceptable) but not precise count
  • Hand-like isn't good enough; must be anatomically correct
  • Other imperfections acceptable: stripe-like buttons, apple variations

Current Progress and Solutions

  • Midjourney v5 shows hand improvement but not fully fixed
  • Don't ask AI to hold umbrellas (still problematic)
  • AI companies working on natural scenery, other improvements users notice

Two Main Solution Approaches

  1. More Training Data
    • Requires massive computing power increases
    • Training beyond handful of images needs tremendous resources
    • Scaling challenge for model retraining
  2. Human Feedback Integration
    • ChatGPT model: used human feedback for sentence rating
    • Fine-tuned model to generate human-convincing content
    • Potential solution: people rating generated images
    • Engineering challenge: labeling massive datasets
    • Training models to match human preferences

Broader Pattern Recognition

  • Not just hands: teeth, abs affected
  • Any pattern with "large amount of something" problematic
  • AI doesn't learn rules like "there are this many [items]"
  • Trained on different examples without consistent counting rules

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome