AI summary by TwinMind

Alvy Ray Smith | A Biography of the Pixel | Talks at Google

Talks at Google

56m 6s

Introduction and Book Overview

  • Dr. Alvy Ray Smith, Pixar co-founder and computer graphics pioneer, discusses his book "A Biography of the Pixel"
  • Smith: first director of computer graphics at Lucasfilm, first graphics fellow at Microsoft, two Technical Academy Awards
  • Book examines pixel technology from multiple angles: art, technology, entertainment, business, history
  • Story spans from Fourier waves through Turing machines to first digital movies from Pixar, DreamWorks, Blue Sky

What is a Pixel - Fundamental Definition

  • Common misconception: pixels are NOT little abutting squares
  • Correct definition: pixel = sample according to sampling theorem at particular frequency representing a picture
  • Picture viewed as two-dimensional signal sampled at specific rate defined by sampling theorem
  • Can discard infinity of points in signal, maintain only samples (pixels) without losing picture information
  • Reconstruction possible: throw away infinity, not lose your picture - "mindblower" theorem

Sampling Theorem History and Attribution

  • Commonly attributed to Claude Shannon, but Smith challenges this attribution
  • Shannon never claimed he proved it first, just provided details of existing knowledge
  • Smith credits Vladimir Kotelnikov, Russian communist who proved it in 1933
  • Suggests American reluctance to credit communist for important discovery
  • Kotelnikov's theorem: sample at regular intervals in 2D, can reconstruct original signal perfectly

Fourier Theory and Wave Mathematics

  • Fourier (around 1800): all pictures are sums of regular waves (different frequencies/amplitudes)
  • Two-dimensional Fourier waves look like corrugations with sine wave cross-sections
  • Any picture (TV images, personal photos, Milky Way) = sum of regular waves
  • Fourier also discovered greenhouse effect around 1800, involved with Napoleon and Rosetta Stone
  • Modern media world based on this "mindblower" - Zoom, nearly all pictures now digital

Sampling Rate and Reconstruction

  • Magic frequency: sample at twice the highest frequency in the sum (Kotelnikov's rule)
  • Whole modern media world based on this principle
  • Perfect reconstruction possible "10 years later on other side of planet"
  • Display devices have array of "display elements" (not pixels) - analog blobs of light
  • Pixels (invisible samples) spread by display elements to reconstruct original picture

Theory vs. Engineering Reality

  • Ideal theory: spread pixels using "sinc" function (ripples to infinity)
  • Engineering reality: can't handle infinite width, use finite approximation
  • Science vs. engineering dichotomy throughout technology development
  • Works well despite approximations

Early Cinema and Naive Sampling

  • Movie pioneers (1895) approached sampling "completely naively" - no understanding of theory
  • Movies sample in time using "frames" - one after another
  • Current system: project individual frames with blackness between, brain creates motion
  • Regular movies show each frame twice (not true sampling theory)
  • Animation: sometimes each frame shown four times ("shooting on twos" - 12 fps, each frame hits retina 4 times)

Ideal Movie System Theory

  • Smith proposes sampling theorem-based movie system (never built)
  • Would sample visual flow at twice highest frequency
  • Reconstruction: several frames overlap in time, add together
  • Result: ideal reconstruction of visual flow before entering pupil
  • Would see "little bit of few frames ago, little bit of one frame ago, current frame, future frames"

Animation Language and Technical Workarounds

  • Technical limitations forced animators to develop visual language not true to reality
  • Speed lines, puffs, anticipation, squash and stretch, exaggeration
  • These became "language to the brain" - clues for proper interpretation
  • Smith's hand-clapping animation experiment: realistic version boring, exaggerated version fun
  • Technical constraints created artistic language that persists even when constraints removed

Moore's Law Impact

  • Gordon Moore announced law in 1965
  • Smith's intuitive version: "everything good about computers gets better by order of magnitude every five years"
  • Order of magnitude = factor of 10, about limit of human conceptual change
  • From 1965 to present: 100 billion x improvement factor
  • Enables reconceptualizations every five years
  • Moore's Law still continuing despite frequent death announcements
  • IBM announced 2 nanometer technology - approaching trillion x improvement

Early Pixar and Moore's Law Strategy

  • 1975 at New York Institute of Technology: conceived first digital movie goal
  • Didn't understand would take 20 years
  • Smith and Ed Catmull calculated costs on envelopes: 1975 = several billion dollars, decade timeline
  • Strategy: assume Moore's Law holds, prepare technologies and algorithms
  • Competitor Gary Demos/John Whitney Jr. bought Cray supercomputer
  • Smith calculated: economically unfeasible unless everyone signs up for Cray time (they didn't)

Pixar's Technical-Artistic Culture

  • Deliberate choice to stress equity across disciplines
  • Smith: artist background (learned oil painting from Uncle George) + computer expertise
  • Disallowed looking down on either artists or technologists
  • Required mutual admiration and dignity assignment
  • Business required both technically creative and artistically creative people working together
  • Counter-examples: graphics houses looking down on "technoids," Microsoft treating art as lesser

Jam Sessions and Collaborative Development

  • Experimental overnight sessions between technical and artistic people
  • Smith and David DiFrancisco example: alternated on single paint tablet
  • "No idea where paintings were going" - mostly junk but fun
  • Artists key to designing software interfaces
  • Iterative process: "scratch heads when they ask for things, then do it and say 'oh that's what they meant'"

Finding Nemo Technical Restraint Example

  • Originally created underwater world as realistic as real underwater world
  • Had to "back it off" because gorgeous visuals took attention from characters
  • Deliberate decision: story and character animation first, technology as tool
  • "Don't make it quite so good" - blur it down

Recommendations for Future Creators

  • SIGGRAPH as primary recommendation for 40+ years
  • Annual conference where all industry players show off to peers
  • At height: 40,000 attendees yearly
  • Technical papers, artistic productions, art shows, educational content
  • "Rich environment" for meeting key people and seeing cutting edge

Andre and Wally B at SIGGRAPH 1984

  • Demonstrated solved motion blur problem
  • John Lasseter's first work with group (later Pixar)
  • 5,000 SIGGRAPH attendees understood significance and "screamed heads off"
  • George Lucas attended (visiting girlfriend) but didn't understand breakthrough
  • Lucas saw crude aspects, SIGGRAPH audience filled in blanks with Moore's Law projection
  • Good management: Lucas didn't share disappointment with team

The Movie Goal and Toy Story Path

  • Goal since 1975: first completely digital movie in world
  • Didn't know format (2D/3D) or content, just completely computer-generated
  • Path that worked: new 3D modeling approach nobody had seen
  • "Flipped everybody out" and changed moviemaking world
  • Five years as hardware company waiting for Moore's Law to enable movie production
  • Japanese Monkey King project calculations showed still too expensive/time-consuming

Historical Research Challenges

  • Surprised by "lousy histories of technologies" - expected to "look it up and write it down"
  • Every looked-up source was wrong
  • Examples: sampling theorem attribution, movie invention, television invention
  • Avoided "simple narrative" of single genius having idea
  • Developed flowcharts/genealogies showing complex interactions, borrowing, stealing
  • "Much more satisfying" and truthful approach to technological history

Audience Q&A Highlights

  • Windup toys question: directed to John Lasseter as actual collector
  • Screen element limits: Moore's Law doesn't apply to display elements (have physical size requirements)
  • AI in entertainment: Smith's 1965 Stanford AI experience, changed to achievable goals
  • Recent AI developments impressive but maintains skepticism due to marketing history
  • "Zebras and horses" paper example: AI solving problems too complex for human understanding
  • Definition: "if you can understand it, it's not going to be intelligent"

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome