AI summary by TwinMind

A camera that can see around corners | David Lindell

TED

7m 35s

Introduction and Vision for Self-Driving Cars

  • Future self-driving cars will be safer and more reliable than humans
  • Requirements for this future:
    • Technologies allowing cars to respond faster than humans
    • Algorithms that can drive better than humans
    • Cameras that can see more than humans
  • Key scenario: self-driving car making blind turn with oncoming car or child about to run into street
  • Solution: camera with "superpower" to see around corners and detect potential hazards

Research Background and Team

  • PhD student from Stanford computational imaging lab
  • Working on camera system for past few years
  • Camera can image objects hidden around corners or blocked from direct line-of-sight

Outdoor Experiment Demo

  • Camera system scanning side of building with laser
  • Target scene hidden around corner behind curtain
  • Camera system cannot see scene directly
  • Successfully captures 3D geometry of hidden scene

Ultra-High Speed Camera Technology

  • High-speed camera operating at trillion frames per second
  • Fast enough to capture movement of light itself
  • Speed comparison: light travels 3.3 nanoseconds per meter
    • In same time, superhero moving 3x speed of sound moves less than width of human hair
  • Camera captures photons at 50 trillionth of second (50 picoseconds) time frames

Technical Process

  • Ultra-high speed camera paired with laser sending short light pulses
  • Each pulse travels to visible wall, some light scatters back to camera
  • Wall used to scatter light around corner to hidden object and back
  • Measurement repeated many times to capture arrival times of many photons from different wall locations
  • Creates trillion frame per second video of wall

Light Wave Visualization

  • At trillion frames per second, can see waves of light scattered back from hidden scene
  • Waves splash against wall carrying information about hidden object
  • Measurements passed into reconstruction algorithm to recover 3D geometry of hidden scene

Indoor Scene Experiment

  • Variety of different hidden objects with different appearances
  • Objects reflect light differently:
    • Glossy dragon statue
    • Mirrored disco ball
    • White discus thrower statue
  • 3D volume visualization created by stacking video frames together
  • Time represented as depth dimension of cube
  • Bright dots: reflections from mirrored facets of disco ball
  • Bright streaks arriving soonest: from glossy dragon statue closest to wall
  • Other streaks: from book and statue reflections
  • Frame-by-frame video shows scattered light progression
  • Light waves last for just trillions of a second
  • Despite different reflection properties, can still reconstruct object shapes

Real-Time Human Tracking Demo

  • Person wearing reflective suit
  • Camera system scanning wall at 4 times per second
  • Captures enough photons to see person's location and actions
  • Photons scatter: wall → tracksuit → wall → camera
  • Indirect video captured in real-time without directly imaging person

Applications

  • Self-driving cars
  • Biomedical imaging for seeing into tiny body structures
  • Camera systems on robots for planetary exploration

Recent Advancements

  • Technology impossible just 2 years ago
  • Now can image large room-sized hidden scenes outdoors at real-time rates
  • Significant progress toward practical technology for cars

Remaining Challenges

  • Imaging hidden scenes at long distances with very few photons
  • Using low-power, eye-safe lasers
  • Creating images from photons scattered multiple times (beyond single bounce)
  • Miniaturizing large, bulky prototype system for:
    • Biomedical imaging
    • Improved home security systems
  • Exploring other applications for new imaging modality

Future Outlook

  • Exciting new technology with undiscovered applications
  • Self-driving car future may seem distant but technologies already in development
  • Rapid pace of scientific discovery and innovation
  • New capabilities could emerge unexpectedly

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome