AI summary by TwinMind
A camera that can see around corners | David Lindell
TED
7m 35s
Introduction and Vision for Self-Driving Cars
- Future self-driving cars will be safer and more reliable than humans
- Requirements for this future:
- Technologies allowing cars to respond faster than humans
- Algorithms that can drive better than humans
- Cameras that can see more than humans
- Key scenario: self-driving car making blind turn with oncoming car or child about to run into street
- Solution: camera with "superpower" to see around corners and detect potential hazards
Research Background and Team
- PhD student from Stanford computational imaging lab
- Working on camera system for past few years
- Camera can image objects hidden around corners or blocked from direct line-of-sight
Outdoor Experiment Demo
- Camera system scanning side of building with laser
- Target scene hidden around corner behind curtain
- Camera system cannot see scene directly
- Successfully captures 3D geometry of hidden scene
Ultra-High Speed Camera Technology
- High-speed camera operating at trillion frames per second
- Fast enough to capture movement of light itself
- Speed comparison: light travels 3.3 nanoseconds per meter
- In same time, superhero moving 3x speed of sound moves less than width of human hair
- Camera captures photons at 50 trillionth of second (50 picoseconds) time frames
Technical Process
- Ultra-high speed camera paired with laser sending short light pulses
- Each pulse travels to visible wall, some light scatters back to camera
- Wall used to scatter light around corner to hidden object and back
- Measurement repeated many times to capture arrival times of many photons from different wall locations
- Creates trillion frame per second video of wall
Light Wave Visualization
- At trillion frames per second, can see waves of light scattered back from hidden scene
- Waves splash against wall carrying information about hidden object
- Measurements passed into reconstruction algorithm to recover 3D geometry of hidden scene
Indoor Scene Experiment
- Variety of different hidden objects with different appearances
- Objects reflect light differently:
- Glossy dragon statue
- Mirrored disco ball
- White discus thrower statue
- 3D volume visualization created by stacking video frames together
- Time represented as depth dimension of cube
- Bright dots: reflections from mirrored facets of disco ball
- Bright streaks arriving soonest: from glossy dragon statue closest to wall
- Other streaks: from book and statue reflections
- Frame-by-frame video shows scattered light progression
- Light waves last for just trillions of a second
- Despite different reflection properties, can still reconstruct object shapes
Real-Time Human Tracking Demo
- Person wearing reflective suit
- Camera system scanning wall at 4 times per second
- Captures enough photons to see person's location and actions
- Photons scatter: wall → tracksuit → wall → camera
- Indirect video captured in real-time without directly imaging person
Applications
- Self-driving cars
- Biomedical imaging for seeing into tiny body structures
- Camera systems on robots for planetary exploration
Recent Advancements
- Technology impossible just 2 years ago
- Now can image large room-sized hidden scenes outdoors at real-time rates
- Significant progress toward practical technology for cars
Remaining Challenges
- Imaging hidden scenes at long distances with very few photons
- Using low-power, eye-safe lasers
- Creating images from photons scattered multiple times (beyond single bounce)
- Miniaturizing large, bulky prototype system for:
- Biomedical imaging
- Improved home security systems
- Exploring other applications for new imaging modality
Future Outlook
- Exciting new technology with undiscovered applications
- Self-driving car future may seem distant but technologies already in development
- Rapid pace of scientific discovery and innovation
- New capabilities could emerge unexpectedly
