AI summary by TwinMind
Lecture 8: Wavelengths and colors
MIT OpenCourseWare
1h 25m 27s
Historical Color Photography Methods
- Early color imaging: capture three sequential images with green, red, blue filters
- Project back through three projectors with same filters
- Creates color appearance to human eye
- Century-old technique now digitized by Library of Congress
- Color wheel systems: rotating wheel in front of projector (DLP)
- Projects one color component at time (green/red/blue)
- Human eye integrates over time via rapid succession
- Light integrated temporally vs spatially
Three CCD Sensor Systems
- Use dichroic prisms/mirrors to split light into three wavelength regions
- Three separate monochrome CCDs capture each wavelength
- Most popular approach in still cameras and digital cameras
Bayer Mosaic Pattern
- Invented by Bayer at Kodak, patented mid-late 70s
- 2x2 pixel arrangement: two green, one red, one blue filter
- Pattern tiled across entire sensor
- Each pixel senses only one wavelength band
- Demosaicing algorithms interpolate between neighboring pixels
- Estimate missing color information via interpolation
- "Hallucinating" higher resolution (3x factor increase)
- 4 megapixel camera breakdown: 2 megapixel green, 1 megapixel red, 1 megapixel blue
- Advantages vs three CCD: cheaper, less optics, easier alignment
- Disadvantages: blocks 2/3 of light, reconstruction artifacts
- Moiré patterns on fine details (text, stripes)
- Low pass filter added to reduce high frequency artifacts
- Color artifacts near edges
Advanced Sensor Designs
- Nikon's recent approach: combines dichroic mirrors at pixel level
- Similar to three CCD but at semiconductor level
- No light loss unlike traditional filters
- Foveon X3 sensor: vertical stacking approach
- Three layers at different depths, each sensitive to different wavelengths
- Top: blue sensitive, middle: green, bottom: red
- Similar to film emulsion layers
- Single pixel captures all three colors
- No spatial multiplexing, no low pass filter needed
- Marketing: 12 megapixel = 4 megapixel x 3 elements
- Disadvantages: less wavelength separation control, requires software processing
- Actual spectral response highly overlapped, not clean RGB separation
Alternative Bayer Patterns
- RGB not sacred - first proposed, most common
- Other patterns: RGBC (red, green, blue, clear)
- CMY (cyan, magenta, yellow) allows more light
- Various trade-offs between different sensor types
- Kodak studies compare different approaches
Electromagnetic Spectrum Context
- Visible spectrum: 400-700 nanometers (blue to red)
- Only wavelength region with atmospheric transparency
- Natural illumination concentrated in this region
- Human/animal vision evolved for this range
- Shorter wavelengths (UV): more dangerous, cancerous
- Longer wavelengths: generally harmless
- Energy relationship: E = hν (frequency), higher frequency = more energy
- Wavelength × frequency = speed of light (3×10⁸ m/s)
Spectroscopy and Dispersion
- Spectroscope: prism-based optical element
- Refractive index varies with wavelength
- Different wavelengths bend at different angles
- Can decompose light into constituent wavelengths
- Demonstration: handheld spectroscope with 400-700nm scale
- Fluorescent lights show spiky spectrum (sharp blue, little green/red)
- Diffraction grating: similar dispersion effect to prism
Multi-spectral Imaging Techniques
- Data cube concept: x, y spatial dimensions + λ wavelength dimension
- 3D data mapped onto 2D sensor using time dimension
- Three approaches:
- Staring: complete scene capture with filters (like early color photography)
- Push broom: line-by-line scanning as platform moves
- Whisk broom: point-by-point scanning with spectral analysis per point
Thermal Imaging Applications
- Wavelength range: 1-15 microns (6-8 micron typical for humans)
- Defense applications: heat-seeking missiles
- Civilian applications: building heat loss detection, HVAC testing, pipe leaks
- Price dropping from $20,000 to more accessible levels
- Material properties differ in thermal vs visible:
- Glass opaque in thermal IR
- Metal surfaces act as mirrors in thermal (smooth at longer wavelengths)
- Lie detection research: blood flow changes around eyes detectable in thermal
- University of Houston/Colorado research
- Claims performance comparable to traditional lie detectors
- Results difficult to reproduce
Near-infrared Photography
- CCD sensors naturally sensitive to 700nm-1μm range
- Manufacturers add IR block filters above 700nm
- Remove filter for IR photography
- Fake colors added (no natural color beyond 700nm)
- Kodak color IR film provides different wavelength responses
- Effects: sky becomes black, vegetation becomes bright white
- Tree bark absorbs more IR, appears darker
Remote Sensing Applications
- Multi-spectral crop classification
- Different vegetation types have distinct reflectance profiles
- Appear same green to human eye but different spectrally
- 5-10nm bands, 20-30 channels typical
- High-dimensional signal processing for classification
- Extends into IR range for optimal discrimination
UV Photography
- Flowers show dramatic differences in UV vs visible
- UV patterns guide bee navigation ("landing strips")
- Requires special quartz lenses (glass absorbs UV)
- Alternative: Fresnel zone plates with fluorescent sensor coatings
- Film photography more common than digital for UV
- Pseudo-coloring used for visualization (monochrome UV images)
Human Color Perception
- Chromaticity diagram: horseshoe-shaped gamut of human-visible colors
- Spectral purity colors on perimeter (400-700nm)
- Interior points: combinations of multiple wavelengths
- sRGB color space: much smaller triangle within full gamut
- Color vs wavelength distinction:
- Wavelength: physical property of light
- Color: human perceptual response
- Same frequency/energy in different media (water example: 630nm→420nm wavelength shift)
Device Color Limitations
- Most devices occupy small subset of human-visible gamut
- sRGB triangle defined by three fixed primaries
- Cannot represent colors outside triangle
- Wider gamut requires primaries closer to spectral edge
- Trade-off: wider gamut vs light throughput (sharper filters block more light)
- Adaptive color primaries concept: adjust based on scene content
- Audio equalizer analogy: tune different frequencies for different content
Agile Spectrum Camera System
- Optical design: pinhole → prism → lens → mask plane → sensor
- Spectral light field concept: x (spatial) vs λ (wavelength) instead of x vs θ (angle)
- Rainbow plane: all rays of given wavelength converge to single point
- Mask in rainbow plane controls spectral sensitivity
- Applications demonstrated:
- Glare reduction: block LED wavelengths to see background text
- High dynamic range: spectral rather than temporal approach
- Metameric object discrimination: objects appearing same in white light but different under monochromatic illumination
Agile Spectrum Projector
- Similar optical design for projection
- Adaptive color primaries for specific scene content
- Wider color gamut than traditional RGB
- Cyan projection improvement (traditional weak point)
- Scene-specific optimization: yellow+blue projector for yellow+blue scene
- Color blindness assistance: specific wavelengths help deuteranopes distinguish red/green
3D Display Applications
- Wavelength multiplexing for stereo separation
- Different blue wavelengths for left/right eyes
- Appears normal blue to naked eye
- Color filters separate channels for each eye
- Alternative to temporal or polarization multiplexing
Combined Spectral-Spatial Light Fields
- Integration of traditional light field (x,θ) with spectral dimension (λ)
- Prism creates overlapping spectral light fields
- Coded aperture approach: replace pinhole with designed mask
- Deconvolution recovers full spectral information
- Light field camera + prism combination
- Linear variable interference filter: replaces θ with λ dimension
- Trade-off: spatial pinhole vs spectral pinhole (wavelength selectivity)
System Limitations
- Diffraction artifacts from pinhole requirement
- Small f-number needed for adequate light gathering
- Light throughput vs spectral resolution trade-off
- Future work: coded aperture masks for improved performance
