AI summary by TwinMind
Lecture 1: Introduction to Machine Vision
MIT OpenCourseWare
1h 19m 52s
Course Administration and Structure
- Course: Machine Vision 6801/6866, classroom far from instructor's office
- No textbook required; materials on website including Robot Vision book chapters 1-2
- Assignment Schedule:
- 5 homework problems (every second week) + 2 take-home quizzes
- Collaboration allowed on homework, NOT on quizzes
- No final exam
- First homework due this week despite first class
- 6866 Differences:
- Additional term project required (implementing Machine Vision method)
- Proposal due ~1 month from start
- Preference for dynamic problems over static image analysis
- Optional Android development session if sufficient interest
- Grading:
- 6801: 50% homework, 50% quizzes
- 6866: 33% homework, 33% quizzes, 33% project
- Collaboration encouraged on projects to ensure early start and progress
Course Objectives and Scope
- Main Goal: Learn to recover environmental information from images using inverse graphics approach
- Physics-based Machine Vision: light rays → surface reflection → image formation
- Foundation for advanced Machine Vision courses
- Teaches feature extraction for learning approaches (texture, distance, shape, size)
- Applications of engineering math/physics (calculus, vectors, matrices, linear algebra, ODEs)
- NOT covered: Image processing, pattern recognition, machine learning, computational imaging, human vision
Mathematical Framework
- Engineering-level math: calculus, derivatives, vectors, matrices, some linear algebra
- Prerequisite 6.03 for convolution concepts in image formation
- Emphasis on direct computations vs. black box approaches
- Least squares optimization as primary method (closed-form solutions preferred)
- Focus on noise sensitivity and robustness analysis
Machine Vision Applications Demonstrated
- Motion Recovery: Observer motion from time-varying images, autonomous vehicles
- Time to Contact: Monocular estimation without camera calibration parameters
- Single camera approach (no stereo required)
- Focal length independent (image expansion scales with motion)
- Focus of expansion detection for motion vector determination
- Surface Shape Recovery: 3D shape from monocular images using shading cues
- Industrial Applications: Bin picking problem - object recognition and manipulation
- Photometric Stereo: Three-light calibration system for surface orientation
- Analog Computing: Hardware implementation for real-time focus of expansion
Image Formation Theory - Projection
- Pinhole Camera Model: Simplified lens system approximation
- Camera-centric coordinate system:
- Origin at center of projection
- Z-axis aligned with optical axis
- Image plane at distance f (focal length)
- Perspective Projection Equations:
- x = fX/Z, y = fY/Z (component form)
- r = (f/Z·R·ẑ)R (vector form)
- Key Properties:
- Nonlinear (division by Z)
- Enables depth recovery from motion
- Similar triangles relationship
Image Formation Theory - Brightness
- Brightness Dependencies:
- Linear relationship with illumination (over many orders of magnitude)
- Surface reflectance properties (moon reflects only 11% - "black as coal")
- Distance independence in normal viewing (inverse square law cancelled by area scaling)
- Surface orientation (foreshortening effects)
- Foreshortening Effects:
- Incident illumination varies as cos(θᵢ) where θᵢ is incident angle
- Viewing geometry also affected by surface orientation
- Both illumination and observation directions matter
- Shape from Shading Problem:
- Surface orientation has 2 degrees of freedom (unit normal vector)
- Single brightness measurement provides 1 constraint
- Under-constrained problem requiring additional information
Photometric Stereo Solution
- Three-light calibration system using ceiling fluorescents
- Sphere calibration object for known surface orientations
- Each light provides one constraint per pixel
- RGB visualization combining three illumination conditions
- Simultaneous recovery of surface orientation and reflectance
- Noise sensitivity analysis - some surface orientations more reliable than others
Industrial Bin Picking Application
- Problem: Pick randomly oriented parts from pile (vs. palletized parts)
- Challenges: Arbitrary 3D orientations, occlusion, shadows, inter-reflections
- Solution approach:
- Photometric stereo for surface orientation
- Extended Gaussian image representation for compact shape description
- Single camera + infrared beam for depth sensing
- Robot coordinate system calibration using surveyor marks
- Coordinate transformations between vision system and robot kinematics
Fundamental Limitations and Constraints
- Ill-posed problems: No solution, infinite solutions, or sensitive to initial conditions
- Eight-point algorithm example: Works on perfect data, fails with small errors
- Noise considerations:
- 8-bit quantization (1 part in 256)
- Small pixel size (few microns)
- Limited photon count per pixel (~1 million or less)
- Statistical counting noise
- Why vision works: Special world structure
- Straight-line light propagation through air
- Solid surfaces (not translucent volumes)
- 2D-to-2D correspondence (not true 3D-to-2D volumetric)
Vector Notation and Mathematical Conventions
- Vector notation: Bold in publications, underlined on blackboard
- Unit vectors: Hat notation (â)
- Column vector convention: All vectors treated as column vectors
- Transpose notation: T superscript for row representation
- Dot products: For extracting vector components (R·ẑ for Z-component)
