AI summary by TwinMind

Lecture 10: Characteristic Strip Expansion, Shape from Shading, Iterative Solutions

MIT OpenCourseWare

1h 26m 33s

Course Announcements

  • Three homework problems completed
  • First take-home quiz next week
    • Counts twice as much as homework
    • No collaboration allowed (unlike homework)
  • 686 students: proposal due on 22nd
    • Short description of term project
    • Machine vision problem implementation (Windows, Matlab, Android, etc.)
    • Can be theoretical/mathematical solution
    • Can use OpenCV but must add original contribution
    • Email for project ideas/sources

Term Project Examples

  • Time to contact implementation on Android phone
    • Android Studio with dummy project shell provided
  • Subpixel edge detection methods implementation
    • Use Matlab or convenient platform
  • Shape from shading with different reflectance maps
    • More theoretical/mathematical approach
  • Examples to be posted on Stella website

Course Structure Overview

  • First part: image projection, perspective projection, motion analysis
  • Current focus: image brightness measurements and shape from shading
  • Next: industrial machine vision methods using patents
    • Patents used instead of papers (industry doesn't publish for credit)
    • Learn patent language for entrepreneurs/startups
  • Transition from partial differential equations to practical applications

Image Irradiance Equation Foundation

  • Brightness at image point = reflectance map of surface orientation
  • Depends on: illumination, surface material, geometry (surface orientation)
  • Local relationship: image brightness depends on corresponding object point
  • Reflectance map summarizes detailed reflecting properties (BRDF)
  • BRDF: 4 parameters → Reflectance map: 2 parameters
  • Orthographic projection assumption (simplifies mathematics)
    • Corresponds to telecentric lens or very far distance
    • Like looking at moon with small visual angle

Hapka Model Solution Review

  • Moon/rocky planet reflecting properties
  • Can integrate in ecliptic plane direction only
  • Characteristic curves with update rules:
    • x, y coordinate changes
    • Height change: dz = p·ps + q·qs
  • Direct relationship: E²·rs - 1 = p·ps + q·qs
  • March along curves adjusting x, y, z at each point
  • Need initial curve for boundary conditions
  • Can go bidirectionally from initial curve

General Shape from Shading Method

  • Five coupled ordinary differential equations
  • Variables: x, y, z, p, q (position + orientation)
  • Characteristic strips: curves + surface orientation
  • Update rules using chain rule for p, q changes
  • Need second derivatives (Hessian matrix): r, s, t
  • More unknowns than equations initially

Breakthrough: Using Image Irradiance Equation

  • Brightness gradient: ∇E = [rp, rq] · H · [δx, δy]
  • Pattern matching approach: δx = ε·rp, δy = ε·rq
  • Eliminates need to solve for Hessian matrix H
  • Direction determined by reflectance map gradient
  • Five differential equations:
    • dx/dψ = ε·rp
    • dy/dψ = ε·rq
    • dp/dψ = ε·ex
    • dq/dψ = ε·ey
    • dz/dψ = ε·(p·rp + q·rq)

Step Size Control Methods

  • Constant step size in height (z-direction)
    • Divide by (p·rp + q·rq)
    • Move from contour to contour
  • Constant step size in image plane
    • Divide by √(rp² + rq²)
    • Fixed pixel increments
  • Constant step size in 3D space
    • Divide by √(rp² + rq² + (p·rp + q·rq)²)
  • Constant brightness increments
    • Divide by (ex·rp + ey·rq)
    • Move between isophotes

Specific Reflectance Map Applications

  • Hapka model: reduces to original special case
  • SEM model: R = f(√(p² + q²))
    • Update direction = gradient direction
    • Simple uphill/downhill movement
  • Lambertian: more complex due to square root terms

Parallelization and Implementation

  • Base characteristics: projection into image plane
  • Solutions along curves are independent
  • Parallelizable computation (not fully parallel)
  • Wave front propagation approach
  • Interpolate/merge characteristics as needed
  • Monitor spacing between curves

Initial Conditions Problem

  • Need initial curve for solution
  • Worse: need initial strip (position + orientation)
  • Two constraints available:
    • Image irradiance equation: E(x,y) = R(p,q)
    • Curve constraint: dz/dθ = p·dx/dθ + q·dy/dθ
  • Two equations, two unknowns (p, q) - solvable
  • Still need initial curve (problematic)

Stationary Points Solution

  • Occluding boundary: known orientation but infinite slope
  • Global extrema in reflectance map
  • Lambertian: maximum brightness point
  • SEM: minimum brightness point
  • Unique surface orientation determination
  • Problem: stationary points don't move (rp = rq = 0)

Local Shape Estimation

  • Move slightly away from stationary point
  • Use second derivatives of brightness
  • Construct curved surface patch around stationary point
  • Example: z = x² + 2y² surface
  • Gradient of image gives curvature information
  • Start solutions from constructed patch boundary

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome