Summary by TwinMind · 47m 10s

Summary by TwinMind · 47m 10s

Lecture 5: Floats and Approximation Methods

Lecture 5: Floats and Approximation Methods

MIT OpenCourseWare

MIT OpenCourseWare

Binary Number Representation Recap

  • Motivation: Python code showing X = 0, loop adding 0.1 ten times, X ≠ 1 (false)
    • Actual value: 0.9999999999999999 ≠ 1
    • Demonstrates floating point precision issues
  • Integer to binary conversion algorithm:
    • Start with empty string, while num > 0:
      • Get remainder when dividing by 2 (0 or 1)
      • Prepend remainder to result string
      • Integer divide num by 2
    • Example: 19 → 10011 (binary)
    • Negative numbers: convert absolute value, prepend negative sign

Fraction to Binary Conversion

  • Base 10 fractions: 0.ABC = A/10 + B/100 + C/1000
  • Base 2 fractions: 0.ABC = A/2 + B/4 + C/8 (powers of 2)
  • Recipe for decimal to binary fraction conversion:
    • Find smallest power of 2 that multiplies decimal to give whole number
    • Convert whole number to binary
    • Shift decimal point left by power amount
  • Example: 0.375 = 3/8
    • 0.375 × 8 = 3
    • 3 in binary = 11
    • Shift 3 positions: 0.011

Binary Fraction Conversion Algorithm Implementation

  • Code finds magical power P where (X × 2^P) % 1 = 0
  • Loop increments P until no decimal remainder
  • Convert resulting integer to binary using previous algorithm
  • Shift decimal point P positions left by padding with zeros
  • Example: 0.625 requires P=3, gives 0.101 binary representation
  • Problem: some numbers require very large P (0.1 needs ~50 digits)

Floating Point Representation in Memory

  • Numbers stored as two pieces: significant digit + power of two
  • Examples:
    • (1, 1) → 1.0 × 2^1 = 2.0
    • (1, -1) → 1.0 × 2^-1 = 0.5
    • (125, -2) → 125 in binary shifted 2 positions = 31.25
  • Called "float" because decimal point "floats"
  • Memory limitation: 32 or 64 bits for significant digits
  • Numbers requiring >32 digits get truncated → approximation errors
  • Error magnitude: ~2^-32 ≈ 2×10^-10
  • Errors accumulate over multiple operations

Approximation Method Algorithm

  • Problem: exhaustive enumeration limited to finite integer sets
  • Solution: use small increments instead of integer steps
  • Goal: find root where |root² - X| < epsilon
  • Algorithm steps:
    • Start with guess = 0 (too small)
    • Increment by small value (0.1, 0.01, 0.001, etc.)
    • Check if |guess² - X| < epsilon
    • Continue until within epsilon or overshoot

Approximation Method Parameters

  • Two key parameters:
    • Epsilon: how close to final answer (accuracy)
    • Increment: how much to change guess each iteration
  • Trade-offs:
    • Smaller increment → more accurate, slower
    • Larger epsilon → faster, less accurate

Implementation and Issues

  • Code structure:
    • Initialize: x, epsilon, increment, num_guesses, guess=0
    • While |guess² - x| >= epsilon: increment guess, count guesses
    • Print result and guess count
  • Example results:
    • √36 ≈ 5.999100089910089 (59992 guesses)
    • √24 ≈ 4.898979485776156 (48990 guesses)
  • Infinite loop problem: algorithm can overshoot epsilon boundary
    • Example: 54321 never terminates
    • Solution: add condition guess² < x to prevent overshoot

Debugging and Solutions

  • Added print statements every 100,000 guesses to track progress
  • Visualization shows overshoot problem:
    • Guess approaches target from below
    • One increment jumps past epsilon boundary
    • Subsequent guesses get farther from target
  • Fix: add stopping condition when guess² > x
  • Alternative solutions:
    • Decrease increment (10x smaller → 10x slower)
    • Increase epsilon (less accurate)

Key Lessons and Trade-offs

  • Cannot use == for float comparisons due to precision errors
  • Must use |value - target| < epsilon for exit conditions
  • Watch for overshoot in increment-based algorithms
  • Speed vs accuracy trade-off:
    • Smaller increment: more accurate, much slower
    • Larger epsilon: faster, less accurate
  • Example: 54321 with increment 0.0000001 took ~10 seconds, 23 million guesses