AI summary by TwinMind
Lecture 5: Floats and Approximation Methods
MIT OpenCourseWare
47m 10s
Binary Number Representation Recap
- Motivation: Python code showing X = 0, loop adding 0.1 ten times, X ≠ 1 (false)
- Actual value: 0.9999999999999999 ≠ 1
- Demonstrates floating point precision issues
- Integer to binary conversion algorithm:
- Start with empty string, while num > 0:
- Get remainder when dividing by 2 (0 or 1)
- Prepend remainder to result string
- Integer divide num by 2
- Example: 19 → 10011 (binary)
- Negative numbers: convert absolute value, prepend negative sign
- Start with empty string, while num > 0:
Fraction to Binary Conversion
- Base 10 fractions: 0.ABC = A/10 + B/100 + C/1000
- Base 2 fractions: 0.ABC = A/2 + B/4 + C/8 (powers of 2)
- Recipe for decimal to binary fraction conversion:
- Find smallest power of 2 that multiplies decimal to give whole number
- Convert whole number to binary
- Shift decimal point left by power amount
- Example: 0.375 = 3/8
- 0.375 × 8 = 3
- 3 in binary = 11
- Shift 3 positions: 0.011
Binary Fraction Conversion Algorithm Implementation
- Code finds magical power P where (X × 2^P) % 1 = 0
- Loop increments P until no decimal remainder
- Convert resulting integer to binary using previous algorithm
- Shift decimal point P positions left by padding with zeros
- Example: 0.625 requires P=3, gives 0.101 binary representation
- Problem: some numbers require very large P (0.1 needs ~50 digits)
Floating Point Representation in Memory
- Numbers stored as two pieces: significant digit + power of two
- Examples:
- (1, 1) → 1.0 × 2^1 = 2.0
- (1, -1) → 1.0 × 2^-1 = 0.5
- (125, -2) → 125 in binary shifted 2 positions = 31.25
- Called "float" because decimal point "floats"
- Memory limitation: 32 or 64 bits for significant digits
- Numbers requiring >32 digits get truncated → approximation errors
- Error magnitude: ~2^-32 ≈ 2×10^-10
- Errors accumulate over multiple operations
Approximation Method Algorithm
- Problem: exhaustive enumeration limited to finite integer sets
- Solution: use small increments instead of integer steps
- Goal: find root where |root² - X| < epsilon
- Algorithm steps:
- Start with guess = 0 (too small)
- Increment by small value (0.1, 0.01, 0.001, etc.)
- Check if |guess² - X| < epsilon
- Continue until within epsilon or overshoot
Approximation Method Parameters
- Two key parameters:
- Epsilon: how close to final answer (accuracy)
- Increment: how much to change guess each iteration
- Trade-offs:
- Smaller increment → more accurate, slower
- Larger epsilon → faster, less accurate
Implementation and Issues
- Code structure:
- Initialize: x, epsilon, increment, num_guesses, guess=0
- While |guess² - x| >= epsilon: increment guess, count guesses
- Print result and guess count
- Example results:
- √36 ≈ 5.999100089910089 (59992 guesses)
- √24 ≈ 4.898979485776156 (48990 guesses)
- Infinite loop problem: algorithm can overshoot epsilon boundary
- Example: 54321 never terminates
- Solution: add condition guess² < x to prevent overshoot
Debugging and Solutions
- Added print statements every 100,000 guesses to track progress
- Visualization shows overshoot problem:
- Guess approaches target from below
- One increment jumps past epsilon boundary
- Subsequent guesses get farther from target
- Fix: add stopping condition when guess² > x
- Alternative solutions:
- Decrease increment (10x smaller → 10x slower)
- Increase epsilon (less accurate)
Key Lessons and Trade-offs
- Cannot use == for float comparisons due to precision errors
- Must use |value - target| < epsilon for exit conditions
- Watch for overshoot in increment-based algorithms
- Speed vs accuracy trade-off:
- Smaller increment: more accurate, much slower
- Larger epsilon: faster, less accurate
- Example: 54321 with increment 0.0000001 took ~10 seconds, 23 million guesses
