AI summary by TwinMind
Lecture 23: Complexity Classes Examples
MIT OpenCourseWare
1h 21m 18s
Introduction and Review of Theta Notation
- Recap of Theta notation as preferred over Big O for asymptotic bounds on worst-case runtime
- Theta focuses on dominant term, dropping additive/multiplicative constants and slower-growing terms
- Method for complexity analysis:
- Identify inputs to function
- Find code parts dependent on inputs (direct/indirect)
- Count operations as function of inputs
- Apply Theta rules (law of addition/multiplication)
Complexity Classes Overview
- Constant (Θ(1)): No dependence on input
- Logarithmic (Θ(log n)): Logarithmic growth
- Linear (Θ(n)): Linear growth
- Log-linear (Θ(n log n)): Linear times logarithmic
- Polynomial (Θ(n^k)): Polynomial growth (n², n³, etc.)
- Exponential (Θ(k^n)): Exponential growth (2^n, 3^n, etc.)
- Goal: Keep algorithms in top 4-5 classes for efficiency
Constant Complexity Examples
- Built-in constant operations: indexing lists, appending to lists, dictionary key access
- Examples:
add(x, y): Simple arithmetic - Θ(1)miles_to_km(miles): Multiplication - Θ(1)- Loop with fixed iterations (100 times) regardless of input - Θ(1)
Linear Complexity Examples
- Built-in linear operations:
e in list, copying lists, list equality checks, list deletion - Examples:
multiply(x, y): Loop y times adding x - Θ(y) with respect to y, Θ(1) with respect to x- String digit sum: Loop through string characters - Θ(length of string)
- Iterative factorial: Loop from 2 to n+1 - Θ(n)
- Compound interest calculation: Loop n_months times - Θ(n_months)
- Iterative Fibonacci: Loop n times - Θ(n)
Real-World vs Theoretical Analysis
- Factorial timing experiment showed linear growth until ~640-1280 input size
- Beyond that point, growth became quadratic due to large number storage limitations
- Demonstrates difference between theoretical Θ(n) and practical implementation constraints
Recursive Function Analysis
- Analyze by counting function calls rather than loops
- Recursive factorial example:
- Calls itself n times (n, n-1, n-2, ..., 0)
- Each call has Θ(1) overhead
- Overall complexity: Θ(n)
- Iterative vs recursive factorial both Θ(n) - choice based on other factors
Polynomial Complexity Examples
- Nested loops create polynomial complexity
- Simple nested loop: outer Θ(n) × inner Θ(n) = Θ(n²)
is_subset(L1, L2):- Outer loop: Θ(length L1)
- Inner loop: Θ(length L2)
- Overall: Θ(length L1 × length L2)
intersect(L1, L2):- First part: nested loops Θ(length L1 × length L2)
- Second part: iterate through temporary list also Θ(length L1 × length L2)
- Overall: Θ(length L1 × length L2)
diameter(points):- Outer loop: Θ(length L)
- Inner loop starts at i, averages to ~0.5×length L
- Still Θ(length L²) overall
Exponential Complexity Examples
- Recursive Fibonacci: Θ(2^n)
- Each call spawns 2 recursive calls
- Creates binary tree structure of function calls
- Tree may thin on right side but still exponential growth
generate_subsets(list):- Creates all possible subsets of input list
- Number of subsets: 2^n where n is list length
- Additional overhead for copying lists: Θ(n × 2^n)
- Recursive structure: extract element, generate subsets of remainder, combine results
Logarithmic Complexity
count_digits(n):- Converts number to string and iterates through digits
- Relationship: length of string = log₁₀(n)
- Overall complexity: Θ(log n)
- Key insight: indirect relationship between input magnitude and loop iterations
Linear vs Binary Search Algorithms
- Linear search (unsorted list): Θ(length L)
- Can optimize with early return but worst-case still Θ(n)
- Linear search (sorted list): Still Θ(length L) worst-case
- Can exit early if element larger than target, but worst-case unchanged
- Binary search: Θ(log n)
- Divides search space in half each iteration
- Requires sorted list
- Two implementations:
- Copying sublists: Θ(n log n) due to copy overhead
- Using indices: Θ(log n) with constant overhead
When to Sort Before Searching
- Single search: Never sort first (sorting takes at least Θ(n))
- Multiple searches: Sort once, then use binary search k times
- Amortized cost makes sense when k is large
- Trade-off: Θ(sort) + k×Θ(log n) vs k×Θ(n)
