AI summary by TwinMind

Lecture 23: Complexity Classes Examples

MIT OpenCourseWare

1h 21m 18s

Introduction and Review of Theta Notation

  • Recap of Theta notation as preferred over Big O for asymptotic bounds on worst-case runtime
  • Theta focuses on dominant term, dropping additive/multiplicative constants and slower-growing terms
  • Method for complexity analysis:
    • Identify inputs to function
    • Find code parts dependent on inputs (direct/indirect)
    • Count operations as function of inputs
    • Apply Theta rules (law of addition/multiplication)

Complexity Classes Overview

  • Constant (Θ(1)): No dependence on input
  • Logarithmic (Θ(log n)): Logarithmic growth
  • Linear (Θ(n)): Linear growth
  • Log-linear (Θ(n log n)): Linear times logarithmic
  • Polynomial (Θ(n^k)): Polynomial growth (n², n³, etc.)
  • Exponential (Θ(k^n)): Exponential growth (2^n, 3^n, etc.)
  • Goal: Keep algorithms in top 4-5 classes for efficiency

Constant Complexity Examples

  • Built-in constant operations: indexing lists, appending to lists, dictionary key access
  • Examples:
    • add(x, y): Simple arithmetic - Θ(1)
    • miles_to_km(miles): Multiplication - Θ(1)
    • Loop with fixed iterations (100 times) regardless of input - Θ(1)

Linear Complexity Examples

  • Built-in linear operations: e in list, copying lists, list equality checks, list deletion
  • Examples:
    • multiply(x, y): Loop y times adding x - Θ(y) with respect to y, Θ(1) with respect to x
    • String digit sum: Loop through string characters - Θ(length of string)
    • Iterative factorial: Loop from 2 to n+1 - Θ(n)
    • Compound interest calculation: Loop n_months times - Θ(n_months)
    • Iterative Fibonacci: Loop n times - Θ(n)

Real-World vs Theoretical Analysis

  • Factorial timing experiment showed linear growth until ~640-1280 input size
  • Beyond that point, growth became quadratic due to large number storage limitations
  • Demonstrates difference between theoretical Θ(n) and practical implementation constraints

Recursive Function Analysis

  • Analyze by counting function calls rather than loops
  • Recursive factorial example:
    • Calls itself n times (n, n-1, n-2, ..., 0)
    • Each call has Θ(1) overhead
    • Overall complexity: Θ(n)
  • Iterative vs recursive factorial both Θ(n) - choice based on other factors

Polynomial Complexity Examples

  • Nested loops create polynomial complexity
  • Simple nested loop: outer Θ(n) × inner Θ(n) = Θ(n²)
  • is_subset(L1, L2):
    • Outer loop: Θ(length L1)
    • Inner loop: Θ(length L2)
    • Overall: Θ(length L1 × length L2)
  • intersect(L1, L2):
    • First part: nested loops Θ(length L1 × length L2)
    • Second part: iterate through temporary list also Θ(length L1 × length L2)
    • Overall: Θ(length L1 × length L2)
  • diameter(points):
    • Outer loop: Θ(length L)
    • Inner loop starts at i, averages to ~0.5×length L
    • Still Θ(length L²) overall

Exponential Complexity Examples

  • Recursive Fibonacci: Θ(2^n)
    • Each call spawns 2 recursive calls
    • Creates binary tree structure of function calls
    • Tree may thin on right side but still exponential growth
  • generate_subsets(list):
    • Creates all possible subsets of input list
    • Number of subsets: 2^n where n is list length
    • Additional overhead for copying lists: Θ(n × 2^n)
    • Recursive structure: extract element, generate subsets of remainder, combine results

Logarithmic Complexity

  • count_digits(n):
    • Converts number to string and iterates through digits
    • Relationship: length of string = log₁₀(n)
    • Overall complexity: Θ(log n)
  • Key insight: indirect relationship between input magnitude and loop iterations

Linear vs Binary Search Algorithms

  • Linear search (unsorted list): Θ(length L)
    • Can optimize with early return but worst-case still Θ(n)
  • Linear search (sorted list): Still Θ(length L) worst-case
    • Can exit early if element larger than target, but worst-case unchanged
  • Binary search: Θ(log n)
    • Divides search space in half each iteration
    • Requires sorted list
    • Two implementations:
      • Copying sublists: Θ(n log n) due to copy overhead
      • Using indices: Θ(log n) with constant overhead

When to Sort Before Searching

  • Single search: Never sort first (sorting takes at least Θ(n))
  • Multiple searches: Sort once, then use binary search k times
  • Amortized cost makes sense when k is large
  • Trade-off: Θ(sort) + k×Θ(log n) vs k×Θ(n)

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome