Introduction and Context

• Josh Starmer presents StatQuest on why dividing by n underestimates variance
• Assumes prior knowledge of population parameter estimation
• Builds on previous StatQuest covering mean, variance, and standard deviation estimation
• References gene X measurement in liver cells example from previous episode

Population vs Sample Statistics Review

• Population mean μ = sum of measurements / number of measurements = average measurement
• Population variance = average of squared distances between data and population mean
• Population standard deviation = square root of variance
• Estimated population mean x̄ = sum of measurements / number of measurements
• Estimated population variance = sum of squared differences / (n-1)
• Estimated standard deviation = square root of estimated variance
• Key point: dividing by (n-1) compensates for using sample mean instead of population mean

First Example Demonstration

• Used 5 measurements to demonstrate variance calculation around different points
• Calculated variance around zero: 391
• Calculated variance around point 5: 240
• Showed variance around sample mean vs population mean
• Key finding: variance around sample mean < variance around population mean when dividing by n
• Conclusion: using sample mean with division by n underestimates population variance

Second Example Confirmation

• Collected 5 new measurements from same population
• Calculated variance around zero: 616
• Repeated process with multiple points
• Confirmed same pattern: minimum variance occurs at sample mean
• Again showed sample mean division by n underestimates population variance

Mathematical Proof Setup

• Replaced x̄ with unknown value V in variance formula
• Created general formula for variance around any point V
• Used calculus to find minimum variance point

Derivative Calculation

• Applied chain rule to find derivative of variance formula with respect to V
• Derivative = -2/n × Σ(Xi - V)
• Set derivative = 0 to find minimum variance point
• Solved for V where slope = 0

Proof with Specific Data

• Plugged in n=5 and actual measurements
• Multiplied both sides by 5/(-2) to solve for V
• Result: V = sample mean x̄ = 17.6
• Confirmed minimum variance occurs when V = sample mean

Proof with General 5 Measurements

• Replaced specific data with unknown values X₁, X₂, X₃, X₄, X₅
• Applied same derivative process
• Result: V = average of 5 measurements = sample mean x̄
• Conclusion: regardless of specific measurements, minimum variance at sample mean

Proof for Any Sample Size

• Extended proof to n measurements (X₁ through Xₙ)
• Derivative = -2/n × Σ(Xi - V)
• Set equal to zero and solved for V
• Result: V = sample mean x̄ for any sample size
• Universal conclusion: sample mean always minimizes variance

Why Squaring vs Absolute Value

• Explained choice of squared differences over absolute value
• Squared differences create smooth curve suitable for calculus
• Absolute value creates sharp angle at minimum, making derivative undefined
• Calculus approach much easier with squared terms

Final Summary and Implications

• Division by n underestimates population variance
• Sample mean differences consistently smaller than population mean differences
• Population mean differences result in larger average (what we're estimating)
• Therefore: use (n-1) for population variance estimation
• Teaser: future StatQuest on expected values will explain why (n-1) specifically, not (n-0.5) or (n-2)