AI summary by TwinMind
Maximum Likelihood For the Normal Distribution, step-by-step!!!
StatQuest with Josh Starmer
19m 50s
Introduction and Prerequisites
- Josh Stormer presents StatQuest on maximum likelihood for normal distribution
- Follows up on previous StatQuests: "Maximum Likelihood Clearly Explained" and "Probability vs Likelihood"
- Assumes familiarity with normal distribution
- References "Normal Distribution Clearly Explained" StatQuest
Normal Distribution Parameters
- Normal distribution equation has two parameters:
- μ (mu): determines location of distribution's mean
- Smaller μ moves mean left
- Larger μ moves mean right
- σ (sigma): standard deviation, determines distribution width
- Larger σ makes curve shorter and wider
- Smaller σ makes curve taller and narrower
- μ (mu): determines location of distribution's mean
- Goal: use likelihood to find optimal μ and σ parameters given data
Single Measurement Example
- Simplest dataset: single mouse weighing 32 grams (x = 32)
- Test normal distribution with μ = 28, σ = 2
- Calculate likelihood by plugging values into likelihood function
- Result: likelihood = 0.03 (y-axis value)
- Test μ = 30, σ = 2: likelihood = 0.12 (higher)
- Fix σ = 2, vary μ values to find maximum likelihood
- Plot likelihood vs μ values on graph
- Maximum occurs where slope = 0, at μ = 32
- Note: need more than one measurement to optimally estimate σ
Two Sample Dataset
- Expand to two mice: x₁ = 32g, x₂ = 34g
- Test μ = 28, σ = 2 distribution
- Calculate individual likelihoods for each measurement
- Since measurements are independent, total likelihood = product of individual likelihoods
- For n data points: multiply all n individual likelihood functions together
Mathematical Derivation Setup
- General likelihood function without specified μ and σ values
- Equals product of likelihood functions for n individual measurements
- Need two derivatives:
- Derivative with respect to μ (treating σ as constant)
- Derivative with respect to σ (treating μ as constant)
- Find maximum likelihood estimates where derivatives equal zero
Log Transformation
- Take log of likelihood function to simplify derivative calculations
- Log likelihood and likelihood functions peak at same μ and σ values
- Step-by-step log transformations:
- Convert multiplication to addition
- Transform individual likelihood functions
- Convert 1/√ to exponent -1/2
- Convert negative exponent to multiplication, remove log(e)
- Convert multiplication to addition for constants
- Convert σ² exponent to 2×log(σ)
- Cancel 2/2 terms
Simplified Log Likelihood Function
- Combine terms for n data points
- Final simplified form: -n/2×log(2π) - n×log(σ) - 1/(2σ²)×Σ(xᵢ-μ)²
Derivative with Respect to μ
- First two terms don't contain μ, derivatives = 0
- Apply chain rule to third term
- Simplify to get: 1/σ²×Σ(xᵢ-μ)
- Further simplification: 1/σ²×(Σxᵢ - nμ)
Derivative with Respect to σ
- First term derivative = 0
- Second term derivative = -n/σ
- Third term requires rewriting 1/σ² as σ⁻²
- Apply derivative rules, simplify
- Final form: -n/σ + 1/σ³×Σ(xᵢ-μ)²
Solving for Maximum Likelihood Estimates
For μ:
- Set derivative with respect to μ = 0
- Multiply by σ², add nμ to both sides
- Divide by n: μ = (1/n)×Σxᵢ
- Result: μ̂ = sample mean
For σ:
- Set derivative with respect to σ = 0
- Multiply by σ, add n, multiply by σ²
- Divide by n, take square root
- Result: σ̂ = sample standard deviation
Final Results
- Maximum likelihood estimate for μ = mean of data
- Maximum likelihood estimate for σ = standard deviation of data
- Mathematical proof confirms intuitive solutions
- Mean determines center of normal distribution
- Standard deviation determines width of normal curve
