AI summary by TwinMind

Maximum Likelihood For the Normal Distribution, step-by-step!!!

StatQuest with Josh Starmer

19m 50s

Introduction and Prerequisites

  • Josh Stormer presents StatQuest on maximum likelihood for normal distribution
  • Follows up on previous StatQuests: "Maximum Likelihood Clearly Explained" and "Probability vs Likelihood"
  • Assumes familiarity with normal distribution
  • References "Normal Distribution Clearly Explained" StatQuest

Normal Distribution Parameters

  • Normal distribution equation has two parameters:
    • μ (mu): determines location of distribution's mean
      • Smaller μ moves mean left
      • Larger μ moves mean right
    • σ (sigma): standard deviation, determines distribution width
      • Larger σ makes curve shorter and wider
      • Smaller σ makes curve taller and narrower
  • Goal: use likelihood to find optimal μ and σ parameters given data

Single Measurement Example

  • Simplest dataset: single mouse weighing 32 grams (x = 32)
  • Test normal distribution with μ = 28, σ = 2
  • Calculate likelihood by plugging values into likelihood function
  • Result: likelihood = 0.03 (y-axis value)
  • Test μ = 30, σ = 2: likelihood = 0.12 (higher)
  • Fix σ = 2, vary μ values to find maximum likelihood
  • Plot likelihood vs μ values on graph
  • Maximum occurs where slope = 0, at μ = 32
  • Note: need more than one measurement to optimally estimate σ

Two Sample Dataset

  • Expand to two mice: x₁ = 32g, x₂ = 34g
  • Test μ = 28, σ = 2 distribution
  • Calculate individual likelihoods for each measurement
  • Since measurements are independent, total likelihood = product of individual likelihoods
  • For n data points: multiply all n individual likelihood functions together

Mathematical Derivation Setup

  • General likelihood function without specified μ and σ values
  • Equals product of likelihood functions for n individual measurements
  • Need two derivatives:
    • Derivative with respect to μ (treating σ as constant)
    • Derivative with respect to σ (treating μ as constant)
  • Find maximum likelihood estimates where derivatives equal zero

Log Transformation

  • Take log of likelihood function to simplify derivative calculations
  • Log likelihood and likelihood functions peak at same μ and σ values
  • Step-by-step log transformations:
    1. Convert multiplication to addition
    2. Transform individual likelihood functions
    3. Convert 1/√ to exponent -1/2
    4. Convert negative exponent to multiplication, remove log(e)
    5. Convert multiplication to addition for constants
    6. Convert σ² exponent to 2×log(σ)
    7. Cancel 2/2 terms

Simplified Log Likelihood Function

  • Combine terms for n data points
  • Final simplified form: -n/2×log(2π) - n×log(σ) - 1/(2σ²)×Σ(xᵢ-μ)²

Derivative with Respect to μ

  • First two terms don't contain μ, derivatives = 0
  • Apply chain rule to third term
  • Simplify to get: 1/σ²×Σ(xᵢ-μ)
  • Further simplification: 1/σ²×(Σxᵢ - nμ)

Derivative with Respect to σ

  • First term derivative = 0
  • Second term derivative = -n/σ
  • Third term requires rewriting 1/σ² as σ⁻²
  • Apply derivative rules, simplify
  • Final form: -n/σ + 1/σ³×Σ(xᵢ-μ)²

Solving for Maximum Likelihood Estimates

For μ:

  • Set derivative with respect to μ = 0
  • Multiply by σ², add nμ to both sides
  • Divide by n: μ = (1/n)×Σxᵢ
  • Result: μ̂ = sample mean

For σ:

  • Set derivative with respect to σ = 0
  • Multiply by σ, add n, multiply by σ²
  • Divide by n, take square root
  • Result: σ̂ = sample standard deviation

Final Results

  • Maximum likelihood estimate for μ = mean of data
  • Maximum likelihood estimate for σ = standard deviation of data
  • Mathematical proof confirms intuitive solutions
  • Mean determines center of normal distribution
  • Standard deviation determines width of normal curve

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome