AI summary by TwinMind

Saturated Models and Deviance

StatQuest with Josh Starmer

18m 39s

Introduction and Setup

  • Josh Stormer presenting StatQuest on saturated models and deviance statistics
  • Background noise warning: jackhammer construction nearby
  • Context: saturated models appear in log-likelihood based R-squared (McFadden's pseudo R-squared), residual deviance formulas
  • Part of generalized linear models framework
  • Applicable to logistic regression, linear models, other methods

Mouse Weight Example - Model Comparison Framework

  • Example: weighing mice with known standard deviation
  • Only need to estimate mean to fit normal curve
  • Null model: 1 parameter (mean only)
    • Likelihood = 0.03
    • Log-likelihood = -3.51
  • Proposed model: 2 parameters (two means)
    • Likelihood = 3.57
    • Log-likelihood = 1.27
  • Saturated model: 1 parameter per data point (6 total)
    • Maximum possible parameters
    • Likelihood = 1291.5
    • Log-likelihood = 7.16

Saturated Model Concept

  • Provides best-case scenario benchmark
  • Maxes out number of parameters possible
  • Establishes upper bound for model performance
  • Contrasts with null model (worst-case scenario)
  • Goal: proposed model likelihood should exceed null model, approach saturated model

Log-Likelihood Based R-Squared

  • Traditional R-squared boundaries:
    • Lower bound: sums of squares from null model
    • Upper bound: fixed value of 0 (perfect fit)
    • Results in R-squared = 1 for perfect fit
  • Log-likelihood R-squared problem:
    • No fixed upper bound value across all situations
    • Without saturated model: R-squared = 1.36 (impossible, should be 0-1)
    • With saturated model: R-squared = 0.45 (reasonable)
  • Solution: saturated model provides situation-specific upper bound
  • Ensures R-squared ranges between 0 and 1

Residual Deviance

  • Formula: 2 × (log-likelihood of saturated model - log-likelihood of proposed model)
  • Distribution: chi-squared with degrees of freedom = parameters in saturated model - parameters in proposed model
  • Example calculation: 2 × (7.16 - 1.27) = 11.78
  • Degrees of freedom: 6 - 2 = 4
  • P-value: 0.02 (significantly different from saturated model)
  • Requirement: models must be nested (proposed model = simpler version of saturated)

Null Deviance

  • Formula: 2 × (log-likelihood of saturated model - log-likelihood of null model)
  • Distribution: chi-squared with degrees of freedom = parameters in saturated model - parameters in null model
  • Example calculation: 2 × (7.16 - (-3.51)) = 21.34
  • Degrees of freedom: 6 - 1 = 5
  • P-value: ≈ 0 (significantly different from null model)

P-Value for R-Squared

  • Method 1: Difference between deviances
    • Chi-squared value = null deviance - residual deviance
    • 21.34 - 11.78 = 9.56
    • Degrees of freedom = proposed model parameters - null model parameters = 2 - 1 = 1
    • P-value = 0.0002
  • Method 2: Direct comparison (alternative approach)
    • 2 × (log-likelihood of proposed model - log-likelihood of null model)
    • Same result, but deviance method more commonly reported

Logistic Regression Simplification

  • Key insight: saturated model log-likelihood = 0 in logistic regression
  • Reason: saturated model fits data perfectly, likelihood = 1, log(1) = 0
  • Simplified equations:
    • R-squared reduces to simpler form without saturated model term
    • Residual deviance = -2 × log-likelihood of proposed model
    • Null deviance = -2 × log-likelihood of null model
  • Sign convention: deviances are positive (negative log-likelihoods × -2)
  • Conclusion: can ignore saturated model entirely in logistic regression

Summary

  • Saturated model provides upper bound for log-likelihood calculations
  • Essential for proper R-squared scaling in general case
  • Enables p-value calculation through deviance statistics
  • In logistic regression specifically: upper bound always zero, can be ignored
  • Deviance statistics remain useful for model comparison and significance testing

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome