AI summary by TwinMind
Saturated Models and Deviance
StatQuest with Josh Starmer
18m 39s
Introduction and Setup
- Josh Stormer presenting StatQuest on saturated models and deviance statistics
- Background noise warning: jackhammer construction nearby
- Context: saturated models appear in log-likelihood based R-squared (McFadden's pseudo R-squared), residual deviance formulas
- Part of generalized linear models framework
- Applicable to logistic regression, linear models, other methods
Mouse Weight Example - Model Comparison Framework
- Example: weighing mice with known standard deviation
- Only need to estimate mean to fit normal curve
- Null model: 1 parameter (mean only)
- Likelihood = 0.03
- Log-likelihood = -3.51
- Proposed model: 2 parameters (two means)
- Likelihood = 3.57
- Log-likelihood = 1.27
- Saturated model: 1 parameter per data point (6 total)
- Maximum possible parameters
- Likelihood = 1291.5
- Log-likelihood = 7.16
Saturated Model Concept
- Provides best-case scenario benchmark
- Maxes out number of parameters possible
- Establishes upper bound for model performance
- Contrasts with null model (worst-case scenario)
- Goal: proposed model likelihood should exceed null model, approach saturated model
Log-Likelihood Based R-Squared
- Traditional R-squared boundaries:
- Lower bound: sums of squares from null model
- Upper bound: fixed value of 0 (perfect fit)
- Results in R-squared = 1 for perfect fit
- Log-likelihood R-squared problem:
- No fixed upper bound value across all situations
- Without saturated model: R-squared = 1.36 (impossible, should be 0-1)
- With saturated model: R-squared = 0.45 (reasonable)
- Solution: saturated model provides situation-specific upper bound
- Ensures R-squared ranges between 0 and 1
Residual Deviance
- Formula: 2 × (log-likelihood of saturated model - log-likelihood of proposed model)
- Distribution: chi-squared with degrees of freedom = parameters in saturated model - parameters in proposed model
- Example calculation: 2 × (7.16 - 1.27) = 11.78
- Degrees of freedom: 6 - 2 = 4
- P-value: 0.02 (significantly different from saturated model)
- Requirement: models must be nested (proposed model = simpler version of saturated)
Null Deviance
- Formula: 2 × (log-likelihood of saturated model - log-likelihood of null model)
- Distribution: chi-squared with degrees of freedom = parameters in saturated model - parameters in null model
- Example calculation: 2 × (7.16 - (-3.51)) = 21.34
- Degrees of freedom: 6 - 1 = 5
- P-value: ≈ 0 (significantly different from null model)
P-Value for R-Squared
- Method 1: Difference between deviances
- Chi-squared value = null deviance - residual deviance
- 21.34 - 11.78 = 9.56
- Degrees of freedom = proposed model parameters - null model parameters = 2 - 1 = 1
- P-value = 0.0002
- Method 2: Direct comparison (alternative approach)
- 2 × (log-likelihood of proposed model - log-likelihood of null model)
- Same result, but deviance method more commonly reported
Logistic Regression Simplification
- Key insight: saturated model log-likelihood = 0 in logistic regression
- Reason: saturated model fits data perfectly, likelihood = 1, log(1) = 0
- Simplified equations:
- R-squared reduces to simpler form without saturated model term
- Residual deviance = -2 × log-likelihood of proposed model
- Null deviance = -2 × log-likelihood of null model
- Sign convention: deviances are positive (negative log-likelihoods × -2)
- Conclusion: can ignore saturated model entirely in logistic regression
Summary
- Saturated model provides upper bound for log-likelihood calculations
- Essential for proper R-squared scaling in general case
- Enables p-value calculation through deviance statistics
- In logistic regression specifically: upper bound always zero, can be ignored
- Deviance statistics remain useful for model comparison and significance testing
