AI summary by TwinMind
Logistic Regression Details Pt1: Coefficients
StatQuest with Josh Starmer
19m 2s
Introduction and Context
- Part 1 of logistic regression series by Josh Starmer
- Follows up on "Logistic Regression Clearly Explained"
- Focus: coefficients determination and interpretation
- Covers continuous variables (weight predicting obesity) and discrete variables (mutated gene vs obesity)
- Logistic regression = specific type of generalized linear model (GLM)
Linear Regression Review
- Basic equation: y-intercept + slope
- Can predict values for any input (including negative weights)
- No domain/range limitations makes solving easier
- Foundation for understanding logistic regression
Key Difference: Y-Axis Transformation
- Linear regression: y-axis can be any number (-∞ to +∞)
- Logistic regression: y-axis confined to probabilities (0 to 1)
- Solution: transform y-axis from probability to log odds using logit function
- Logit transformation allows y-axis to range from -∞ to +∞
Logit Function Transformation Details
- P = 0.5 → log odds = 0 (center of new y-axis)
- P = 0.731 → log odds = 1
- P = 0.88 → log odds = 2
- P = 0.95 → log odds = 3
- P = 1 → log odds = +∞ (obese samples)
- P = 0 → log odds = -∞ (non-obese samples)
- Transforms squiggly S-curve into straight line
Continuous Variable Coefficients (Weight → Obesity)
- First coefficient: y-axis intercept = -3.476
- When weight = 0, log odds of obesity = -3.476
- Standard error provided
- Z-value = estimate ÷ standard error
- Z-value < 2 standard deviations → not statistically significant
- Large p-value confirms non-significance
- Second coefficient: slope = 1.825
- For every 1 unit weight increase, log odds increase by 1.825
- Also not statistically significant (small sample size)
- Uses Wald's test for significance testing
Discrete Variable Setup (Gene Mutation → Obesity)
- Similar to t-test using linear models
- Left side: mice with normal gene
- Right side: mice with mutated gene
- Some mice obese, some not in each group
Linear Model T-Test Review
- Fit two lines: mean for normal gene group, mean for mutated gene group
- Equation: mean(normal) + difference(mutated - normal)
- Design matrix with columns for coefficients
- First column (b1): turns on/off first coefficient
- Second column (b2): turns on/off second coefficient
- Testing if difference coefficient = 0
Discrete Variable Logistic Regression
- Transform y-axis to log odds of obesity
- First line: log odds for normal gene mice
- Second line: log odds for mutated gene mice
- Coefficients:
- Intercept: log odds for gene normal
- Gene mutant term: log odds ratio (difference on log scale)
- Log odds ratio shows how mutation increases/decreases obesity odds
Numerical Example
- Log odds gene normal = log(2/9)
- Log odds gene mutated = log(7/3)
- Calculate coefficients through subtraction
- Intercept = log odds for normal gene = -1.5
- Gene mutant coefficient = log odds ratio
Statistical Significance Testing
- Intercept Z-value = -1.9
- Less than 2 standard deviations from 0
- Not significantly different from 0
- P-value > 0.05 confirms
- Gene mutant Z-value > 2
- Statistically significant
- P-value < 0.05 confirms
Key Insights
- Logistic regression coefficients work exactly like linear models
- Only difference: coefficients expressed in log odds scale
- All linear model techniques apply: multiple regression, ANOVA
- Foundation for advanced logistic regression applications
