AI summary by TwinMind
Logistic Regression in R, Clearly Explained!!!!
StatQuest with Josh Starmer
17m 15s
Data Setup and Preparation
- Using UCI machine learning repository heart disease dataset (same as random forests tutorial)
- Created URL variable pointing to data location
- Used head() function to view first 6 rows - columns initially unlabeled
- Added column names from UCI website specifications
- Structure revealed data type issues requiring cleanup
Data Cleaning Process
- Converted question marks to NA values throughout dataset
- Sex column transformation:
- Changed 0 to "F" (female), 1 to "M" (male)
- Converted to factor variable
- Converted multiple columns to appropriate factor types per UCI specifications
- CA column fixes:
- Originally contained question marks instead of NAs
- Corrected assumption from strings to integers
- Converted to factor
- Applied same process to Thal column
- HD (heart disease) column transformation:
- Used ifelse() function to convert 0 to "healthy", 1 to "unhealthy"
- Made into factor for readability
Missing Data Analysis
- Initial dataset: 303 samples total
- Found 6 samples with NA values
- NA sample characteristics:
- 5 of 6 samples were male
- 2 of 6 had heart disease
- Decision: removed NA samples rather than imputing values
- Final dataset: 297 samples after removal
Data Distribution Verification
- Used xtabs() function to verify balanced representation
- Checked heart disease vs sex distribution:
- Both healthy/unhealthy patients represented across male/female samples
- Verified all 4 chest pain (CP) levels had sufficient patient representation
- Identified potential issue: resting electrocardiographic results level 1 only had 4 patients
- Decided to proceed despite low representation
Simple Logistic Regression Model
- Built basic model predicting heart disease using only gender
- GLM function call structure:
- Formula syntax: sex predicts heart disease
- Specified binomial family for logistic regression
- Stored results in "logistic" variable
- Model equation: Heart Disease = -1.0438 + 1.2737 × (patient is male)
- Coefficient interpretations:
- Intercept (-1.0438): log odds of female having heart disease
- Male coefficient (1.2737): increase in log odds for males vs females
- Represents log odds ratio of male vs female heart disease risk
- Statistical significance: both p-values well below 0.05
- Wald test computed for both coefficients
Model Output Interpretation
- Deviance residuals: well-centered on zero, roughly symmetrical
- Dispersion parameter: set to 1 (default for logistic regression)
- Variance derived from mean rather than estimated separately
- Potential for underestimated variance if needed
- Null deviance vs residual deviance for model comparison
- AIC: residual deviance adjusted for number of parameters
- Fisher scoring iterations: convergence speed indicator
Comprehensive Logistic Regression Model
- Built full model using all variables: HD ~ . (dot notation)
- Key findings from summary:
- Age not useful predictor (large p-value)
- Median age 56 - mostly older patients explains lack of age effect
- Gender remained good predictor
- Model improvement metrics:
- Residual deviance much smaller than simple model
- AIC much smaller than simple model
Model Performance Evaluation
- McFadden's pseudo R-squared calculation:
- Extracted log-likelihood from null model (null deviance ÷ -2)
- Extracted log-likelihood from full model (residual deviance ÷ -2)
- Calculated pseudo R² = 0.55
- Interpreted as overall effect size
- P-value calculation using chi-squared distribution
- Result: tiny p-value indicating R² not due to chance
Visualization and Results
- Created graph showing predicted probabilities vs actual heart disease status
- Patient classification results:
- Turquoise patients (with heart disease): predicted high probability
- Salmon patients (without heart disease): predicted low probability
- Model performed well in separating groups
- Graph creation process:
- Built data frame with probabilities and actual status
- Sorted by probability (low to high)
- Added ranking column
- Used ggplot2 and cowplot libraries
- Saved as PDF file
