AI summary by TwinMind
StatQuest: Random Forests in R
StatQuest with Josh Starmer
15m 9s
Introduction and Setup
- Josh Stormer StatQuest tutorial on building, using, and evaluating random forests in R
- Prerequisites: two previous stat quests on random forest theory
- All code downloadable via description link
- Libraries loaded:
- ggplot2 for graphs
- cowplot for improved ggplot2 defaults (overwrites ggSave function)
- randomForest package
Data Acquisition and Initial Processing
- Dataset: Heart disease data from UCI machine learning repository
- URL variable set to data location
- Data read using read.csv() from URL
- Initial data inspection with head() function
- Problem: no column labels initially
Data Cleaning and Preprocessing
- Column names added from UCI website documentation
- Structure analysis with str() function revealed issues:
- Sex column: should be factor (0=female, 1=male)
- CP (chest pain): should be factor (levels 1-3=different pain types, 4=no chest pain)
- CA and Thal: correctly factors but contained "?" instead of NA values
- Cleaning steps performed:
- Question marks converted to NAs
- Sex column: 0→F (female), 1→M (male), converted to factor
- Multiple columns converted to factors per UCI specifications
- CA column: corrected from strings to integers then factor
- Thal column: same treatment as CA
- HD (heart disease): converted to factor using ifelse (0→"healthy", 1→"unhealthy")
Missing Value Imputation
- Random seed set for reproducibility
- rfImpute() function used for NA imputation
- Formula: HD ~ . (predict heart disease using all other columns)
- Dataset: data
- Iterations: 20 (theory suggests 4-6 sufficient, but 20 didn't improve estimates)
- Results saved as data.imputed
- Out-of-bag (OOB) error rate printed after each iteration
- Error rate stabilized, indicating optimal estimates achieved
Random Forest Model Building
- randomForest() function called with:
- Formula: HD ~ . (predict heart disease from all other columns)
- Dataset: data.imputed
- proximity=TRUE (for clustering analysis)
- Model saved as "model"
Model Performance Evaluation
- Model summary output included:
- Type: Classification (vs regression or unsupervised)
- Number of trees: 500 (default)
- Variables tried at each split: 3 (default for classification = √(number of variables))
- OOB error estimate: 83.5% correctly classified
- Confusion matrix:
- 141 healthy correctly classified as healthy
- 27 unhealthy incorrectly classified as healthy
- 23 healthy incorrectly classified as unhealthy
- 112 unhealthy correctly classified as unhealthy
Optimal Number of Trees Analysis
- Error rate plotting using model$err.rate matrix
- Matrix structure: OOB error rate, healthy error rate, unhealthy error rate by tree number
- Data formatted for ggplot2 visualization
- Graph showed:
- Blue line: unhealthy classification error rate
- Green line: overall OOB error rate
- Red line: healthy classification error rate
- Error rates decreased with more trees
- Test with 1000 trees:
- Same OOB error rate as 500 trees
- No improvement in confusion matrix
- Error rates stabilized after ~500 trees
- Conclusion: 500 trees sufficient
Optimal Variables per Split Analysis
- Loop testing mtry values 1-10
- For each value:
- Built random forest with 1000 trees
- Recorded OOB error rate from final iteration
- Results: mtry=3 (default) had lowest OOB error rate
- Conclusion: default value was optimal
Multidimensional Scaling (MDS) Plot
- Distance matrix created from 1 - proximity matrix
- cmdscale() applied for classical multidimensional scaling
- Percentage variation calculated for x and y axes
- Results plotted with ggplot2:
- Unhealthy samples clustered on left side
- Healthy samples clustered on right side
- X-axis: 47% of variation
- Y-axis: 14% of variation
- Patient 253 noted as potential misdiagnosis candidate
- New patients clustering in unhealthy region would indicate likely heart disease
