AI summary by TwinMind

StatQuest: Random Forests in R

StatQuest with Josh Starmer

15m 9s

Introduction and Setup

  • Josh Stormer StatQuest tutorial on building, using, and evaluating random forests in R
  • Prerequisites: two previous stat quests on random forest theory
  • All code downloadable via description link
  • Libraries loaded:
    • ggplot2 for graphs
    • cowplot for improved ggplot2 defaults (overwrites ggSave function)
    • randomForest package

Data Acquisition and Initial Processing

  • Dataset: Heart disease data from UCI machine learning repository
  • URL variable set to data location
  • Data read using read.csv() from URL
  • Initial data inspection with head() function
  • Problem: no column labels initially

Data Cleaning and Preprocessing

  • Column names added from UCI website documentation
  • Structure analysis with str() function revealed issues:
    • Sex column: should be factor (0=female, 1=male)
    • CP (chest pain): should be factor (levels 1-3=different pain types, 4=no chest pain)
    • CA and Thal: correctly factors but contained "?" instead of NA values
  • Cleaning steps performed:
    • Question marks converted to NAs
    • Sex column: 0→F (female), 1→M (male), converted to factor
    • Multiple columns converted to factors per UCI specifications
    • CA column: corrected from strings to integers then factor
    • Thal column: same treatment as CA
    • HD (heart disease): converted to factor using ifelse (0→"healthy", 1→"unhealthy")

Missing Value Imputation

  • Random seed set for reproducibility
  • rfImpute() function used for NA imputation
    • Formula: HD ~ . (predict heart disease using all other columns)
    • Dataset: data
    • Iterations: 20 (theory suggests 4-6 sufficient, but 20 didn't improve estimates)
    • Results saved as data.imputed
  • Out-of-bag (OOB) error rate printed after each iteration
  • Error rate stabilized, indicating optimal estimates achieved

Random Forest Model Building

  • randomForest() function called with:
    • Formula: HD ~ . (predict heart disease from all other columns)
    • Dataset: data.imputed
    • proximity=TRUE (for clustering analysis)
  • Model saved as "model"

Model Performance Evaluation

  • Model summary output included:
    • Type: Classification (vs regression or unsupervised)
    • Number of trees: 500 (default)
    • Variables tried at each split: 3 (default for classification = √(number of variables))
    • OOB error estimate: 83.5% correctly classified
    • Confusion matrix:
      • 141 healthy correctly classified as healthy
      • 27 unhealthy incorrectly classified as healthy
      • 23 healthy incorrectly classified as unhealthy
      • 112 unhealthy correctly classified as unhealthy

Optimal Number of Trees Analysis

  • Error rate plotting using model$err.rate matrix
  • Matrix structure: OOB error rate, healthy error rate, unhealthy error rate by tree number
  • Data formatted for ggplot2 visualization
  • Graph showed:
    • Blue line: unhealthy classification error rate
    • Green line: overall OOB error rate
    • Red line: healthy classification error rate
    • Error rates decreased with more trees
  • Test with 1000 trees:
    • Same OOB error rate as 500 trees
    • No improvement in confusion matrix
    • Error rates stabilized after ~500 trees
    • Conclusion: 500 trees sufficient

Optimal Variables per Split Analysis

  • Loop testing mtry values 1-10
  • For each value:
    • Built random forest with 1000 trees
    • Recorded OOB error rate from final iteration
  • Results: mtry=3 (default) had lowest OOB error rate
  • Conclusion: default value was optimal

Multidimensional Scaling (MDS) Plot

  • Distance matrix created from 1 - proximity matrix
  • cmdscale() applied for classical multidimensional scaling
  • Percentage variation calculated for x and y axes
  • Results plotted with ggplot2:
    • Unhealthy samples clustered on left side
    • Healthy samples clustered on right side
    • X-axis: 47% of variation
    • Y-axis: 14% of variation
    • Patient 253 noted as potential misdiagnosis candidate
    • New patients clustering in unhealthy region would indicate likely heart disease

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome