AI summary by TwinMind

StatQuest: Random Forests Part 1 - Building, Using and Evaluating

StatQuest with Josh Starmer

9m 54s

Introduction to Random Forests

  • Decision trees: easy to build, use, interpret but lack accuracy
  • Quote from "Elements of Statistical Learning" (machine learning Bible): trees have accuracy issues preventing them from being ideal predictive tools
  • Trees work great with training data but inflexible with new samples
  • Random forests combine decision tree simplicity with flexibility for vast accuracy improvement

Step 1: Creating Bootstrap Dataset

  • Example uses 4-sample dataset (artificially small)
  • Bootstrap dataset creation: randomly select samples from original with replacement
  • Same sample can be picked multiple times
  • Demonstration: 4 random selections created bootstrap dataset with duplicates

Step 2: Building Decision Tree with Variable Subset

  • Create decision tree using bootstrap dataset
  • Only use random subset of variables/columns at each step
  • Example: consider only 2 variables at each step (instead of all 4)
  • Root node selection: randomly chose "good blood circulation" and "blocked arteries" as candidates
  • "Good blood circulation" performed best for sample separation
  • Continue tree building considering random subset at each node split
  • Process creates tree using: 1) bootstrap dataset, 2) random variable subsets

Random Forest Construction Process

  • Repeat steps 1-2 multiple times (ideally hundreds)
  • Example shows 6 trees for space constraints
  • Bootstrap sampling + variable subset selection creates wide tree variety
  • Variety makes random forests more effective than individual decision trees

Using Random Forest for Prediction

  • New patient example: determine heart disease presence
  • Run patient data through each tree in forest
  • Track each tree's prediction (yes/no votes)
  • Final prediction: majority vote across all trees
  • Example: "yes" received most votes → conclude patient has heart disease
  • Terminology: bootstrapping + aggregation = "bagging"

Evaluating Random Forest Accuracy

  • Bootstrap sampling creates duplicates, leaving ~1/3 original data unused
  • Unused data = "out-of-bag" dataset (not "out-of-boot" as presenter would prefer)
  • Out-of-bag samples not used in tree creation
  • Run out-of-bag samples through trees built without them
  • Example results: some trees correctly/incorrectly labeled out-of-bag samples
  • Final out-of-bag prediction: majority vote
  • Accuracy measurement: proportion of correctly classified out-of-bag samples
  • Out-of-bag error: proportion incorrectly classified

Optimization Process

  • Compare different variable counts per step (e.g., 2 vs 3 variables)
  • Test multiple settings, choose most accurate random forest
  • Process: 1) build random forest, 2) estimate accuracy, 3) adjust parameters, 4) repeat
  • Starting point: typically square root of total variables
  • Try settings above/below starting value

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome