AI summary by TwinMind

StatQuest: Decision Trees, Part 2 - Feature Selection and Missing Data

StatQuest with Josh Starmer

5m 16s

Decision Tree Review

  • Previous StatQuest built decision tree for heart disease prediction
  • Tree structure: good blood circulation → blocked arteries → chest pain
  • Final prediction: 17 people with chest pain had heart disease, 3 did not
  • Patients without chest pain likely don't have heart disease
  • Some branches became leaf nodes when chest pain didn't reduce impurity

Feature Selection in Decision Trees

  • Automatic feature selection occurs when features don't reduce impurity
  • Example: if chest pain never reduced impurity score, it wouldn't be used
  • Tree would only contain good circulation and blocked arteries features
  • Can set threshold requiring large enough impurity reduction
  • Results in simpler trees that avoid overfitting
  • Overfitting definition: tree performs well on training data but poorly on new data
  • Decision trees prone to overfitting
  • Requiring large impurity reductions prevents overfitting

Missing Data Handling Strategies

  • Original approach: skip patients with missing data (blocked arteries example)
  • Alternative approaches for categorical data:
    • Use most common option (if "yes" occurs more than "no", fill with "yes")
    • Find column with highest correlation to missing feature
    • Example: chest pain highly correlated with blocked arteries
      • Patient 1: both "no"
      • Patient 2: both "yes"
      • Patient 3: both "no"
      • Patient 4: chest pain "yes" → predict blocked arteries "yes"

Missing Continuous Data Methods

  • For weight data example:
    • Replace with mean or median
    • Use correlation with other variables (height highly correlated with weight)
    • Perform linear regression between correlated columns
    • Use least squares line to predict missing weight values
  • Multiple strategies available for guessing missing data values

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome