AI summary by TwinMind
StatQuest: Decision Trees, Part 2 - Feature Selection and Missing Data
StatQuest with Josh Starmer
5m 16s
Decision Tree Review
- Previous StatQuest built decision tree for heart disease prediction
- Tree structure: good blood circulation → blocked arteries → chest pain
- Final prediction: 17 people with chest pain had heart disease, 3 did not
- Patients without chest pain likely don't have heart disease
- Some branches became leaf nodes when chest pain didn't reduce impurity
Feature Selection in Decision Trees
- Automatic feature selection occurs when features don't reduce impurity
- Example: if chest pain never reduced impurity score, it wouldn't be used
- Tree would only contain good circulation and blocked arteries features
- Can set threshold requiring large enough impurity reduction
- Results in simpler trees that avoid overfitting
- Overfitting definition: tree performs well on training data but poorly on new data
- Decision trees prone to overfitting
- Requiring large impurity reductions prevents overfitting
Missing Data Handling Strategies
- Original approach: skip patients with missing data (blocked arteries example)
- Alternative approaches for categorical data:
- Use most common option (if "yes" occurs more than "no", fill with "yes")
- Find column with highest correlation to missing feature
- Example: chest pain highly correlated with blocked arteries
- Patient 1: both "no"
- Patient 2: both "yes"
- Patient 3: both "no"
- Patient 4: chest pain "yes" → predict blocked arteries "yes"
Missing Continuous Data Methods
- For weight data example:
- Replace with mean or median
- Use correlation with other variables (height highly correlated with weight)
- Perform linear regression between correlated columns
- Use least squares line to predict missing weight values
- Multiple strategies available for guessing missing data values
