AI summary by TwinMind
Machine Learning Fundamentals: Bias and Variance
StatQuest with Josh Starmer
6m 36s
Introduction & Setup
- Josh Stormer presenting StatQuest on machine learning fundamentals
- Hurricane Florence occurring during recording but didn't stop production
- Topic: bias and variance in machine learning, clearly explained
Data Example: Mouse Weight vs Height
- Measured weight and height of multiple mice, plotted on graph
- Pattern observed: light mice tend to be short, heavier mice tend to be taller
- After certain weight threshold, mice don't get taller, just more obese
- Goal: predict mouse height given its weight
- True mathematical relationship unknown, so using ML methods to approximate
- True relationship curve kept in figure for reference
Data Splitting
- Split data into two sets:
- Training set (blue dots) - for training ML algorithms
- Testing set (green dots) - for testing algorithms
Linear Regression Method
- First ML algorithm: linear regression (aka least squares)
- Fits straight line to training set
- Limitation: straight line lacks flexibility to replicate arc in true relationship
- No matter how well fitted, line will never curve
- Bias definition: inability for ML method to capture true relationship
- Linear regression has relatively large amount of bias due to inflexibility
Squiggly Line Method
- Second ML method: fits flexible squiggly line to training set
- Super flexible, hugs training set along arc of true relationship
- Can handle arc in true relationship between weight and height
- Has very little bias due to flexibility
Training Set Performance Comparison
- Compared fit quality using sums of squares calculation:
- Measure distances from fit lines to data
- Square distances (prevents negative distances canceling positive ones)
- Add squared distances up
- Squiggly line results: distances between line and data all = 0
- Winner on training set: squiggly line (perfect fit)
Testing Set Performance
- Calculated sums of squares for testing set
- Winner on testing set: straight line
- Key finding: squiggly line did great on training set but terrible on testing set
Variance Definition & Analysis
- Variance: difference in fits between different data sets
- Squiggly line characteristics:
- Low bias (flexible, adapts to curve)
- High variance (vastly different sums of squares for different data sets)
- Hard to predict performance on future data sets
- Performance inconsistent - sometimes good, sometimes terrible
- Straight line characteristics:
- Relatively high bias (cannot capture curve)
- Relatively low variance (similar sums of squares across different data sets)
- Consistently good predictions (not great, but reliable)
Overfitting Terminology
- Overfitting definition: when model fits training set really well but not testing set
- Squiggly line example of overfitting
Ideal Algorithm Goals
- Low bias: accurately model true relationship
- Low variance: produce consistent predictions across different data sets
- Achieved by finding sweet spot between simple and complex models
Methods for Finding Sweet Spot
- Three commonly used methods mentioned:
- Regularization
- Boosting
- Bagging
- Random forest StatQuest shows bagging example in action
- Future StatQuests will cover regularization and boosting
