AI summary by TwinMind

Machine Learning Fundamentals: Bias and Variance

StatQuest with Josh Starmer

6m 36s

Introduction & Setup

  • Josh Stormer presenting StatQuest on machine learning fundamentals
  • Hurricane Florence occurring during recording but didn't stop production
  • Topic: bias and variance in machine learning, clearly explained

Data Example: Mouse Weight vs Height

  • Measured weight and height of multiple mice, plotted on graph
  • Pattern observed: light mice tend to be short, heavier mice tend to be taller
  • After certain weight threshold, mice don't get taller, just more obese
  • Goal: predict mouse height given its weight
  • True mathematical relationship unknown, so using ML methods to approximate
  • True relationship curve kept in figure for reference

Data Splitting

  • Split data into two sets:
    • Training set (blue dots) - for training ML algorithms
    • Testing set (green dots) - for testing algorithms

Linear Regression Method

  • First ML algorithm: linear regression (aka least squares)
  • Fits straight line to training set
  • Limitation: straight line lacks flexibility to replicate arc in true relationship
  • No matter how well fitted, line will never curve
  • Bias definition: inability for ML method to capture true relationship
  • Linear regression has relatively large amount of bias due to inflexibility

Squiggly Line Method

  • Second ML method: fits flexible squiggly line to training set
  • Super flexible, hugs training set along arc of true relationship
  • Can handle arc in true relationship between weight and height
  • Has very little bias due to flexibility

Training Set Performance Comparison

  • Compared fit quality using sums of squares calculation:
    • Measure distances from fit lines to data
    • Square distances (prevents negative distances canceling positive ones)
    • Add squared distances up
  • Squiggly line results: distances between line and data all = 0
  • Winner on training set: squiggly line (perfect fit)

Testing Set Performance

  • Calculated sums of squares for testing set
  • Winner on testing set: straight line
  • Key finding: squiggly line did great on training set but terrible on testing set

Variance Definition & Analysis

  • Variance: difference in fits between different data sets
  • Squiggly line characteristics:
    • Low bias (flexible, adapts to curve)
    • High variance (vastly different sums of squares for different data sets)
    • Hard to predict performance on future data sets
    • Performance inconsistent - sometimes good, sometimes terrible
  • Straight line characteristics:
    • Relatively high bias (cannot capture curve)
    • Relatively low variance (similar sums of squares across different data sets)
    • Consistently good predictions (not great, but reliable)

Overfitting Terminology

  • Overfitting definition: when model fits training set really well but not testing set
  • Squiggly line example of overfitting

Ideal Algorithm Goals

  • Low bias: accurately model true relationship
  • Low variance: produce consistent predictions across different data sets
  • Achieved by finding sweet spot between simple and complex models

Methods for Finding Sweet Spot

  • Three commonly used methods mentioned:
    • Regularization
    • Boosting
    • Bagging
  • Random forest StatQuest shows bagging example in action
  • Future StatQuests will cover regularization and boosting

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome