AI summary by TwinMind

Regularization Part 2: Lasso (L1) Regression

StatQuest with Josh Starmer

8m 19s

Introduction and Context

  • Part 2 of regularization series by Josh Stormer (StatQuest)
  • Builds on Ridge regression video
  • Focus: Lasso regression similarities/differences with Ridge

Ridge Regression Review

  • Training setup: mouse weight/size data split into training (red dots) and testing (green dots)
  • Least squares approach: minimized sum of squared residuals
    • Low bias on training data
    • High variance on testing data (poor generalization)
  • Ridge regression: minimized sum of squared residuals + λ × slope²
    • More bias than least squares
    • Significant variance reduction
    • Better long-term predictions

Lasso Regression Definition

  • Formula: minimizes sum of squared residuals + λ × |slope| (absolute value vs squared)
  • λ determined via cross-validation (0 to positive infinity)
  • Results in orange line with slight bias but less variance than least squares

Similarities Between Ridge and Lasso

  • Both reduce prediction sensitivity to small training datasets
  • Both applicable to:
    • Simple linear regression contexts
    • Logistic regression settings
    • Complex models combining continuous (weight) and discrete (high fat diet) data
  • Both penalties exclude y-intercept from regularization
  • Both can shrink parameters unequally based on data patterns

Key Difference: Variable Selection Capability

  • Ridge regression: can only shrink slopes asymptotically close to 0 (never exactly 0)
  • Lasso regression: can shrink slopes all the way to 0
  • Demonstration with complex equation predicting size:
    • Useful predictors: weight, high fat diet
    • Useless predictors: astrological sign, airspeed of swallow
    • Ridge: shrinks all parameters but keeps all variables
    • Lasso: shrinks useful parameters slightly, eliminates useless variables completely

Performance Comparison

  • Lasso regression: better at reducing variance in models with many useless variables
  • Ridge regression: tends to perform better when most variables are useful
  • Lasso advantage: creates simpler, more interpretable final equations

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome