AI summary by TwinMind

Ridge, Lasso and Elastic-Net Regression in R

StatQuest with Josh Starmer

17m 51s

Introduction & GLMnet Overview

  • Elastic regression tutorial using GLMnet library in R
  • GLMnet = Generalized Linear Models (GLM) + elastic net
  • Supports linear regression, logistic regression, other models
  • GLMnet parameterization differs from standard elastic net formulation

GLMnet Parameter System

  • Uses single lambda + alpha instead of two separate lambdas
  • Alpha controls penalty mixture:
    • Alpha = 0: pure Ridge regression (lasso penalty = 0)
    • Alpha = 1: pure Lasso regression (ridge penalty = 0)
    • Alpha between 0-1: mixture of both penalties
  • Lambda controls overall penalty strength:
    • Lambda = 0: standard least squares/maximum likelihood
    • Lambda > 0: elastic net penalty applied, shrinks parameter estimates

Synthetic Dataset Creation

  • N = 1,000 samples, P = 5,000 parameters
  • Only 15 parameters truly predictive, remaining 4,985 = random noise
  • Matrix X: 1,000 × 5,000 from standard normal distribution (mean=0, sd=1)
  • Vector Y: sum of first 15 columns + noise from standard normal
  • Data split: 66% training (0.66 × N), 34% testing

Ridge Regression Implementation (Alpha = 0)

  • cv.glmnet() function with 10-fold cross-validation
  • Parameters:
    • type.measure = "MSE" (mean squared error)
    • alpha = 0 (Ridge regression)
    • family = "gaussian" (linear regression)
  • Lambda selection: lambda.1se (simplest model within 1 SE of minimum)
  • Results: MSE = 14.47 on test set

Lasso Regression Implementation (Alpha = 1)

  • Same cv.glmnet() setup, alpha = 1
  • Results: MSE = 1.19 (much better than Ridge)
  • Demonstrates Lasso's superiority for sparse data

Elastic Net Implementation (Alpha = 0.5)

  • Alpha = 0.5 (50/50 Ridge/Lasso mixture)
  • Results: MSE = 1.24 (slightly worse than pure Lasso)

Comprehensive Alpha Testing

  • Loop through alpha values: 0, 0.1, 0.2, ..., 1.0
  • Store all fits in list_of_fits
  • Calculate MSE for each alpha value on test set
  • Results show alpha = 1 (Lasso) performs best for this dataset
  • Slight variations due to random initialization of parameters

Key Technical Notes

  • cv.glmnet() requires separate X and Y inputs (no formula notation)
  • For logistic regression: family = "binomial", type.measure = "deviance"
  • set.seed() recommended for reproducible results
  • lambda.1se preferred over lambda.min for simpler models

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome