AI summary by TwinMind

Logistic Regression Details Pt 2: Maximum Likelihood

StatQuest with Josh Starmer

10m 23s

Introduction and Context

  • Follow-up video on logistic regression series by Josh Starmer (StatQuest)
  • Previous video covered logistic regression coefficients (Part 1)
  • Focus: how logistic regression lines are optimized using maximum likelihood
  • Comparison with linear regression fitting methods

Linear Regression Review

  • Uses least squares method for line fitting
  • Process: measure residuals (distances between data and line)
  • Square residuals to prevent negative/positive cancellation
  • Sum all squared residuals
  • Rotate line iteratively to find smallest sum of squared residuals
  • Line with smallest sum = best fit

Logistic Regression Setup

  • Example: determining effect of weight on obesity in mice
  • Data: obese mice vs non-obese mice
  • Goal: draw best fitting squiggle for data
  • Y-axis transformation: probability of obesity → log odds of obesity

Problem with Least Squares in Logistic Regression

  • Log odds transformation pushes raw data to positive/negative infinity
  • Residuals (distances from data points to line) = positive/negative infinity
  • Cannot use least squares method
  • Solution: maximum likelihood estimation

Maximum Likelihood Process

  • Project original data points onto candidate line
  • Each sample gets candidate log-odds value
    • Example: log odds = 2.1, log odds = 1.4
  • Transform candidate log odds to candidate probabilities
  • Formula: P = e^(log odds) / (1 + e^(log odds))

Mathematical Transformation Details

  • Converting probability → log odds equation to log odds → probability
  • Steps:
    • Exponentiate both sides
    • Multiply both sides by (1 - P)
    • Multiply (1 - P) and e^(log odds)
    • Add P × e^(log odds) to both sides
    • Factor out P on left side
    • Divide both sides by (1 + e^(log odds))

Likelihood Calculations

Obese Mice Likelihoods

  • Likelihood = predicted probability (y-axis value where point intersects squiggle)
  • Example values: 0.49, 0.9, 0.91, 0.92
  • Overall likelihood for obese mice = product of individual likelihoods

Non-obese Mice Likelihoods

  • Likelihood = 1 - probability of being obese
  • Examples:
    • Mouse with 0.9 obesity probability → 1 - 0.9 = 0.1 likelihood not obese
    • Mouse with 0.3 obesity probability → 1 - 0.3 = 0.7 likelihood not obese
    • Mice with 0.01 obesity probability → 1 - 0.01 = 0.99 likelihood not obese

Log-Likelihood Calculation

  • Statisticians prefer log-likelihood over likelihood
  • Same squiggle maximizes both likelihood and log-likelihood
  • Log-likelihood: add logs of individual likelihoods (vs multiplying likelihoods)
  • Example result: log-likelihood = -3.77

Optimization Process

  • Rotate line and recalculate log-likelihood
  • Project data onto new line
  • Transform log odds to probabilities
  • Calculate new log-likelihood
  • Example: second line log-likelihood = -4.15 (worse than first)
  • Algorithm intelligently rotates line to increase log-likelihood
  • Finds optimal fit after few rotations
  • Final result: line that maximizes likelihood = best fit

Future Topics

  • Need r-squared value and p-value for model evaluation
  • Challenge: must calculate without usual residuals
  • Next video: r-squared and p-values for logistic regression

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome