AI summary by TwinMind

Support Vector Machines Part 2: The Polynomial Kernel (Part 2 of 3)

StatQuest with Josh Starmer

7m 15s

Introduction and Context

  • Educational video by Josh Stormer on StatQuest
  • Focus: Support vector machines part 2, specifically polynomial kernel
  • Prerequisites: Assumes familiarity with basic support vector machines
  • Reference to previous StatQuest video (link in description)

Training Data Example

  • Dataset: Drug dosage measurements from patients
  • Red dots: Patients not cured
  • Green dots: Patients cured
  • Problem: Drug only works with correct dosage (not too small/large)
  • Challenge: Training data had significant overlap, couldn't find satisfying linear support vector classifier

Polynomial Kernel Solution

  • Added Y-axis coordinate by squaring original dosage measurements
  • Enabled linear separation in higher dimension
  • Used support vector machine with polynomial kernel
  • Successfully computed relationships between observations in higher dimension
  • Found good support vector classifier based on high-dimensional relationships

Polynomial Kernel Formula

  • Formula: (a·b + r)^d
  • Parameters:
    • a, b: Two different observations in dataset
    • r: Coefficient of polynomial
    • d: Degree of polynomial

First Example (r=1/2, d=2)

  • Set r = 1/2, d = 2
  • Expanded polynomial through multiplication
  • Resulted in dot product with three terms
  • High-dimensional coordinates:
    • X-axis: Original coordinates
    • Y-axis: Squared original coordinates
    • Z-axis: Constant (can be ignored)

Second Example (r=1, d=2)

  • Set r = 1, d = 2
  • Generated different polynomial and dot product
  • New coordinates:
    • X-axis: √2 × original dosage values
    • Y-axis: Original dosage values squared
    • Z-axis: Constant (ignored)
  • Both examples enable support vector classifier in higher dimension

Key Insight - Kernel Trick

  • Only need to calculate dot products between point pairs
  • Don't need to actually transform data to higher dimensions
  • Kernel directly computes high-dimensional relationships
  • Example calculation: Plugging dosages into kernel yields 2.25 as relationship value
  • Provides all necessary information for support vector classifier

Parameter Selection

  • r and d values determined using cross-validation
  • Once parameters selected, plug observations into formula
  • Calculate high-dimensional relationships through kernel math

Conclusion

  • Polynomial kernel enables SVM classification in cases where linear separation impossible
  • Kernel trick avoids explicit high-dimensional transformation
  • Cross-validation determines optimal parameters

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome