AI summary by TwinMind

Support Vector Machines Part 3: The Radial (RBF) Kernel (Part 3 of 3)

StatQuest with Josh Starmer

15m 52s

Introduction & Context

  • StatQuest video by Josh Starmer on Support Vector Machines Part 3: Radial Kernel
  • Assumes familiarity with SVMs and polynomial kernel
  • Focus on radial kernel parameters, high-dimensional relationships, infinite dimensions

Training Data Problem

  • Drug dosage dataset: red dots (not cured), green dots (cured)
  • Drug only works with correct dosage (not too small/large)
  • Overlapping data prevented satisfying support vector classifier
  • Solution: Support Vector Machine with radial kernel (RBF - Radial Basis Function)

Radial Kernel Behavior

  • Finds support vector classifiers in infinite dimensions
  • Behaves like weighted nearest neighbor model
  • Closest observations (nearest neighbors) = high influence on classification
  • Distant observations = little influence on classification
  • Uses closest observations' classifications for new observation classification

Radial Kernel Mathematics

  • Variables a and b = two different dosage measurements
  • Squared distance between observations calculated
  • Influence = function of squared distance
  • Gamma parameter (determined by cross-validation) scales squared distance
  • Example calculations:
    • Gamma = 1, close observations: result = 0.11
    • Gamma = 2, same observations: result = 0.01 (less influence)
    • Gamma = 1, distant observations: result ≈ 0 (minimal influence)
  • Output represents high-dimensional relationship between observations

Polynomial Kernel Intuition

  • Simplified training dataset to 2 observations
  • R = 0 polynomial kernel analysis:
    • D = 2: single coordinate = original measurement squared
    • D = 3: single coordinate = original measurement cubed
    • D = 1: data stays in original location
  • Adding multiple polynomial kernels (R = 0):
    • D = 1 + D = 2: 2-dimensional coordinates
    • D = 1 + D = 2 + D = 3: 3-dimensional coordinates
    • Continuing to D = infinity: infinite dimensional coordinates

Mathematical Derivation (Advanced)

  • Radial kernel mathematical expansion
  • Set gamma = 1/2 to simplify calculations
  • Taylor series expansion introduction:
    • Function f(X) = infinite sum
    • e^X Taylor series derivation
    • Setting a = 0, e^0 = 1
    • Final form: e^X = 1 + X + X²/2! + X³/3! + ...

Radial Kernel as Infinite Sum

  • Taylor series expansion of e^(aB) term
  • Each term contains polynomial kernel with R = 0, D = 0 to infinity
  • Polynomial kernel equivalents:
    • R = 0, D = 0: equals 1
    • R = 0, D = 1: equals a × b
    • R = 0, D = 2: equals a × b²
  • Dot product conversion for e^(aB)
  • Final radial kernel = dot product with infinite dimensional coordinates
  • S = square root of first term for complete dot product form

Conclusion

  • Radial kernel equals dot product with infinite dimensional coordinates
  • Plugging numbers into radial kernel yields high-dimensional relationship between observations

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome