AI summary by TwinMind

StatQuest: t-SNE, Clearly Explained

StatQuest with Josh Starmer

11m 47s

Introduction & Overview

  • StatQuest episode on t-SNE (t-distributed Stochastic Neighbor Embedding)
  • Produced by UNC Chapel Hill Genetics Department
  • Episode requested by viewers via comments and emails
  • t-SNE takes high-dimensional data sets and reduces to low-dimensional graphs while retaining original information
  • Recommends watching PCA StatQuest for dimensional reduction background

Basic Concept Demonstration

  • Uses simple 2D scatter plot → 1D number line example to explain concepts
  • Standard projection onto x or y axes creates "big mess" that doesn't preserve clustering
  • t-SNE finds way to project data preserving clustering structure
  • Process: start with original scatter plot → place points randomly on number line → move points iteratively until clustered

Point Movement Mechanism

  • Points move based on attraction/repulsion forces
  • Attraction: points close in original high-dimensional space pull together
  • Repulsion: points far apart in original space push away
  • Each step moves one point slightly based on net force from all other points
  • Iterative process continues until clustering preserved

Step 1: Calculate Similarity Scores in Original Space

  • Measure distances between all point pairs
  • Plot distances on normal curve centered on point of interest
  • Distance from point to curve = "unscaled similarity"
  • Process repeated for all points vs. point of interest
  • Normal distribution ensures: distant points = very low similarity, close points = high similarity

Scaling Similarities

  • Unscaled similarities must be scaled to sum to 1
  • Width of normal curve depends on local data density
  • Less dense regions → wider curves
  • Example: curve with standard deviation = 1 vs. standard deviation = 2
  • Scaling makes similarity scores equivalent across different density regions
  • Final scaled scores: tight cluster (0.82, 0.18) = loose cluster (0.82, 0.18)

Perplexity Parameter

  • t-SNE has perplexity parameter = expected density around each point
  • Affects curve width calculations
  • Makes clusters more similar than might be expected

Similarity Matrix Creation

  • Calculate similarity scores for every point as point of interest
  • Handle asymmetric similarities by averaging scores from both directions
  • Result: matrix where rows/columns represent similarity scores from each point
  • Red = high similarity, white = low similarity
  • Point similarity to itself set to zero (doesn't help clustering)

Step 2: Random Projection & t-Distribution Similarities

  • Randomly project original data onto number line (low-dimensional space)
  • Calculate similarity scores using t-distribution instead of normal distribution
  • t-distribution: less tall in middle, taller tails compared to normal distribution
  • Create second similarity matrix for projected data
  • Initial projection creates "mess" compared to original similarity matrix

Optimization Process

  • Goal: make projected similarity matrix match original similarity matrix
  • Move points in small steps, choosing direction that improves matrix matching
  • Like chess game - can't solve all at once, requires iterative moves
  • Each step makes low-dimensional similarities more like high-dimensional similarities

Why t-Distribution is Used

  • Without t-distribution, clusters would clump in middle and be harder to distinguish
  • t-distribution's heavier tails prevent over-clustering in center
  • Enables better separation of distinct clusters in final visualization

Scalability

  • Simple 2D→1D example uses same concepts as complex high-dimensional datasets
  • Method scales to much larger, more complex data visualizations

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome