AI summary by TwinMind
StatQuest: t-SNE, Clearly Explained
StatQuest with Josh Starmer
11m 47s
Introduction & Overview
- StatQuest episode on t-SNE (t-distributed Stochastic Neighbor Embedding)
- Produced by UNC Chapel Hill Genetics Department
- Episode requested by viewers via comments and emails
- t-SNE takes high-dimensional data sets and reduces to low-dimensional graphs while retaining original information
- Recommends watching PCA StatQuest for dimensional reduction background
Basic Concept Demonstration
- Uses simple 2D scatter plot → 1D number line example to explain concepts
- Standard projection onto x or y axes creates "big mess" that doesn't preserve clustering
- t-SNE finds way to project data preserving clustering structure
- Process: start with original scatter plot → place points randomly on number line → move points iteratively until clustered
Point Movement Mechanism
- Points move based on attraction/repulsion forces
- Attraction: points close in original high-dimensional space pull together
- Repulsion: points far apart in original space push away
- Each step moves one point slightly based on net force from all other points
- Iterative process continues until clustering preserved
Step 1: Calculate Similarity Scores in Original Space
- Measure distances between all point pairs
- Plot distances on normal curve centered on point of interest
- Distance from point to curve = "unscaled similarity"
- Process repeated for all points vs. point of interest
- Normal distribution ensures: distant points = very low similarity, close points = high similarity
Scaling Similarities
- Unscaled similarities must be scaled to sum to 1
- Width of normal curve depends on local data density
- Less dense regions → wider curves
- Example: curve with standard deviation = 1 vs. standard deviation = 2
- Scaling makes similarity scores equivalent across different density regions
- Final scaled scores: tight cluster (0.82, 0.18) = loose cluster (0.82, 0.18)
Perplexity Parameter
- t-SNE has perplexity parameter = expected density around each point
- Affects curve width calculations
- Makes clusters more similar than might be expected
Similarity Matrix Creation
- Calculate similarity scores for every point as point of interest
- Handle asymmetric similarities by averaging scores from both directions
- Result: matrix where rows/columns represent similarity scores from each point
- Red = high similarity, white = low similarity
- Point similarity to itself set to zero (doesn't help clustering)
Step 2: Random Projection & t-Distribution Similarities
- Randomly project original data onto number line (low-dimensional space)
- Calculate similarity scores using t-distribution instead of normal distribution
- t-distribution: less tall in middle, taller tails compared to normal distribution
- Create second similarity matrix for projected data
- Initial projection creates "mess" compared to original similarity matrix
Optimization Process
- Goal: make projected similarity matrix match original similarity matrix
- Move points in small steps, choosing direction that improves matrix matching
- Like chess game - can't solve all at once, requires iterative moves
- Each step makes low-dimensional similarities more like high-dimensional similarities
Why t-Distribution is Used
- Without t-distribution, clusters would clump in middle and be harder to distinguish
- t-distribution's heavier tails prevent over-clustering in center
- Enables better separation of distinct clusters in final visualization
Scalability
- Simple 2D→1D example uses same concepts as complex high-dimensional datasets
- Method scales to much larger, more complex data visualizations
