AI summary by TwinMind
StatQuest: Principal Component Analysis (PCA), Step-by-Step
StatQuest with Josh Starmer
21m 57s
Introduction & Data Setup
- StatQuest tutorial on Principal Component Analysis (PCA) using Singular Value Decomposition (SVD)
- Instructor: Josh Starmer
- Dataset: Gene transcription measurements (Gene 1 & Gene 2) in 6 different mice
- Alternative examples: students (math/reading scores), businesses (market cap/employees)
Visualization Concepts
- 1 gene: Number line plot showing mice 1,2,3 (high values) vs mice 4,5,6 (low values)
- 2 genes: 2D X-Y graph (Gene 1 = x-axis, Gene 2 = y-axis)
- Mice 1,2,3 cluster on right side
- Mice 4,5,6 cluster on lower left
- 3 genes: 3D graph with additional axis
- Smaller dots = larger Gene 3 values (further away)
- Larger dots = smaller Gene 3 values (closer)
- 4+ genes: Cannot visualize directly, requires PCA for 2D representation
PCA Process - 2 Gene Example
- Calculate averages for Gene 1 and Gene 2
- Find center of data
- Shift data so center sits on origin
- Maintains relative positioning of data points
- Fit line through origin by rotation until optimal fit achieved
Mathematical Foundation - Line Fitting
- Two equivalent approaches for measuring fit:
- Minimize distances from data points to line
- Maximize distances from projected points to origin
- Pythagorean theorem relationship: A² = B² + C²
- A = distance from point to origin (constant)
- B = distance from point to line
- C = distance from projected point to origin
- PCA maximizes sum of squared distances from projected points to origin (easier calculation)
PC1 Calculation Process
- Project data onto line, measure distances D1 through D6
- Square all distances to prevent negative/positive cancellation
- Sum squared distances = SS distances
- Rotate line and repeat until maximum SS distances found
- Optimal line = Principal Component 1 (PC1)
PC1 Characteristics
- Slope = 0.25 (4 units Gene 1 : 1 unit Gene 2)
- "Cocktail recipe": Mix 4 parts Gene 1 with 1 part Gene 2
- Linear combination terminology: Mathematical description of variable mixing
- Gene 1 more important for data spread description
Vector Scaling & Terminology
- Original recipe length = 4.12 units
- Scale to unit vector (length = 1) by dividing by 4.12
- Scaled values: 0.97 parts Gene 1, 0.242 parts Gene 2
- Terminology:
- Singular Vector/Eigenvector for PC1: Unit vector proportions
- Loading Scores: Proportions of each gene
- Eigenvalue for PC1: Average of sum of squared distances
- Singular Value for PC1: Square root of sum of squared distances
PC2 Calculation
- PC2 = Line perpendicular to PC1 through origin (no optimization needed in 2D)
- Recipe: -1 parts Gene 1 to 4 parts Gene 2
- Unit vector: -0.242 parts Gene 1, 0.97 parts Gene 2
- Gene 2 is 4x more important than Gene 1 for PC2 projections
Final PCA Plot Creation
- Rotate so PC1 is horizontal
- Use projected points to position samples in PCA plot
- Sample positioning based on projection coordinates
Variation Analysis Example
- PC1 variation = 15, PC2 variation = 3
- Total variation = 18
- PC1 accounts for 15/18 = 83% of total variation
- PC2 accounts for 3/18 = 17% of total variation
- Scree Plot: Graphical representation of variation percentages per PC
3-Gene Example Extension
- Same process: Center data, find PC1 (best fitting line)
- PC1 recipe has 3 ingredients (Gene 3 most important)
- PC2: Next best line perpendicular to PC1 (Gene 1 most important)
- PC3: Best line perpendicular to both PC1 and PC2
- Variation breakdown: PC1 = 79%, PC2 = 15%, PC3 = 6%
- PC1 + PC2 = 94% variation → Good 2D approximation of 3D data
Higher Dimensional Applications
- 4+ genes: Cannot visualize but PCA math still works
- Example: PC1 + PC2 = 90% variation → Use 2D PCA plot
- Poor case: PC3 + PC4 account for substantial variation → 2D representation less accurate
- Even noisy PCA plots useful for identifying data clusters
Practical Considerations
- Number of PCs = minimum of (number of variables, number of samples)
- PCA enables dimensionality reduction while preserving most variation
- Cluster identification possible even with noisy representations
