AI summary by TwinMind

StatQuest: Principal Component Analysis (PCA), Step-by-Step

StatQuest with Josh Starmer

21m 57s

Introduction & Data Setup

  • StatQuest tutorial on Principal Component Analysis (PCA) using Singular Value Decomposition (SVD)
  • Instructor: Josh Starmer
  • Dataset: Gene transcription measurements (Gene 1 & Gene 2) in 6 different mice
  • Alternative examples: students (math/reading scores), businesses (market cap/employees)

Visualization Concepts

  • 1 gene: Number line plot showing mice 1,2,3 (high values) vs mice 4,5,6 (low values)
  • 2 genes: 2D X-Y graph (Gene 1 = x-axis, Gene 2 = y-axis)
    • Mice 1,2,3 cluster on right side
    • Mice 4,5,6 cluster on lower left
  • 3 genes: 3D graph with additional axis
    • Smaller dots = larger Gene 3 values (further away)
    • Larger dots = smaller Gene 3 values (closer)
  • 4+ genes: Cannot visualize directly, requires PCA for 2D representation

PCA Process - 2 Gene Example

  • Calculate averages for Gene 1 and Gene 2
  • Find center of data
  • Shift data so center sits on origin
    • Maintains relative positioning of data points
  • Fit line through origin by rotation until optimal fit achieved

Mathematical Foundation - Line Fitting

  • Two equivalent approaches for measuring fit:
    • Minimize distances from data points to line
    • Maximize distances from projected points to origin
  • Pythagorean theorem relationship: A² = B² + C²
    • A = distance from point to origin (constant)
    • B = distance from point to line
    • C = distance from projected point to origin
  • PCA maximizes sum of squared distances from projected points to origin (easier calculation)

PC1 Calculation Process

  • Project data onto line, measure distances D1 through D6
  • Square all distances to prevent negative/positive cancellation
  • Sum squared distances = SS distances
  • Rotate line and repeat until maximum SS distances found
  • Optimal line = Principal Component 1 (PC1)

PC1 Characteristics

  • Slope = 0.25 (4 units Gene 1 : 1 unit Gene 2)
  • "Cocktail recipe": Mix 4 parts Gene 1 with 1 part Gene 2
  • Linear combination terminology: Mathematical description of variable mixing
  • Gene 1 more important for data spread description

Vector Scaling & Terminology

  • Original recipe length = 4.12 units
  • Scale to unit vector (length = 1) by dividing by 4.12
  • Scaled values: 0.97 parts Gene 1, 0.242 parts Gene 2
  • Terminology:
    • Singular Vector/Eigenvector for PC1: Unit vector proportions
    • Loading Scores: Proportions of each gene
    • Eigenvalue for PC1: Average of sum of squared distances
    • Singular Value for PC1: Square root of sum of squared distances

PC2 Calculation

  • PC2 = Line perpendicular to PC1 through origin (no optimization needed in 2D)
  • Recipe: -1 parts Gene 1 to 4 parts Gene 2
  • Unit vector: -0.242 parts Gene 1, 0.97 parts Gene 2
  • Gene 2 is 4x more important than Gene 1 for PC2 projections

Final PCA Plot Creation

  • Rotate so PC1 is horizontal
  • Use projected points to position samples in PCA plot
  • Sample positioning based on projection coordinates

Variation Analysis Example

  • PC1 variation = 15, PC2 variation = 3
  • Total variation = 18
  • PC1 accounts for 15/18 = 83% of total variation
  • PC2 accounts for 3/18 = 17% of total variation
  • Scree Plot: Graphical representation of variation percentages per PC

3-Gene Example Extension

  • Same process: Center data, find PC1 (best fitting line)
  • PC1 recipe has 3 ingredients (Gene 3 most important)
  • PC2: Next best line perpendicular to PC1 (Gene 1 most important)
  • PC3: Best line perpendicular to both PC1 and PC2
  • Variation breakdown: PC1 = 79%, PC2 = 15%, PC3 = 6%
  • PC1 + PC2 = 94% variation → Good 2D approximation of 3D data

Higher Dimensional Applications

  • 4+ genes: Cannot visualize but PCA math still works
  • Example: PC1 + PC2 = 90% variation → Use 2D PCA plot
  • Poor case: PC3 + PC4 account for substantial variation → 2D representation less accurate
  • Even noisy PCA plots useful for identifying data clusters

Practical Considerations

  • Number of PCs = minimum of (number of variables, number of samples)
  • PCA enables dimensionality reduction while preserving most variation
  • Cluster identification possible even with noisy representations

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome