AI summary by TwinMind
StatQuest: PCA - Practical Tips
StatQuest with Josh Starmer
8m 19s
Data Scaling Requirements
- Variables must be on same scale to avoid bias
- Example: Math scores (0-100) vs Reading scores (0-10)
- Without scaling: PC1 = 0.99 math + 0.1 reading (biased toward math)
- With scaling: PC1 = 0.77 math + 0.77 reading (equal contribution)
- Standard practice: divide each variable by its standard deviation
- Wide range variables → large standard deviation → more scaling
- Narrow range variables → small standard deviation → minimal scaling
Data Centering Importance
- First step in PCA process - not all programs do this by default
- Without centering: PCA tries to fit line through origin
- Results in unexpected principal components
- Must verify PCA program centers data or center manually
Principal Component Limits
- Maximum PCs = minimum of (number of variables, number of samples)
- Two-dimensional example (math + reading scores):
- PC1: best fitting line through origin
- PC2: line perpendicular to PC1
- PC3: impossible in 2D space (cannot be perpendicular to both PC1 and PC2)
Sample Size Impact on PC Count
- Perfect correlation scenario:
- PC1 accounts for 100% variation
- PC2 eigenvalue = 0 (all projections go to origin)
- Effectively only 1 meaningful PC
- Two students example:
- Two points define single line
- Only 1 PC with meaningful eigenvalue
- PC2 eigenvalue = 0
- Three students, three variables:
- Three points define plane (2 axes)
- Maximum 2 PCs with eigenvalues > 0
- PC3 eigenvalue = 0
Mathematical Constraints
- Technically: 1 PC per variable in dataset
- Practically: fewer samples than variables limits meaningful PCs
- Upper bound = number of samples for PCs with eigenvalues > 0
