AI summary by TwinMind
StatQuest: edgeR and DESeq2, part 2 - Independent Filtering
StatQuest with Josh Starmer
21m 24s
Introduction to Multiple Testing Problem in RNA-Seq
- StatQuest part 2 on EdgeR and DESeq2 analysis
- Focus on filtering genes with low read counts (independent filtering)
- Multiple testing problem: 5% false positive rate acceptable for 1-2 genes, but problematic for ~20,000 genes in genome
- 5% of 20,000 = 1,000 false positives
- FDR and Benjamini-Hochberg method compensate for multiple testing
- Additional problem: bogus tests reduce power of true positive detection
Statistical Example Demonstrating the Problem
- Mouse strain weight comparison example using t-tests
- Two overlapping distributions (strain X vs strain Y)
- 1,000 samples from different distributions: 949 true positives, 51 false negatives
- Adding 1,000 tests from same distribution (no difference):
- 993 p-values <0.05: 949 true positives + 44 false positives (4% false positive rate)
- After FDR: 846 significant results (827 true positives = 89% survival, 19 false positives = 2%)
- Increasing to 6,000 total tests (1,000 true + 5,000 bogus):
- 1,215 p-values <0.05: 949 true positives + 266 false positives (22% false positive rate)
- After FDR: 256 significant results (250 true positives = 26% survival, 6 false positives = 2%)
- Increasing to 11,000 total tests (1,000 true + 10,000 bogus):
- 1,430 p-values <0.05: 949 true positives + 481 false positives (34% false positive rate)
- After FDR: 56 significant results (54 true positives = 6% survival, 2 false positives = 4%)
Key Finding on Bogus Tests Impact
- Each increase in bogus tests reduces true positive survival rate after FDR adjustment
- FDR controls false positive percentage regardless of test number
- Filtering out bogus tests improves power to detect true positives
- Room for improvement beyond Benjamini-Hochberg method
Gene Filtering Rationale
- Genes with super low read counts not informative
- Hard to get accurate read counts with only 1-2 transcripts in one sample type, 3-4 in another
- Even biologically interesting genes may be filtered if counts too low
EdgeR Filtering Method
- Remove all genes except those with >1 CPM in ≥2 samples
- CPM = Counts Per Million (compensates for read depth differences between libraries)
- CPM calculation formula: (reads for gene X in sample Y) / (total reads in sample Y / 1 million)
- Example kidney dataset with 5.32 million total reads:
- Gene 1: 65 reads → 12.3 CPM (keep - 2 samples >1 CPM)
- Gene 2: 5 reads → 0.9 CPM (keep - 2 samples >1 CPM)
- Gene 3: high reads in 1 sample only → filter out
- Gene 4: <1 CPM in both samples with reads → filter out
- Gene 5: reads in all 4 samples but none >1 CPM → filter out
- Gene 6: >1 CPM in 2 samples from different tissues → keep
CPM Scaling Factor Considerations
- 5 million reads: scaling factor = 5, so 5 reads = 1 CPM
- 80 million reads: scaling factor = 80, so 1 CPM = 80 reads (might be too high)
- 50,000 reads: scaling factor = 0.05, so 1 read = 20 CPM (very noisy range)
- CPM cutoffs need adjustment based on sequencing depth
Real Data Example
- Dataset: 22 million reads/sample average, 4 wild-type + 4 knockout samples
- Ran EdgeR without filtering first, then applied different CPM cutoffs
- Recommended threshold of 1 CPM too strict for high-read dataset
- Lower threshold (0.2-0.4) identified ~200 more significant genes
- Moral: try different CPM cutoffs after calculating p-values
DESeq2 vs EdgeR Filtering Differences
Difference 1: Sample vs Average Approach
- EdgeR: examines individual samples, requires ≥2 samples above cutoff
- DESeq2: looks at average normalized reads across all samples
- Outlier example: Gene with one very high sample (5,000 CPM) and three low samples (0 CPM)
- EdgeR: would filter out (only 1 sample above cutoff)
- DESeq2: average = 1,250 CPM, would likely pass
- DESeq2 has outlier detection method (kicks in with >2 samples per category)
Difference 2: Quantiles vs CPM Cutoffs
- EdgeR: plots significant genes vs minimum CPM cutoff
- DESeq2: plots significant genes vs quantiles (percentiles)
- Quantile advantages:
- 0 = 0% genes below cutoff
- 0.2 = 20% genes below cutoff
- 0.4 = 40% genes below cutoff
- Quantiles independent of sequencing depth (10% always <0.1 quantile regardless of 8M or 80M reads)
Difference 3: Curve Fitting and Threshold Selection
- DESeq2 fits curve to smooth out random noise bumps
- Locates maximum on fitted curve
- Threshold = maximum location - standard deviation between fitted curve and raw values
- First quantile within noise range of peak becomes CPM cutoff
- If no raw values above threshold, no filtering performed
Summary of Methods
- EdgeR: keeps genes with >minimum CPM in ≥2 samples
- DESeq2: keeps genes with average CPM > minimum CPM, uses quantile plots, fits curves, threshold = peak minus noise
Practical Advice
- EdgeR users: determine CPM cutoff after calculating p-values
- Can apply DESeq2's method to EdgeR's gene selection criteria
- Template code available on StatQuest website
- Cite both publications if mixing methods
- DESeq2 users: be careful of outliers with only 2 samples per category
