AI summary by TwinMind

StatQuest: edgeR and DESeq2, part 2 - Independent Filtering

StatQuest with Josh Starmer

21m 24s

Introduction to Multiple Testing Problem in RNA-Seq

  • StatQuest part 2 on EdgeR and DESeq2 analysis
  • Focus on filtering genes with low read counts (independent filtering)
  • Multiple testing problem: 5% false positive rate acceptable for 1-2 genes, but problematic for ~20,000 genes in genome
    • 5% of 20,000 = 1,000 false positives
  • FDR and Benjamini-Hochberg method compensate for multiple testing
  • Additional problem: bogus tests reduce power of true positive detection

Statistical Example Demonstrating the Problem

  • Mouse strain weight comparison example using t-tests
  • Two overlapping distributions (strain X vs strain Y)
  • 1,000 samples from different distributions: 949 true positives, 51 false negatives
  • Adding 1,000 tests from same distribution (no difference):
    • 993 p-values <0.05: 949 true positives + 44 false positives (4% false positive rate)
    • After FDR: 846 significant results (827 true positives = 89% survival, 19 false positives = 2%)
  • Increasing to 6,000 total tests (1,000 true + 5,000 bogus):
    • 1,215 p-values <0.05: 949 true positives + 266 false positives (22% false positive rate)
    • After FDR: 256 significant results (250 true positives = 26% survival, 6 false positives = 2%)
  • Increasing to 11,000 total tests (1,000 true + 10,000 bogus):
    • 1,430 p-values <0.05: 949 true positives + 481 false positives (34% false positive rate)
    • After FDR: 56 significant results (54 true positives = 6% survival, 2 false positives = 4%)

Key Finding on Bogus Tests Impact

  • Each increase in bogus tests reduces true positive survival rate after FDR adjustment
  • FDR controls false positive percentage regardless of test number
  • Filtering out bogus tests improves power to detect true positives
  • Room for improvement beyond Benjamini-Hochberg method

Gene Filtering Rationale

  • Genes with super low read counts not informative
  • Hard to get accurate read counts with only 1-2 transcripts in one sample type, 3-4 in another
  • Even biologically interesting genes may be filtered if counts too low

EdgeR Filtering Method

  • Remove all genes except those with >1 CPM in ≥2 samples
  • CPM = Counts Per Million (compensates for read depth differences between libraries)
  • CPM calculation formula: (reads for gene X in sample Y) / (total reads in sample Y / 1 million)
  • Example kidney dataset with 5.32 million total reads:
    • Gene 1: 65 reads → 12.3 CPM (keep - 2 samples >1 CPM)
    • Gene 2: 5 reads → 0.9 CPM (keep - 2 samples >1 CPM)
    • Gene 3: high reads in 1 sample only → filter out
    • Gene 4: <1 CPM in both samples with reads → filter out
    • Gene 5: reads in all 4 samples but none >1 CPM → filter out
    • Gene 6: >1 CPM in 2 samples from different tissues → keep

CPM Scaling Factor Considerations

  • 5 million reads: scaling factor = 5, so 5 reads = 1 CPM
  • 80 million reads: scaling factor = 80, so 1 CPM = 80 reads (might be too high)
  • 50,000 reads: scaling factor = 0.05, so 1 read = 20 CPM (very noisy range)
  • CPM cutoffs need adjustment based on sequencing depth

Real Data Example

  • Dataset: 22 million reads/sample average, 4 wild-type + 4 knockout samples
  • Ran EdgeR without filtering first, then applied different CPM cutoffs
  • Recommended threshold of 1 CPM too strict for high-read dataset
  • Lower threshold (0.2-0.4) identified ~200 more significant genes
  • Moral: try different CPM cutoffs after calculating p-values

DESeq2 vs EdgeR Filtering Differences

Difference 1: Sample vs Average Approach

  • EdgeR: examines individual samples, requires ≥2 samples above cutoff
  • DESeq2: looks at average normalized reads across all samples
  • Outlier example: Gene with one very high sample (5,000 CPM) and three low samples (0 CPM)
    • EdgeR: would filter out (only 1 sample above cutoff)
    • DESeq2: average = 1,250 CPM, would likely pass
  • DESeq2 has outlier detection method (kicks in with >2 samples per category)

Difference 2: Quantiles vs CPM Cutoffs

  • EdgeR: plots significant genes vs minimum CPM cutoff
  • DESeq2: plots significant genes vs quantiles (percentiles)
  • Quantile advantages:
    • 0 = 0% genes below cutoff
    • 0.2 = 20% genes below cutoff
    • 0.4 = 40% genes below cutoff
  • Quantiles independent of sequencing depth (10% always <0.1 quantile regardless of 8M or 80M reads)

Difference 3: Curve Fitting and Threshold Selection

  • DESeq2 fits curve to smooth out random noise bumps
  • Locates maximum on fitted curve
  • Threshold = maximum location - standard deviation between fitted curve and raw values
  • First quantile within noise range of peak becomes CPM cutoff
  • If no raw values above threshold, no filtering performed

Summary of Methods

  • EdgeR: keeps genes with >minimum CPM in ≥2 samples
  • DESeq2: keeps genes with average CPM > minimum CPM, uses quantile plots, fits curves, threshold = peak minus noise

Practical Advice

  • EdgeR users: determine CPM cutoff after calculating p-values
  • Can apply DESeq2's method to EdgeR's gene selection criteria
  • Template code available on StatQuest website
  • Cite both publications if mixing methods
  • DESeq2 users: be careful of outliers with only 2 samples per category

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome