Introduction to False Discovery Rates

• StatQuest episode from UNC Chapel Hill genetics department
• Focus: False Discovery Rate (FDR) in high throughput sequencing
• Main concept: FDR = tool to weed out bad data that looks good

RNA Sequencing Example Setup

• Example: measuring gene expression using RNA-seq
• Gene X (imaginary gene) plotted with y-axis = gene counts, x-axis = samples
• Comparing normal wild-type mice vs drug-treated mice
• RNA sequencing imperfect → slightly different values each measurement
• Normal mice measurements follow bell-shaped curve distribution
  • Most values close to mean (middle of curve)
  • Rare large values (right side)
  • Rare small values (left side)

Statistical Testing Basics

• Sample 1: 3 measurements from normal mice (middle of distribution)
• Sample 2: 3 measurements from normal mice (middle of distribution)
• Statistical test comparison: P-value >0.05 (samples overlap)
• Rare case: samples don't overlap → P-value <0.05 = false positive
• False positive rate: 5% of time when comparing same distribution

Multiple Testing Problem

• Human/mouse cells: ≥10,000 transcribed genes
• Testing all 10,000 genes from same mice type
• 5% of 10,000 = 500 false positives
• Result: 500 genes appear interesting when they're not

P-Value Distribution Patterns

Same Distribution Testing (10,000 tests)

• P-values uniformly distributed
• ~510 P-values (5.1%) <0.05
• Each bin contains ~5% of P-values (~500 per bin)
• Equal probability for any bin

Different Distribution Testing

• Black distribution: control mice
• Red distribution: drug-treated mice (increased gene transcription)
• Higher likelihood samples separated/don't overlap
• Most P-values <0.05 (statistically significant)
• P-values >0.05 = false negatives (can reduce by increasing sample size)

Real Experiment Scenario

• Testing all active genes in neuronal cells
• Drug affects 1,000 genes → different distributions → skewed P-values
• 9,000 genes unaffected → same distribution → uniform P-values
• Combined histogram = sum of both patterns
• Uniform P-values from unaffected genes
• Left-side P-values = mixture of affected + unaffected genes

Eyeball Method for True Positives

• Identify uniform distribution region
• ~450 P-values per bin in uniform part
• Extend line as cutoff for true positives
• Focus on P-values <0.05: ~450 above line (false positives), ~450 below line
• Isolate true positives: consider smallest 450 P-values
• Works because affected gene P-values skewed within bins, unaffected spread evenly

Benjamini-Hochberg Method Details

• Converts eyeball method into mathematical formula
• Adjusts P-values by making them larger
• Example: P-value 0.04 → FDR-adjusted 0.06 (no longer significant)
• FDR <0.05 cutoff: <5% of significant results are false positives
• Trade-off: not all true positives remain significant after adjustment

Mathematical Procedure

10-Sample Example

• Order P-values smallest to largest
• Rank P-values, create spaces for FDR-adjusted values
• Largest FDR-adjusted P-value = largest original P-value
• For each subsequent value: min(previous adjusted P-value, current P-value × total tests ÷ rank)
• Continue until all P-values adjusted
• False positive P-value becomes non-significant

Large-Scale Example

• Blue boxes: P-values from different distributions (affected genes)
• Red boxes: P-values from same distribution (unaffected genes, including false positives)
• Eyeball method draws line separating false from true positives
• Benjamini-Hochberg results: false positives >0.05, true positives remain <0.05