AI summary by TwinMind

False Discovery Rates, FDR, clearly explained

StatQuest with Josh Starmer

18m 27s

Introduction to False Discovery Rates

  • StatQuest episode from UNC Chapel Hill genetics department
  • Focus: False Discovery Rate (FDR) in high throughput sequencing
  • Main concept: FDR = tool to weed out bad data that looks good

RNA Sequencing Example Setup

  • Example: measuring gene expression using RNA-seq
  • Gene X (imaginary gene) plotted with y-axis = gene counts, x-axis = samples
  • Comparing normal wild-type mice vs drug-treated mice
  • RNA sequencing imperfect → slightly different values each measurement
  • Normal mice measurements follow bell-shaped curve distribution
    • Most values close to mean (middle of curve)
    • Rare large values (right side)
    • Rare small values (left side)

Statistical Testing Basics

  • Sample 1: 3 measurements from normal mice (middle of distribution)
  • Sample 2: 3 measurements from normal mice (middle of distribution)
  • Statistical test comparison: P-value >0.05 (samples overlap)
  • Rare case: samples don't overlap → P-value <0.05 = false positive
  • False positive rate: 5% of time when comparing same distribution

Multiple Testing Problem

  • Human/mouse cells: ≥10,000 transcribed genes
  • Testing all 10,000 genes from same mice type
  • 5% of 10,000 = 500 false positives
  • Result: 500 genes appear interesting when they're not

P-Value Distribution Patterns

Same Distribution Testing (10,000 tests)

  • P-values uniformly distributed
  • ~510 P-values (5.1%) <0.05
  • Each bin contains ~5% of P-values (~500 per bin)
  • Equal probability for any bin

Different Distribution Testing

  • Black distribution: control mice
  • Red distribution: drug-treated mice (increased gene transcription)
  • Higher likelihood samples separated/don't overlap
  • Most P-values <0.05 (statistically significant)
  • P-values >0.05 = false negatives (can reduce by increasing sample size)

Real Experiment Scenario

  • Testing all active genes in neuronal cells
  • Drug affects 1,000 genes → different distributions → skewed P-values
  • 9,000 genes unaffected → same distribution → uniform P-values
  • Combined histogram = sum of both patterns
  • Uniform P-values from unaffected genes
  • Left-side P-values = mixture of affected + unaffected genes

Eyeball Method for True Positives

  • Identify uniform distribution region
  • ~450 P-values per bin in uniform part
  • Extend line as cutoff for true positives
  • Focus on P-values <0.05: ~450 above line (false positives), ~450 below line
  • Isolate true positives: consider smallest 450 P-values
  • Works because affected gene P-values skewed within bins, unaffected spread evenly

Benjamini-Hochberg Method Details

  • Converts eyeball method into mathematical formula
  • Adjusts P-values by making them larger
  • Example: P-value 0.04 → FDR-adjusted 0.06 (no longer significant)
  • FDR <0.05 cutoff: <5% of significant results are false positives
  • Trade-off: not all true positives remain significant after adjustment

Mathematical Procedure

10-Sample Example

  • Order P-values smallest to largest
  • Rank P-values, create spaces for FDR-adjusted values
  • Largest FDR-adjusted P-value = largest original P-value
  • For each subsequent value: min(previous adjusted P-value, current P-value × total tests ÷ rank)
  • Continue until all P-values adjusted
  • False positive P-value becomes non-significant

Large-Scale Example

  • Blue boxes: P-values from different distributions (affected genes)
  • Red boxes: P-values from same distribution (unaffected genes, including false positives)
  • Eyeball method draws line separating false from true positives
  • Benjamini-Hochberg results: false positives >0.05, true positives remain <0.05

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome