Introduction and Prerequisites

• StatQuest episode on statistical power by Josh Starmer
• Prerequisites: familiarity with p-values and normal distribution
• Referenced related quests for background knowledge

Statistical Power Definition and Basic Example

• Power = probability of correctly rejecting null hypothesis
• Alternative definition: probability of correctly getting small p-value
• Example: two mouse diet distributions (special diet vs normal food)
  • Special diet mice weigh less than normal diet mice
  • Minimal overlap between distributions
  • Small sample comparison yielded p-value = 0.0004
  • p-value < 0.05 → correctly reject null hypothesis
  • High probability of repeated correct rejections = high power

Edge Cases and Limitations

• Occasional overlapping data samples produce large p-values (>0.05)
  • Cannot correctly reject null hypothesis despite knowing distributions differ
  • Large p-value = "can't reject null hypothesis"
• When no true difference exists (same distribution):
  • Null hypothesis is true
  • No concept of "correctly rejecting" applies
  • Power concept doesn't apply

Low Power Scenario

• Less effective special diet example:
  • Still creates different distributions but with significant overlap
  • Sample: 3 mice per group
  • Result: p-value = 0.34 → fail to reject null hypothesis
  • Most repeated tests yield large p-values
  • Occasional non-overlapping samples produce small p-values
  • Low probability of correct rejection = low power

Increasing Statistical Power

• Primary method: increase sample size (number of measurements)
• Power analysis determines required sample size for adequate power
• Reference to future StatQuest on power analyses

Summary Points

• Power = probability of correctly rejecting null hypothesis
• High power: distributions with little overlap
• Low power: overlapping distributions + small sample sizes
• Solution: increase sample size
• Power analysis guides sample size determination