AI summary by TwinMind
How to calculate p-values
StatQuest with Josh Starmer
25m 15s
Introduction and P-Value Types
- Focus on two-sided p-values (most common)
- One-sided p-values rarely used, potentially dangerous
- Prerequisites: understanding what p-values are and interpretation
Coin Flip Example - Two Heads
- Hypothesis: coin is special after getting 2 heads in a row
- Null hypothesis: coin is no different from normal coin
- Normal coin: 50% chance heads, 50% chance tails each flip
- Four possible outcomes after 2 flips: HH, HT, TH, TT
- Probability calculations:
- Two heads: 1/4 = 0.25
- Two tails: 1/4 = 0.25
- One head, one tail: 2/4 = 0.5
- Order doesn't matter - each flip independent
P-Value Components and Calculation
- P-value has three parts:
- Probability of observed outcome (0.25 for two heads)
- Probability of equally rare outcomes (0.25 for two tails)
- Probability of rarer/more extreme outcomes (0 in this case)
- Total p-value for two heads: 0.25 + 0.25 + 0 = 0.5
- Since 0.5 > 0.05, fail to reject null hypothesis
- Coin not considered special
Rationale for Including Equally Rare and More Extreme Events
- Flower analogy: rarity becomes less special when other equally rare things exist
- Similar logic for rarer things making observation less special
- Statistical significance requires considering all relevant probabilities
Five Coin Flips Example - Four Heads, One Tail
- 32 total possible outcomes when flipping 5 times
- Outcomes breakdown:
- 5 heads: 1 way
- 4 heads, 1 tail: 5 ways
- 3 heads, 2 tails: 10 ways
- 2 heads, 3 tails: 10 ways
- 1 head, 4 tails: 5 ways
- 5 tails: 1 way
- P-value calculation:
- Observed: 5/32
- Equally rare: 5/32 (1 head, 4 tails)
- More extreme: 2/32 (5 heads or 5 tails)
- Total: 12/32 = 0.375
- Since 0.375 > 0.05, fail to reject null hypothesis
Continuous Distributions - Height Example
- Brazilian women height data (15-49 years, 1996)
- Distribution properties:
- Mean: 155.7 cm
- 95% between 142-169 cm
- 2.5% below 142 cm
- 2.5% above 169 cm
- P-values calculated using area under curve percentages
Height Distribution P-Value Examples
- Person 142 cm tall:
- P-value: 2.5% + 2.5% = 0.05
- Borderline significance
- Person 141 cm tall:
- P-value: 1.6% + 1.6% = 0.032
- Significant (< 0.05), reject null hypothesis
- Person 155.4-156 cm (near mean):
- Probability: 0.04
- More extreme areas: 48% each side
- Total p-value: 0.04 + 0.48 + 0.48 = 1.0
- Not unusual, fail to reject null hypothesis
One-Sided vs Two-Sided P-Values
- Drug recovery time example
- Two-sided approach:
- Tests for change in either direction
- 4.5 days recovery: p-value = 0.016 + 0.016 = 0.032
- 15.5 days recovery: p-value = 0.016 + 0.016 = 0.032
- Both detect unusual results
- One-sided approach:
- Tests for change in predetermined direction only
- 4.5 days recovery: p-value = 0.016 (significant)
- 15.5 days recovery: p-value = 0.98 (not significant)
- Dangerous: fails to detect harmful effects
Dangers of One-Sided P-Values
- Only detects effects in expected direction
- May miss harmful effects going opposite direction
- Should be avoided or used only by experts
- Two-sided p-values provide complete picture
Summary
- P-value components:
- Probability of observation
- Probability of equally rare events
- Probability of more extreme events
- Standard significance threshold: 0.05
- Two-sided p-values recommended for safety
