AI summary by TwinMind

How to calculate p-values

StatQuest with Josh Starmer

25m 15s

Introduction and P-Value Types

  • Focus on two-sided p-values (most common)
  • One-sided p-values rarely used, potentially dangerous
  • Prerequisites: understanding what p-values are and interpretation

Coin Flip Example - Two Heads

  • Hypothesis: coin is special after getting 2 heads in a row
  • Null hypothesis: coin is no different from normal coin
  • Normal coin: 50% chance heads, 50% chance tails each flip
  • Four possible outcomes after 2 flips: HH, HT, TH, TT
  • Probability calculations:
    • Two heads: 1/4 = 0.25
    • Two tails: 1/4 = 0.25
    • One head, one tail: 2/4 = 0.5
  • Order doesn't matter - each flip independent

P-Value Components and Calculation

  • P-value has three parts:
    1. Probability of observed outcome (0.25 for two heads)
    2. Probability of equally rare outcomes (0.25 for two tails)
    3. Probability of rarer/more extreme outcomes (0 in this case)
  • Total p-value for two heads: 0.25 + 0.25 + 0 = 0.5
  • Since 0.5 > 0.05, fail to reject null hypothesis
  • Coin not considered special

Rationale for Including Equally Rare and More Extreme Events

  • Flower analogy: rarity becomes less special when other equally rare things exist
  • Similar logic for rarer things making observation less special
  • Statistical significance requires considering all relevant probabilities

Five Coin Flips Example - Four Heads, One Tail

  • 32 total possible outcomes when flipping 5 times
  • Outcomes breakdown:
    • 5 heads: 1 way
    • 4 heads, 1 tail: 5 ways
    • 3 heads, 2 tails: 10 ways
    • 2 heads, 3 tails: 10 ways
    • 1 head, 4 tails: 5 ways
    • 5 tails: 1 way
  • P-value calculation:
    • Observed: 5/32
    • Equally rare: 5/32 (1 head, 4 tails)
    • More extreme: 2/32 (5 heads or 5 tails)
    • Total: 12/32 = 0.375
  • Since 0.375 > 0.05, fail to reject null hypothesis

Continuous Distributions - Height Example

  • Brazilian women height data (15-49 years, 1996)
  • Distribution properties:
    • Mean: 155.7 cm
    • 95% between 142-169 cm
    • 2.5% below 142 cm
    • 2.5% above 169 cm
  • P-values calculated using area under curve percentages

Height Distribution P-Value Examples

  • Person 142 cm tall:
    • P-value: 2.5% + 2.5% = 0.05
    • Borderline significance
  • Person 141 cm tall:
    • P-value: 1.6% + 1.6% = 0.032
    • Significant (< 0.05), reject null hypothesis
  • Person 155.4-156 cm (near mean):
    • Probability: 0.04
    • More extreme areas: 48% each side
    • Total p-value: 0.04 + 0.48 + 0.48 = 1.0
    • Not unusual, fail to reject null hypothesis

One-Sided vs Two-Sided P-Values

  • Drug recovery time example
  • Two-sided approach:
    • Tests for change in either direction
    • 4.5 days recovery: p-value = 0.016 + 0.016 = 0.032
    • 15.5 days recovery: p-value = 0.016 + 0.016 = 0.032
    • Both detect unusual results
  • One-sided approach:
    • Tests for change in predetermined direction only
    • 4.5 days recovery: p-value = 0.016 (significant)
    • 15.5 days recovery: p-value = 0.98 (not significant)
    • Dangerous: fails to detect harmful effects

Dangers of One-Sided P-Values

  • Only detects effects in expected direction
  • May miss harmful effects going opposite direction
  • Should be avoided or used only by experts
  • Two-sided p-values provide complete picture

Summary

  • P-value components:
    1. Probability of observation
    2. Probability of equally rare events
    3. Probability of more extreme events
  • Standard significance threshold: 0.05
  • Two-sided p-values recommended for safety

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome