AI summary by TwinMind

Using Bootstrapping to Calculate p-values!!!

StatQuest with Josh Starmer

8m 7s

Introduction and Prerequisites

  • StatQuest by Josh Starmer on bootstrapping part 2: calculating p-values
  • Assumes familiarity with:
    • Main ideas behind bootstrapping
    • Hypothesis testing, null hypothesis, p-values

Bootstrap Part 1 Review

  • New drug treatment for illness tested on 8 people
  • Calculated mean value = 0.5
  • Uncertainty if drug actually worked vs. random variation
  • Four steps of bootstrapping:
    1. Make bootstrapped data set
    2. Calculate statistic (mean, median, standard deviation, etc.)
    3. Keep track of calculation
    4. Repeat steps 1-3 until nice distribution obtained
  • Created 95% confidence interval for original mean
  • 95% confidence interval covered zero → could not reject idea that drug does nothing

P-Value Calculation Method

  • Null hypothesis: drug has no effect on illness
  • Perfect world expectation: mean value = 0 if null hypothesis true
  • Observed mean = 0.5
  • Shifted all measurements left 0.5 units → mean of shifted data = 0
  • Shifted data represents true null hypothesis

Bootstrap Process for P-Values

  • Used bootstrapping on centered data (mean = 0)
  • Created bootstrapped data sets with replacement
  • Calculated mean values for each bootstrap
  • Repeated process thousands of times
  • Generated distribution of mean values under null hypothesis

Distribution Results

  • 36% of bootstrap means between -0.5 and 0.5
    • Probability of observing mean between -0.5 and 0.5 = 0.36
  • 16% of bootstrap means ≤ -0.5
    • Probability of observing mean ≤ -0.5 = 0.16
  • 48% of bootstrap means ≥ 0.5
    • Probability of observing mean ≥ 0.5 = 0.48

P-Value Calculation

  • Original observed mean = 0.5
  • P-value = probability of observing mean ≥ 0.5 OR ≤ -0.5 (more extreme than observed)
  • P-value = 0.47 + 0.16 = 0.63
  • 0.63 > 0.05 → fail to reject null hypothesis (drug makes no difference)

Method Generalization

  • Same method works for any statistic
  • Example with median:
    • Calculate median of original data
    • Shift data so median = 0
    • Use bootstrapping to generate histogram of median values around zero
    • Calculate p-value for observed median

Median Example

  • Observed median = 1.8
  • P-value = probability of bootstrap median ≥ 1.8 + probability ≤ -1.8
  • P-value = 0.01 + 0.19 = 0.2
  • Again fail to reject null hypothesis
  • Medians more resilient to outliers than means

Conclusion

  • Bootstrapping can test whatever statistic desired
  • Useful when data has outliers (use medians vs means)

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome