AI summary by TwinMind
Using Bootstrapping to Calculate p-values!!!
StatQuest with Josh Starmer
8m 7s
Introduction and Prerequisites
- StatQuest by Josh Starmer on bootstrapping part 2: calculating p-values
- Assumes familiarity with:
- Main ideas behind bootstrapping
- Hypothesis testing, null hypothesis, p-values
Bootstrap Part 1 Review
- New drug treatment for illness tested on 8 people
- Calculated mean value = 0.5
- Uncertainty if drug actually worked vs. random variation
- Four steps of bootstrapping:
- Make bootstrapped data set
- Calculate statistic (mean, median, standard deviation, etc.)
- Keep track of calculation
- Repeat steps 1-3 until nice distribution obtained
- Created 95% confidence interval for original mean
- 95% confidence interval covered zero → could not reject idea that drug does nothing
P-Value Calculation Method
- Null hypothesis: drug has no effect on illness
- Perfect world expectation: mean value = 0 if null hypothesis true
- Observed mean = 0.5
- Shifted all measurements left 0.5 units → mean of shifted data = 0
- Shifted data represents true null hypothesis
Bootstrap Process for P-Values
- Used bootstrapping on centered data (mean = 0)
- Created bootstrapped data sets with replacement
- Calculated mean values for each bootstrap
- Repeated process thousands of times
- Generated distribution of mean values under null hypothesis
Distribution Results
- 36% of bootstrap means between -0.5 and 0.5
- Probability of observing mean between -0.5 and 0.5 = 0.36
- 16% of bootstrap means ≤ -0.5
- Probability of observing mean ≤ -0.5 = 0.16
- 48% of bootstrap means ≥ 0.5
- Probability of observing mean ≥ 0.5 = 0.48
P-Value Calculation
- Original observed mean = 0.5
- P-value = probability of observing mean ≥ 0.5 OR ≤ -0.5 (more extreme than observed)
- P-value = 0.47 + 0.16 = 0.63
- 0.63 > 0.05 → fail to reject null hypothesis (drug makes no difference)
Method Generalization
- Same method works for any statistic
- Example with median:
- Calculate median of original data
- Shift data so median = 0
- Use bootstrapping to generate histogram of median values around zero
- Calculate p-value for observed median
Median Example
- Observed median = 1.8
- P-value = probability of bootstrap median ≥ 1.8 + probability ≤ -1.8
- P-value = 0.01 + 0.19 = 0.2
- Again fail to reject null hypothesis
- Medians more resilient to outliers than means
Conclusion
- Bootstrapping can test whatever statistic desired
- Useful when data has outliers (use medians vs means)
