Introduction and Global Audience

• Josh Dahmer hosts StatQuest live stream on bootstrapping
• Viewers from worldwide locations: New York, Pennsylvania, Malaysia, Mexico, Germany, Indonesia, India, South Korea, Montreal, Virginia, North Carolina, Moscow, China, Melbourne Australia, Sweden, Japan, Jordan, Romania, Georgia, Hyderabad, Iraq, Iran, Colombia, Brazil, Kerala India, Malta, Argentina, Sicily, Morocco, Austin Texas, Hungary, Bahrain, Portugal, Canada, Vancouver, Jaipur, London, Chile, Bangalore
• Personal note: debating haircut, hair getting long

Bootstrapping Overview

• Simple yet powerful technique used in statistical inference and machine learning
• Has wide variety of uses, "savior of statistics" keeping up with machine learning
• Next live stream planned for first Monday in July to cover machine learning applications

Drug Trial Example Setup

• New drug tested on 8 people with illness
• Results: 5 people felt better, 3 people felt worse
• Mean response = 0.5
• Question: Is 0.5 improvement due to random chance or actual drug effect?
• Hypothesis: "On average the drug has no effect on whether people feel better or worse"
• Need p-value to test hypothesis using bootstrapping

Bootstrap Process - Three Steps

1. Make bootstrap dataset: Sample with replacement from original data
2. Calculate something: Mean, median, standard deviation, etc.
3. Keep track of calculation: Build histogram of results

Statistical Inference Application

• Shifted original data left 0.5 units so mean = 0 (centers on null hypothesis)
• Created new number line, randomly selected 8 values with replacement from shifted data
• Repeated process thousands of times to create histogram of means
• Key insight: 8^8 = 16,777,216 possible combinations, bootstrap estimates full distribution with subset (typically 10,000 iterations)

P-value Calculation

• Histogram shows distribution of means around 0 under null hypothesis
• 36% of bootstrap means between -0.5 and 0.5 (probability = 0.36)
• 16% of bootstrap means ≤ -0.5 (probability = 0.16)
• 48% of bootstrap means ≥ 0.5 (probability = 0.48)
• P-value calculation: P(mean ≥ 0.5) + P(mean ≤ -0.5) = 0.48 + 0.16 = 0.64
• Since 0.64 > 0.05, fail to reject null hypothesis

Flexibility of Bootstrapping

• Can calculate any statistic: mean, median, standard deviation
• Median example: Original median = 0.8, shifted to center on 0
• Bootstrap median distribution has "funky" appearance (medians have strange distributions)
• P-value for median: P(median ≥ 1.8) + P(median ≤ -1.8) = 0.01 + 0.19 = 0.20
• Again fail to reject null hypothesis
• Advantage: Can use medians when data has outliers (more resilient than means)

Additional Applications

• Bootstrap original data (without shifting) to calculate:
  • Standard errors (standard deviation of bootstrap distribution)
  • 95% confidence intervals (covers 95% of bootstrap values)
• Shifting only necessary when testing null hypothesis of zero difference

Q&A Session Highlights

• Why shift data?: To create distribution centered on null hypothesis value (zero) for p-value calculation
• Other hypothesis values: Can shift to any hypothesized value, not just zero
• Implementation: Simple 2-3 line for loop in most programming languages (R, Python/NumPy)
• Two-tailed tests: Preferred approach (referenced previous video on p-value calculation)
• Connection to bagging: Bagging = "bootstrap aggregating" used in random forests, AdaBoost, XGBoost
• Upcoming content: Chain rule video as stepping stone to back propagation and neural networks