AI summary by TwinMind

Live 2020-03-02!!! Virus Models and p-hacking

StatQuest with Josh Starmer

30m 40s

Opening and Acknowledgments

  • Josh Dharma hosts StatQuest livestream wearing StatQuest t-shirt
  • Thanks viewers who used super chat during previous livestream
  • Acknowledges missing super chat shout-outs during last stream due to screen management

Question 1: Corona Virus Epidemic Modeling (Chi Yang)

  • Asked about best models for predicting coronavirus epidemic growth
  • Most commonly used models: differential equation-based models
  • Factors considered:
    • Social networks (real-world, not digital)
    • Disease transmission characteristics (contagiousness, spread patterns)
  • Differential equations described as:
    • Pretty elaborate equations
    • Fun and intuitive
    • Computers do most computational work
  • Josh's experience: took differential equation classes in grad school, found them enjoyable
  • Current focus: statistics and machine learning (differential equations less popular in these areas)
  • Upcoming content priorities: neural networks, time series, other topics
  • May cover differential equations if heavily requested
  • Referenced academic article on mathematical models for epidemics (technical but readable introduction/abstract)

Question 2: P-Hacking Explanation (Josh Star Murr)

  • Context: previous livestream discussed ANOVA with four diet groups
  • Measured weight gain after 4 weeks on different diets
  • Diet C showed obvious difference from Diet A
  • Problem: testing only groups that "look different" = P-hacking

P-Hacking Demonstration

  • Simulated recovery time from illness using normal distribution
  • 95% of area under curve between 5-15 days (2.5% each tail)
  • Took multiple groups of 3 people from same distribution
  • Tested pairs repeatedly:
    • Most p-values > 0.05 (not significant)
    • Eventually found p-value = 0.02 (false positive)
  • Cherry-picking tests that look different leads to false positives
  • No p-value adjustment works on single test

P-Hacking Definition

  • Misuse and abuse of analysis techniques
  • Results in being fooled by false positives
  • Not necessarily malicious - legitimate temptation to test obvious differences
  • Solution: plan tests before seeing data

Subtle Form of P-Hacking

  • Example: p-value = 0.06 (just above 0.05 threshold)
  • Temptation: add more data to get significant result
  • Added one measurement per group → p-value = 0.02
  • Still false positive (all data from same distribution)
  • Josh's simulation results: borderline p-values (0.05-0.1)
    • Added one measurement per group
    • 1/3 became significant (33% false positive rate)

Power Analysis Solution

  • Must do power analysis before first experiment
  • Power = probability of correctly rejecting null hypothesis
  • High separation between distributions = high power
  • Low separation = low power
  • Can increase power by increasing sample size
  • 80% power analysis: determines sample size needed to correctly reject null hypothesis 80% of time
  • When p-value just above 0.05: use current means/standard deviations to do proper power analysis rather than adding data

Q&A Session

Bayesian Statistics Question

  • Josh wants to do Bayesian statistics content
  • Concepts straightforward, implementation gets messy quickly
  • Gets stuck in technical nuances during research
  • Acknowledges many people use Bayesian methods successfully
  • Promises eventual coverage, likely 2021 project
  • Describes it as "mountain to climb and conquer"

Machine Learning Interview Tips

  • Josh lacks direct experience with ML technical interviews
  • Proposes community contribution: post actual interview questions
  • Suggests livestream format answering interview questions
  • Acknowledges his answers may not match what interviewers want
  • References previous livestream example of interview that went poorly due to disagreeing with question

Random vs Fixed Effects

  • High priority on to-do list
  • Described as "itch for years"

Future Vision for StatQuest

  • Long-term dream: expand like Khan Academy
  • Currently solo operation limiting output
  • Fantasy: team of "StatQuesters" covering more topics
  • 10-year vision for comprehensive coverage

Teaching Philosophy and Tips

  • Statistics/ML actually hard for Josh personally
  • Teaching method: explain to himself step-by-step
  • Can't skip big steps in explanation
  • Previous experience: monthly seminars for biologists
  • Biologists often not math-oriented, sat far back looking stunned
  • Used facial expressions to gauge understanding
  • Made mental notes when concepts weren't clear

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome