AI summary by TwinMind

StatQuickie: Thresholds for Significance

StatQuest with Josh Starmer

6m 40s

Origin and Nature of 0.05 Threshold

  • 0.05 threshold relatively arbitrary, not biologically/naturally optimal
  • Used widely in science publications for long time
  • Generally effective: 5% wrong, 95% right - cost-effective for biological sciences
  • Recent issues with p-hacking in high-profile articles

Publishing Data Threshold Guidelines

  • 0.05 almost always appropriate for publication
  • Editors and reviewers expect this standard
  • Going below 0.05 is beneficial when possible
  • Exceptions exist for specific circumstances

Exploratory Data Analysis Threshold

  • Downloaded datasets from internet for hypothesis testing
  • P-values slightly above 0.05 acceptable when:
    • Not publishing immediately
    • Using limited/free datasets
    • Testing basic hypotheses for later validation
    • Generating new hypotheses for future testing
  • Don't need massively tiny p-values for exploration

Effect Size Considerations

  • Small p-value insufficient alone
  • R-squared example: 10% variation explained = poor despite tiny p-value
  • Need good effect size AND small p-value
  • Good correlation/explanation of data essential
  • Publications with p-value 0.000000 but horrible R-squared are meaningless

Extraordinary Claims Standard

  • Extraordinary claims require extraordinary data
  • Example: extraterrestrials flying around NYC in UFOs
  • P-value of 0.047 (< 0.05) completely insufficient
  • Need "crazy small" p-value that's "unbelievable"
  • Threshold must be so extreme it forces acceptance of extraordinary conclusion

Summary Guidelines

  • Publishing: 0.05 threshold fine with good R-squared/correlation
  • Exploration: flexible threshold for validation-intended research
  • Extraordinary claims: extremely small p-values required

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome