AI summary by TwinMind

Pearson's Correlation, Clearly Explained!!!

StatQuest with Josh Starmer

19m 13s

Introduction to Relationships in Data

  • Host: Josh Starburns presenting Stack Quest part 2 on covariance and correlation
  • Focus: relationships between data on x-axis and y-axis
  • Example: mRNA transcripts from gene X vs gene Y in 5 different cells
  • Alternative analogy: green apples vs red apples in 5 grocery stores
  • Each measurement pair represented by blue dot on graph

Basic Correlation Concepts

  • Low values for gene X paired with low values for gene Y
  • High values for gene X paired with high values for gene Y
  • Straight line with positive slope represents trend
  • Predictive capability: gene X = 20 → gene Y ≈ 27
  • Closer data to trendline = smaller prediction range = stronger relationship
  • Further data from trendline = larger prediction range = weaker relationship

Causation vs Correlation Distinction

  • Correlation shows observed trends for predictions/inferences
  • Does NOT imply causation
  • Low gene X values don't cause low gene Y values
  • Possibility of third factor causing observed trend

Correlation Strength Quantification

  • Weak relationship = small correlation value
  • Moderate relationship = moderate correlation value
  • Strong relationship = large correlation value
  • Maximum correlation value = 1
  • Correlation = 1 when straight line with positive slope goes through center of every data point

Scale Independence

  • Correlation doesn't depend on data scale
  • Numbers on axes don't affect correlation
  • Correlation = 1 regardless of slope magnitude (large or small)
  • Correlation = 1 regardless of amount of data

Sample Size and Confidence

  • 2 data points: always possible to draw straight line → correlation = 1
    • Low confidence due to limited data
    • Can always connect any 2 random dots with straight line
  • 3 data points: very small probability of random points forming straight line
    • Higher confidence in correlation
  • More data points = higher confidence in predictions
    • Probability of random straight line decreases with each additional point

P-value Interpretation

  • P-value = probability that randomly drawn dots result in similarly strong/stronger relationship
  • Smaller p-value = more confidence in predictions
  • Example: p-value = 2.2 × 10^-16 (extremely small)
  • Indicates probability of random data creating similar relationship is extremely small

Negative Correlation

  • Correlation = -1 when straight line with negative slope goes through all data points
  • Strong relationship with narrow prediction range
  • Confidence depends on data amount (quantified by p-value)
  • Scale independence applies (correlation = -1 for large and small negative slopes)

Correlation Range and Interpretation

  • Positive slope: strongest relationship = correlation = 1
  • Negative slope: strongest relationship = correlation = -1
  • Imperfect fit: correlation values closer to zero
  • Worse fit = closer to zero correlation
  • No linear relationship = correlation = 0
  • Correlation = 0 means x-axis value provides no information about y-axis value

Non-zero Correlation Usage

  • Any non-zero correlation allows inferences
  • Guesses become more refined as correlation approaches ±1
  • Confidence depends on data amount and p-value
  • Examples with correlation = 0.3:
    • Little data: p-value = 0.8 (low confidence)
    • Moderate data: p-value = 0.08 (moderate confidence)
    • Lots of data: p-value = 0.008 (high confidence)
  • More data increases confidence, not necessarily correlation value
  • Small correlation + lots of data = confident bad guesses

Correlation Calculation Formula

  • Correlation = Covariance(gene X, gene Y) / (√Variance(gene X) × √Variance(gene Y))
  • Requires knowledge of variance and covariance calculations
  • Numerator (covariance): any value between +∞ and -∞
  • Denominator: squeezes covariance to range [-1, 1]
  • Scale independence ensured by denominator
  • Perfect linear fit: covariance equals product of square root variance terms → correlation = ±1

Worked Example

  • Covariance value: 116
  • Gene X variance: 101.8
  • Gene Y variance: 160.3
  • Calculated correlation: 0.9
  • P-value: 0.03 (3% chance random data could produce similar/stronger relationship)

Correlation Interpretation Limitations

  • Correlation values easier to interpret than covariance
  • Still not super intuitive
  • Correlation = 0.9 not obviously twice as good as correlation = 0.64 for predictions
  • R-squared solves interpretation problem
  • R-squared can quantify more complex relationships beyond straight lines

Summary Points

  • Correlation quantifies relationship strength
  • Range: -1 (strongest negative linear) to +1 (strongest positive linear)
  • Imperfect linear fit → values closer to zero
  • No linear relationship → correlation = 0
  • Confidence depends on data amount and p-value
  • More data → smaller p-value → higher confidence

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome