AI summary by TwinMind

Lecture 03: Random Variables, Distributions, and Joint Distributions

MIT OpenCourseWare

1h 12m 9s

Course Schedule Overview

  • Today: Random variables and distributions, possibly joint distributions
  • Tuesday: Esther lecturing - examples, data sources, web scraping techniques, histograms, kernel density plots for estimating distributions
  • Wednesday: Independence of random variables, conditional distributions
  • Friday recitation: R tutorial continuation, possible probability hints for problem set

Random Variables Definition and Concepts

  • Random variable = real valued function with domain as sample space
  • Maps from sample space to real line
  • Represents numerical characteristic of sample space we care about
  • Examples: sum of dice faces, number of three-point field goals Steph Curry makes, number of vegetarian pizza toppings
  • Probability on sample space induces distribution of random variable

Types of Random Variables

Discrete Random Variables

  • Takes finite or countably infinite number of values
  • All previous probability examples were discrete
  • Don't have to be integers but often are

Continuous Random Variables

  • Takes any value in some interval (bounded/unbounded) of real line
  • Class will mostly deal with continuous random variables
  • Reasons:
    • Many discrete random variables adequately approximated by continuous
    • Math often simpler for complicated functions of random variables

Discrete Random Variables - Pizza Example (Hypergeometric)

  • X = number of vegetarian toppings when drawing 4 toppings randomly without replacement
  • 6 nonvegetarian, 5 vegetarian toppings available
  • Used hypergeometric formula from last class
  • For n=3: P(X=0)=6/99, P(X=1)=36/99, etc.
  • Graphed as probability function with vertical lines (point masses)
  • Zero factorial defined as 1

General Hypergeometric Distribution

  • X ~ Hypergeometric(N, K, n)
  • N = total items, K = success items, n = items chosen
  • Interpretation: number of successes in n trials, sampling without replacement
  • Initial probability of success = K/N
  • Valid for x ∈ {max(0, n+K-N), ..., min(K,n)}

Discrete Random Variables - Steph Curry Example (Binomial)

  • X = number of three-point shots made in next 6 attempts
  • Assumptions: shots independent, same probability (0.44) each shot
  • Formula: P(X=x) = (6 choose x) × 0.44^x × 0.56^(6-x)
  • Distribution appears nearly symmetric but isn't (p ≠ 0.5)
  • Calculated probabilities for x = 0,1,2,...,6

General Binomial Distribution

  • X ~ Binomial(n,p)
  • n = number of trials, p = probability of success
  • Formula: P(X=x) = (n choose x) × p^x × (1-p)^(n-x)
  • When p = 0.5, distribution is symmetric
  • As n increases with p = 0.5, approaches normal distribution

Probability Functions (Discrete)

  • Definition: f_X(x) = P(X = x)
  • Properties:
    • 0 ≤ f_X(x) ≤ 1 for all x
    • Σ f_X(x) = 1 over all possible x values
    • P(X ∈ A) = Σ f_X(x) for x ∈ A

Continuous Random Variables and PDFs

  • Rarely start with verbal description like discrete case
  • Given function (density/PDF) describing probabilities for regions
  • Examples: normal density, triangular density, log beta densities

Probability Density Function Definition

  • X continuous if exists non-negative function f_X such that P(X ∈ A) = ∫_A f_X(x)dx
  • Different terminology from probability function to emphasize differences
  • Same notation f_X(x) used for both discrete and continuous

PDF Properties vs Probability Function

  • Similarities:
    • Always non-negative
    • Integrates to 1 (vs sums to 1 for discrete)
    • P(X ∈ A) = ∫_A f_X(x)dx (vs sum for discrete)
  • Differences:
    • PDF can exceed 1 (probability function cannot)
    • P(X = x) = 0 for any specific x (continuous case)
    • PDF value at point x ≠ probability interpretation

Uniform Distribution Example

  • X uniform on interval [a,b]
  • PDF: f_X(x) = 1/(b-a) for x ∈ [a,b], 0 otherwise
  • Probability proportional to length of sub-interval
  • P(X ∈ [c,d]) = (d-c)/(b-a) for [c,d] ⊆ [a,b]
  • Instructor's "favorite random variable"

Cumulative Distribution Function (CDF)

  • Same definition for discrete and continuous: F_X(x) = P(X ≤ x)
  • Properties:
    • 0 ≤ F_X(x) ≤ 1
    • Non-decreasing in x
    • lim_{x→-∞} F_X(x) = 0, lim_{x→∞} F_X(x) = 1
  • Continuous: smooth curves, no jumps
  • Discrete: step functions with jumps at point masses
  • Mixed: combination of smooth parts and jumps

Relationship between PDF/PF and CDF

  • Discrete: can recover PF from CDF by looking at jump sizes
  • Continuous:
    • F_X(x) = ∫_{-∞}^x f_X(t)dt
    • f_X(x) = F'_X(x) (where derivative exists)

Joint Distributions

Motivation

  • Much of data analysis involves joint distributions of random variables
  • Examples:
    • Rainfall and crop growth
    • Regular vs express checkout line lengths
    • Dollar-euro exchange rate and exporting firm stock price

Joint PDF Definition (Continuous)

  • f_{X,Y}(x,y) is surface such that P((X,Y) ∈ A) = ∫∫A f{X,Y}(x,y)dxdy
  • Properties similar to univariate case
  • Discrete analog exists: f_{X,Y}(x,y) = P(X=x, Y=y)

Joint Distribution Example - Headache Medicine

  • X = effective period of naproxen, Y = effective period of acetaminophen
  • Joint PDF: f_{X,Y}(x,y) = λ²e^{-λ(x+y)} for x,y ≥ 0
  • Note: this equals f_X(x) × f_Y(y), indicating independence

Probability Calculations

  • P(headache returns within 3 hours) when taking both simultaneously:
    • P(X ≤ 3, Y ≤ 3) = ∫₀³∫₀³ λ²e^{-λ(x+y)}dydx
    • Result: (1-e^{-3λ})²
  • P(headache returns within 3 hours) when taking sequentially:
    • P(X + Y ≤ 3) = ∫₀³∫₀^{3-x} λ²e^{-λ(x+y)}dydx
    • Integration over triangular region where x + y ≤ 3

Function of Random Variables

  • Defined new random variable Z = X + Y (total effective life)
  • CDF: F_Z(z) = P(Z ≤ z) = P(X + Y ≤ z)
  • PDF: f_Z(z) = λ²ze^{-λz} (by differentiation)
  • Important concept: statistics are functions of random variables
  • Understanding function distributions crucial for statistics

Terminology and Notation Notes

  • No standard terminology across textbooks/sources
  • Instructor will try to be internally consistent
  • "Probability function" for discrete, "probability density function" for continuous
  • Sometimes will use general term "distribution"
  • Mixed random variables exist (combination of discrete and continuous parts)

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome