AI summary by TwinMind
Lecture 03: Random Variables, Distributions, and Joint Distributions
MIT OpenCourseWare
1h 12m 9s
Course Schedule Overview
- Today: Random variables and distributions, possibly joint distributions
- Tuesday: Esther lecturing - examples, data sources, web scraping techniques, histograms, kernel density plots for estimating distributions
- Wednesday: Independence of random variables, conditional distributions
- Friday recitation: R tutorial continuation, possible probability hints for problem set
Random Variables Definition and Concepts
- Random variable = real valued function with domain as sample space
- Maps from sample space to real line
- Represents numerical characteristic of sample space we care about
- Examples: sum of dice faces, number of three-point field goals Steph Curry makes, number of vegetarian pizza toppings
- Probability on sample space induces distribution of random variable
Types of Random Variables
Discrete Random Variables
- Takes finite or countably infinite number of values
- All previous probability examples were discrete
- Don't have to be integers but often are
Continuous Random Variables
- Takes any value in some interval (bounded/unbounded) of real line
- Class will mostly deal with continuous random variables
- Reasons:
- Many discrete random variables adequately approximated by continuous
- Math often simpler for complicated functions of random variables
Discrete Random Variables - Pizza Example (Hypergeometric)
- X = number of vegetarian toppings when drawing 4 toppings randomly without replacement
- 6 nonvegetarian, 5 vegetarian toppings available
- Used hypergeometric formula from last class
- For n=3: P(X=0)=6/99, P(X=1)=36/99, etc.
- Graphed as probability function with vertical lines (point masses)
- Zero factorial defined as 1
General Hypergeometric Distribution
- X ~ Hypergeometric(N, K, n)
- N = total items, K = success items, n = items chosen
- Interpretation: number of successes in n trials, sampling without replacement
- Initial probability of success = K/N
- Valid for x ∈ {max(0, n+K-N), ..., min(K,n)}
Discrete Random Variables - Steph Curry Example (Binomial)
- X = number of three-point shots made in next 6 attempts
- Assumptions: shots independent, same probability (0.44) each shot
- Formula: P(X=x) = (6 choose x) × 0.44^x × 0.56^(6-x)
- Distribution appears nearly symmetric but isn't (p ≠ 0.5)
- Calculated probabilities for x = 0,1,2,...,6
General Binomial Distribution
- X ~ Binomial(n,p)
- n = number of trials, p = probability of success
- Formula: P(X=x) = (n choose x) × p^x × (1-p)^(n-x)
- When p = 0.5, distribution is symmetric
- As n increases with p = 0.5, approaches normal distribution
Probability Functions (Discrete)
- Definition: f_X(x) = P(X = x)
- Properties:
- 0 ≤ f_X(x) ≤ 1 for all x
- Σ f_X(x) = 1 over all possible x values
- P(X ∈ A) = Σ f_X(x) for x ∈ A
Continuous Random Variables and PDFs
- Rarely start with verbal description like discrete case
- Given function (density/PDF) describing probabilities for regions
- Examples: normal density, triangular density, log beta densities
Probability Density Function Definition
- X continuous if exists non-negative function f_X such that P(X ∈ A) = ∫_A f_X(x)dx
- Different terminology from probability function to emphasize differences
- Same notation f_X(x) used for both discrete and continuous
PDF Properties vs Probability Function
- Similarities:
- Always non-negative
- Integrates to 1 (vs sums to 1 for discrete)
- P(X ∈ A) = ∫_A f_X(x)dx (vs sum for discrete)
- Differences:
- PDF can exceed 1 (probability function cannot)
- P(X = x) = 0 for any specific x (continuous case)
- PDF value at point x ≠ probability interpretation
Uniform Distribution Example
- X uniform on interval [a,b]
- PDF: f_X(x) = 1/(b-a) for x ∈ [a,b], 0 otherwise
- Probability proportional to length of sub-interval
- P(X ∈ [c,d]) = (d-c)/(b-a) for [c,d] ⊆ [a,b]
- Instructor's "favorite random variable"
Cumulative Distribution Function (CDF)
- Same definition for discrete and continuous: F_X(x) = P(X ≤ x)
- Properties:
- 0 ≤ F_X(x) ≤ 1
- Non-decreasing in x
- lim_{x→-∞} F_X(x) = 0, lim_{x→∞} F_X(x) = 1
- Continuous: smooth curves, no jumps
- Discrete: step functions with jumps at point masses
- Mixed: combination of smooth parts and jumps
Relationship between PDF/PF and CDF
- Discrete: can recover PF from CDF by looking at jump sizes
- Continuous:
- F_X(x) = ∫_{-∞}^x f_X(t)dt
- f_X(x) = F'_X(x) (where derivative exists)
Joint Distributions
Motivation
- Much of data analysis involves joint distributions of random variables
- Examples:
- Rainfall and crop growth
- Regular vs express checkout line lengths
- Dollar-euro exchange rate and exporting firm stock price
Joint PDF Definition (Continuous)
- f_{X,Y}(x,y) is surface such that P((X,Y) ∈ A) = ∫∫A f{X,Y}(x,y)dxdy
- Properties similar to univariate case
- Discrete analog exists: f_{X,Y}(x,y) = P(X=x, Y=y)
Joint Distribution Example - Headache Medicine
- X = effective period of naproxen, Y = effective period of acetaminophen
- Joint PDF: f_{X,Y}(x,y) = λ²e^{-λ(x+y)} for x,y ≥ 0
- Note: this equals f_X(x) × f_Y(y), indicating independence
Probability Calculations
- P(headache returns within 3 hours) when taking both simultaneously:
- P(X ≤ 3, Y ≤ 3) = ∫₀³∫₀³ λ²e^{-λ(x+y)}dydx
- Result: (1-e^{-3λ})²
- P(headache returns within 3 hours) when taking sequentially:
- P(X + Y ≤ 3) = ∫₀³∫₀^{3-x} λ²e^{-λ(x+y)}dydx
- Integration over triangular region where x + y ≤ 3
Function of Random Variables
- Defined new random variable Z = X + Y (total effective life)
- CDF: F_Z(z) = P(Z ≤ z) = P(X + Y ≤ z)
- PDF: f_Z(z) = λ²ze^{-λz} (by differentiation)
- Important concept: statistics are functions of random variables
- Understanding function distributions crucial for statistics
Terminology and Notation Notes
- No standard terminology across textbooks/sources
- Instructor will try to be internally consistent
- "Probability function" for discrete, "probability density function" for continuous
- Sometimes will use general term "distribution"
- Mixed random variables exist (combination of discrete and continuous parts)
