AI summary by TwinMind
Lecture 11: Special Distributions, continued. The Sample Mean, Central Limit Theorem, and Estimation
MIT OpenCourseWare
1h 13m 43s
Normal Distribution Review and Properties
- Formula for normal distribution density with mean μ and variance σ²
- Bell-shaped, symmetric, thin-tailed characteristics
- As variance decreases: higher peak at mode, more area concentrated around mean
- As variance increases: flatter distribution
- Standard normal distribution properties:
- 68% of area between -1 and 1
- 95% of area between -2 and 2
- 2.1% below -2, 2.1% above 2
- 13.6% between -2 and -1, 13.6% between 1 and 2
- 34.1% between -1 and 0, 34.1% between 0 and 1
Using Normal Distribution Tables
- No closed form formula for PDF integral
- Use distributed tables to find probabilities
- Example: P(Z < -1.23) = 0.1109
- For P(Z > -1.68): use 1 - P(Z < -1.68)
- For positive numbers when only negative table available: use symmetry
- For intervals: P(-1.23 < Z < 1.45) = P(Z < 1.45) - P(Z < -1.23)
- Standardize non-standard normal variables first
R Commands for Normal Distribution
rnorm(): generates random numbers from normal distributiondnorm(): probability density function (height at specific value)pnorm(): area under curve (cumulative probability)lower.tail = TRUE(default): P(Z < x)lower.tail = FALSE: P(Z > x)
qnorm(): inverse of pnorm (quantile function)- Example with normal(mean=2, sd=0.5):
- P(X < X1) = 0.5
- P(X2 < X < X3) using difference of cumulative probabilities
- P(X > X4) using lower.tail = FALSE
Inverse Sampling Method
- Can use qnorm function with uniform random variables
- Practical R algorithms for normal sampling are faster than inverse method
- Principle still applies: uniform → qnorm → normal random variable
Exam Preparation - Practical Skills
- Creating histograms by hand from data tables
- Generating simulated datasets from specific distributions
- Web scraping concepts and applications
- General understanding of data acquisition methods
- Examples of datasets obtainable through web scraping
Sample Mean Introduction
- Definition: arithmetic average of n random variables from IID random sample
- Notation: X̄ₙ = (1/n)∑Xᵢ
- Used for both function of random variables and realizations
- Fundamental tool for estimation
Sample Mean Distribution Properties
- E[X̄] = μ (expectation equals population mean)
- Var(X̄) = σ²/n (variance decreases with sample size)
- Independence needed for variance calculation
- Sample mean has same mean as individual variables
- Sample mean more concentrated around μ than individual observations
Central Limit Theorem
- For IID random variables with finite mean μ and variance σ²
- Standardized sample mean: (X̄ - μ)/(σ/√n)
- As n → ∞: distribution approaches standard normal
- Works for any underlying distribution (remarkable result)
- Sample mean approximately normal for large n: X̄ ~ N(μ, σ²/n)
- Foundation for statistical inference
- Explains importance of normal distribution in statistics
Introduction to Statistics and Estimation
- Statistics = study of estimation and inference
- Bridge between probability theory and statistical practice
- Estimator = function of random variables in random sample
- Chosen to reveal information about underlying distribution
- Examples seen: sample mean, buffalo population problem
Parameters and Estimation Framework
- Parameter = constant indexing family of distributions
- Examples: μ, σ² (normal); λ (exponential); A, B (uniform); n, p (binomial)
- General notation: θ for parameter, θ̂ for estimator
- Goal: determine parameter values from observed data
- Dual notation: estimator (function) vs estimate (realized value)
Uniform Distribution Estimation Example
- X ~ Uniform(0, θ), θ unknown
- Two proposed estimators:
- θ̂₁ = X₍ₙ₎ (maximum order statistic)
- θ̂₂ = 2X̄ (twice sample mean)
- Both seem reasonable approaches
- Performance improves as n increases
- Additional option: θ̂₃ = 2 × sample median
Estimator Assessment Criteria
- Estimators are random variables with distributions
- Assessment based on distributional characteristics
- Unbiasedness: E[θ̂] = θ for all θ ∈ Θ
- Unbiased estimator has expectation equal to true parameter
- Sample mean is unbiased estimator of population mean
- Mathematical proof possible without knowing true parameter value
- Thought experiment: repeated sampling creates distribution of estimates
