AI summary by TwinMind
Lecture 01: Introduction to 14.310x Data Analysis for Social Scientists
MIT OpenCourseWare
1h 0m 42s
Course Introduction and Instructor Background
- Course: 14.310 Data Analysis for Social Scientists
- Instructors: Esther Duflo and Sarah Ellison
- Esther: Development economist, J-PAL director, works with governments on data-driven policy
- Sarah: MIT since 1988, industrial organization research, three daughters who like math
- No prerequisites listed, assumes basic algebra/calculus familiarity
- No required textbook due to experimental nature of course
Data Abundance and Beauty
- Data sources everywhere: World Bank, Census, J-PAL, academic journals, web scraping, Amazon/Facebook/LinkedIn
- Network data visualization example: Somali community Facebook networks in Seattle
- Researcher: Kim acquaintance
- Tools: Netv for harvesting, Gephi for visualization
- Node size proportional to importance in network
- Importance measured by eigenvector centrality (recursive notion similar to early Google PageRank)
- Different connection types: hometown, classes, boarding school, work relationships
Data Insights - Pollution and Mortality Study
- Research by Michael Greenstone and colleagues on pollution effects in China
- Natural experiment: Huai River boundary policy
- North of river: subsidized coal heating
- South of river: no subsidies
- Heat map visualization showing PM10 concentration differences
- Jump of 4.1 microgram per cubic meter at boundary (statistically significant)
- Life expectancy analysis showed corresponding mortality effects
- Demonstrates regression discontinuity design approach
Data Power - Gujarat Pollution Auditing Reform
- Collaboration with Gujarat Pollution Control Board
- Problem: Third-party auditing system with conflict of interest
- Firms hired own auditors directly
- Auditors incentivized to provide clean reports to maintain relationships
- Similar to Enron scandal dynamics
- Solution tested: Randomized controlled trial
- Treatment: Central pool payment system, random auditor assignment, back-checking
- Control: Traditional system
- Results: Treatment group audit data matched back-check data closely
- Timeline: 2-year project, immediate effects, additional years for policy implementation
- Policy change: Gujarat adopted new system based on evidence
Data Deception and Correlation vs Causation
- Autism correlation examples:
- Fabricated MMR vaccine data (Lancet publication later retracted)
- Real but misleading glyphosate-autism correlation
- Organic food sales-autism correlation (tongue-in-cheek example)
- Both series trending upward creates spurious correlation
- Education-GDP correlation analysis:
- Strong cross-sectional correlation between secondary enrollment and log GDP per capita
- Much weaker correlation between education growth and GDP growth
- Potential explanations: reverse causality, omitted variables, convergence effects
Big Data Challenges and Spurious Correlations
- Example of personal data trail: Capoeira classes, statistics websites, dollhouse purchases
- Machine learning applications don't require causality (prediction vs explanation)
- Genetic data fishing expeditions for behavioral predictions
- Spurious correlations website examples:
- Arcade revenue vs computer science doctorates
- Miss America age vs murders by hot objects
- Pool drownings vs Nicholas Cage films
- Pattern emergence inevitable with enough data searching
Course Structure and Learning Objectives
- Probability modeling: Understanding data generation processes
- Statistics: Summarizing and describing data
- Pattern recognition: Variable relationships, exploratory data analysis
- Econometrics and machine learning techniques
- Causality framework and experimental design
- Natural experiments and quasi-experimental methods
- Practical R programming throughout
- Web data collection methods
- Data presentation: beautiful graphs, insightful tables, clear text
Course Logistics
- Experimental course format
- No single comprehensive textbook
- Listed texts cover portions of material:
- First half: probability/statistics texts
- Second half: econometrics texts
- All texts excellent resources but none required for purchase
