AI summary by TwinMind

Lecture 01: Introduction to 14.310x Data Analysis for Social Scientists

MIT OpenCourseWare

1h 0m 42s

Course Introduction and Instructor Background

  • Course: 14.310 Data Analysis for Social Scientists
  • Instructors: Esther Duflo and Sarah Ellison
  • Esther: Development economist, J-PAL director, works with governments on data-driven policy
  • Sarah: MIT since 1988, industrial organization research, three daughters who like math
  • No prerequisites listed, assumes basic algebra/calculus familiarity
  • No required textbook due to experimental nature of course

Data Abundance and Beauty

  • Data sources everywhere: World Bank, Census, J-PAL, academic journals, web scraping, Amazon/Facebook/LinkedIn
  • Network data visualization example: Somali community Facebook networks in Seattle
    • Researcher: Kim acquaintance
    • Tools: Netv for harvesting, Gephi for visualization
    • Node size proportional to importance in network
    • Importance measured by eigenvector centrality (recursive notion similar to early Google PageRank)
    • Different connection types: hometown, classes, boarding school, work relationships

Data Insights - Pollution and Mortality Study

  • Research by Michael Greenstone and colleagues on pollution effects in China
  • Natural experiment: Huai River boundary policy
    • North of river: subsidized coal heating
    • South of river: no subsidies
  • Heat map visualization showing PM10 concentration differences
  • Jump of 4.1 microgram per cubic meter at boundary (statistically significant)
  • Life expectancy analysis showed corresponding mortality effects
  • Demonstrates regression discontinuity design approach

Data Power - Gujarat Pollution Auditing Reform

  • Collaboration with Gujarat Pollution Control Board
  • Problem: Third-party auditing system with conflict of interest
    • Firms hired own auditors directly
    • Auditors incentivized to provide clean reports to maintain relationships
    • Similar to Enron scandal dynamics
  • Solution tested: Randomized controlled trial
    • Treatment: Central pool payment system, random auditor assignment, back-checking
    • Control: Traditional system
  • Results: Treatment group audit data matched back-check data closely
  • Timeline: 2-year project, immediate effects, additional years for policy implementation
  • Policy change: Gujarat adopted new system based on evidence

Data Deception and Correlation vs Causation

  • Autism correlation examples:
    • Fabricated MMR vaccine data (Lancet publication later retracted)
    • Real but misleading glyphosate-autism correlation
    • Organic food sales-autism correlation (tongue-in-cheek example)
  • Both series trending upward creates spurious correlation
  • Education-GDP correlation analysis:
    • Strong cross-sectional correlation between secondary enrollment and log GDP per capita
    • Much weaker correlation between education growth and GDP growth
    • Potential explanations: reverse causality, omitted variables, convergence effects

Big Data Challenges and Spurious Correlations

  • Example of personal data trail: Capoeira classes, statistics websites, dollhouse purchases
  • Machine learning applications don't require causality (prediction vs explanation)
  • Genetic data fishing expeditions for behavioral predictions
  • Spurious correlations website examples:
    • Arcade revenue vs computer science doctorates
    • Miss America age vs murders by hot objects
    • Pool drownings vs Nicholas Cage films
  • Pattern emergence inevitable with enough data searching

Course Structure and Learning Objectives

  • Probability modeling: Understanding data generation processes
  • Statistics: Summarizing and describing data
  • Pattern recognition: Variable relationships, exploratory data analysis
  • Econometrics and machine learning techniques
  • Causality framework and experimental design
  • Natural experiments and quasi-experimental methods
  • Practical R programming throughout
  • Web data collection methods
  • Data presentation: beautiful graphs, insightful tables, clear text

Course Logistics

  • Experimental course format
  • No single comprehensive textbook
  • Listed texts cover portions of material:
    • First half: probability/statistics texts
    • Second half: econometrics texts
  • All texts excellent resources but none required for purchase

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome