AI summary by TwinMind
117. Nate Silver Says We're Bad at Making Predictions | People I (Mostly) Admire
Freakonomics Radio Network
42m 46s
Election Forecasting Success and Challenges
- Silver's track record: 2008 (49/50 states correct), 2012 (50/50 states), 2020 (48/50 states)
- 2016 election: gave Trump 29% chance vs conventional wisdom's 15-18%
- Prediction markets had Trump at 15-18% chance
- Silver's higher probability proved more accurate
- Key insight: states are correlated, not independent
- Upper Midwest states (Wisconsin, Michigan, Pennsylvania) have similar white working-class voter bases
- When Trump outperformed in one, he outperformed in others
- Other models treated states as 50 independent contests
- Huffington Post model: 99% Clinton chance
- Princeton model: 99% Clinton chance
- Silver's model accounted for third-party votes and undecided voters
Data Science Philosophy and Approach
- Self-taught data scientist with economics background (University of Chicago)
- Strengths: dealing with imperfect information, uncertainty, refining best guesses
- Thinks in provisional and probabilistic terms
- Professional poker player background (mid-20s, couple years)
- Prefers 80% quality in 10% or 2% of academic time
- Values getting basic ideas quickly vs exhaustive academic rigor
Academic vs Journalistic Analysis
- Academic publishing problems:
- 90% of effort addressing 74 different possible criticisms
- Adversarial referee system demanding 100% certainty
- Leads to answering tiny problems vs big questions
- Replication crisis despite precision requirements
- Silver's alternative approach:
- Describe data correlations honestly
- Generate multiple theories consistent with data
- Test theories against additional data
- More agnostic, scientific methodology
COVID-19 Analysis Case Study
- Key finding: no political correlation in death rates before vaccines (early 2021)
- Post-vaccine: strong correlation (red states ~35% higher death rates)
- Very red states like West Virginia: even larger gaps
- Yale study confirmation: individual-level voter registration data in Ohio/Florida showed same pattern
- Implications: vaccines clearly effective, other interventions less clear
Career Evolution and 538 Origins
- Started at KPMG consulting (boring)
- Side projects: internet poker, baseball statistics (sabermetrics)
- 2006: Congress banned internet poker, destroyed livelihood
- Became interested in 2006 Congressional elections (vindictive motivation)
- 2008: anonymous writing on Daily Kos about Obama vs Clinton vs McCain
- Founded 538 as independent site
- Viral success during Twitter's early era
- Always built models personally, delegated other tasks
Media Partnerships and Business Model
- New York Times partnership (3 years) - not very lucrative
- ESPN acquisition (2013), transferred to ABC News (2018)
- Disney cost-cutting eliminated 538 team (2022)
- ESPN initially treated as "loss leader" with no business model
- No dedicated ad sales or subscription strategy
- Moved to Substack (Silver Bulletin) for direct subscriber revenue
Prediction Challenges and Public Perception
- Public demands certainty, not probabilistic forecasts
- Parties want voters to believe their candidate will win
- Confirmation bias in political analysis
- Twitter/X enables quick criticism vs thoughtful analysis
- Considering not covering 2024 election due to public pressure
New Book and Future Projects
- Upcoming book on gambling and risk
- Covers: poker, Las Vegas history, sports betting, venture capital, crypto, economic progress
- Emphasis on logical thinking and statistics education
- Advocates for math appreciation courses in schools
- Believes in trying things and quitting if not suitable
