AI summary by TwinMind
How to tell science from pseudoscience
Sabine Hossenfelder
8m 29s
Core Definition of Science
- Science task: explain observations
- Two requirements: A) observations, B) understanding what "explain" means scientifically
- Scientific explanation = model (simplified description of real world)
- Model allows statements about observations that agree with measurements
- Key criterion: model must be simpler than collection of all available data
- Simplification achieved by capturing patterns in data
- Patterns = simplification
Historical Example: Astronomy
- Best example of constructing scientific models
- Focus: finding patterns in celestial object motions
- Visual representation: scientific model = curve connecting data points
- Oversimplified but instructive visualization
Overfitting and Model Limitations
- Model becomes unscientific when it has too much freedom to fit any data
- Overfitting: model doesn't explain anything, better to just collect data
- Rule: model with more free parameters than data points = not scientific
- More complicated models generally allow better fit to data
- Challenge: determine when adding parameters doesn't justify slightly better fit
- Statistical methods exist to evaluate model quality quantitatively
Quantitative vs Non-Quantitative Disciplines
- Not all science disciplines are equally quantitative
- General principle applies: model requiring many assumptions to explain few observations = unscientific
- Especially when simpler explanation exists
- Model explaining no observations = also not scientific
Conspiracy Theories as Pseudoscience Examples
- Flat Earth theory
- Requires assumption NASA covered up evidence since Ptolemy's time
- Must assume consistent data/image faking
- Must assume all airline pilots brainwashed
- 5G mind control theory
- Requires assumption of mind control capability
- Paradox: if minds controlled, how did theorist figure it out?
- COVID vaccine industry engineering theory
- Contrived assumptions required
- Common trait: require many assumptions to agree with reality
- Contrived assumptions = equivalent of overfitting
Scientific Explanations vs Conspiracy Theories
- Scientific explanations are simple ones
- Explain many observations with few assumptions
- Examples of simple scientific explanations:
- Earth is round
- 5G is wireless network
- Coronaviruses jumped to humans before (most likely COVID origin)
Darwinian Evolution vs Creationism
- Darwinian evolution: good scientific theory
- Connects dots by showing organism evolution pathways
- Should be possible to quantify fit to data (though not done)
- Creationism: posits Earth created with everything in place
- Creationism puts in as much information as extracted
- Therefore explains nothing
- Not wrong, but unscientific
Predictions and Scientific Models
- Pseudoscientists often brag about making predictions
- Making predictions ≠ scientific
- Not making predictions ≠ unscientific
- Example: predicting relative illness in coming week
- Will be correct for some people
- Not impressive because no model demonstrated
- Must show prediction better than random guess
- Need to explain model and assumptions
- Simplest explanation for correct guess: coincidence
Models Without Predictions Can Be Scientific
- Scientific if explains lots of existing data
- Pandemic models example:
- Cannot make predictions due to unpredictable policy decisions
- Make projections/scenarios instead
- "If-then" cases: if events not canceled vs canceled
- Scientific because accurately describe recorded epidemic spread
- Simple explanations fitting lots of data
- Climate models similar case:
- Simple explanation: climate change due to increasing CO2 from humans
- Fits data with least assumptions
Evaluation Criteria for Scientific Models
- Ask: how much data can model correctly reproduce?
- Ask: how many assumptions required?
- These questions determine scientific validity
Borderline Cases: Underdeveloped Ideas
- Difficult to distinguish science from pseudoscience for underdeveloped ideas
- Warning signs: constantly told "it's promising" but no demonstrated model fit
- Researchers claim not done with research
- String theory example:
- Could be good science if worked as advertised
- String theorists never reach point of actual usefulness
- Different question: how much time/money to invest in research direction?
- Decision falls to research funders
