AI summary by TwinMind
AI Slop Is Spreading In Science, Too
Sabine Hossenfelder
7m 50s
AI Impact on Scientific Research Quality
- Years of concern about AI impact on science amid existing problems
- Research areas already struggling with nonsense research
- Peer review system failing
- Situation deteriorating
- Examples of suspected AI-generated content:
- Trump administration tariff formula
- UK tech secretary using ChatGPT for policy advice
- Terrence Howard's "solution" to three-body problem (actor believes 1×1=2)
- Half dozen "theories of everything" admittedly written with AI help
Mass Paper Generation Demonstration
- January 2025: Economists demonstrated AI's paper production capability
- Used GPT and Claude to analyze stock data
- Generated 288 complete finance papers
- Not gibberish - used reasonable protocol analyzing real data
- Potential for replication in other fields:
- Cosmology, particle physics possible targets
- Could analyze existing datasets for dark matter models
- Thousands of papers generatable instantly
- Economic incentive driving adoption
Current AI Detection Challenges
- Early AI papers trackable by phrases like "as of my last knowledge update"
- January research: AI-generated papers most common in fish research
- Advanced generative AI eliminates obvious text markers
- ProofFake company findings (January):
- AI-generated scientific images now extremely sophisticated
- Most researchers cannot distinguish AI images from real ones
- Major problem for biology and materials science fields
Case Study: "Vegetative Electron Microscopy"
- Nonsensical term appearing in 20+ scientific papers
- Origin: 1959 two-column article misreading
- "Vegetative cell wall" in one column
- "Electron microscopy" in other column
- AI propagation aided by Farsi language similarity (scanning ≈ vegetative)
- Now published in major publisher journals
- Represents detectable cases - vast majority likely undetected
Scientific Misconduct Detection Problems
- PubPeer platform for discussing paper problems
- System vulnerable to abuse (fake reviews equivalent)
- Sylvian Bernes case study:
- Email requesting paper retractions: "I don't like this group of researchers"
- Threat: "I'll be happy if first author loses his job"
- Follow-up threat to post papers on PubPeer for retractions
- Weaponization similar to fake restaurant reviews
Citation Manipulation and Fake Collaborations
- Previous case: Cat with academic citations from fake papers
- Uploaded to ResearchGate
- Indexed by Google Scholar
- "Possibly tenured by now"
- March preprint: Cluster of authors boosting citations via same method
- ResearchGate recommended Geoffrey Hinton check paper he "wrote" with Yann LeCun
- News to both supposed authors
- Fake collaboration detection
Future Implications
- Science approaching point of drowning in "AI slop"
- Potential positive outcome: Force addressing publish-or-perish culture
- Current trajectory: Science focused on generating "statistically plausible sentences" rather than finding truth
- AI represents beginning of new civilization phase
