AI summary by TwinMind
This new “AI Scientist” can “fully automate discovery”, researchers claim
Sabine Hossenfelder
7m 38s
Current AI Job Displacement in Research
- AI has already replaced jobs requiring fast text generation
- News websites and bot content creation
- CNET began using AI for news end of 2022, articles riddled with mistakes
- Cosmos magazine used AI for articles, received complaints from journalists at risk
- Jobs most replaced so far: fast text generation roles
- Current AI benefits for news websites questionable
Nobel Prizes Signal AI's Scientific Impact
- 2022 Nobel Prize in Physics: went to AI development
- Nobel Prize in Chemistry: partly awarded to DeepMind for AI-driven protein folding advancements
- Demonstrates AI's growing role in scientific breakthroughs
Future AI Integration in Science
- OpenAI, Meta, others likely integrating math software into large language models
- Additional tools being integrated: graphic software, other scientific tools
- Most scientific fields only need standard math software
- Calculus and statistics sufficient for most disciplines
- Exceptions exist (specialized laboratory work)
First AI Scientist Study - End-to-End Research
- Computer scientists presented first AI scientist system
- Complete workflow: hypothesis generation → publication
- Cost: less than $15 per paper
- Claims to "democratize research" but may "flood research with crap"
- AI scientist performs sequence of tasks:
- Brainstorming
- Literature research
- Research plan layout
- Execution
- Paper writing
- Research area: AI science (AI studying itself)
- AI reviewer evaluated AI-generated papers about AI research
- Results: Anthropic Sonet 3.5 consistently produces best papers
- Some papers exceeded acceptance threshold for standard machine learning conferences
- AI-generated papers not published, but code open source on GitHub
Second AI Scientist Study - Virtual Discovery World
- Different approach: created computer game environment called "Discovery World"
- Virtual world governed by discoverable laws:
- Chemistry
- Physics
- Epidemiology
- Rocket science
- Quest-based discovery system (e.g., people fall ill, figure out why)
- Comparison: GPT-4 vs humans discovering laws
- Results: humans succeeded more often, especially in completing research
- Humans maintain edge "for now"
Implications for Scientific Fields
- Once AI reliably performs math calculations → flood of AI-written research articles
- Automation possibilities in physics research:
- Finding new solutions to Einstein's field equations (trial and error process)
- Inventing new particles to explain high energy physics anomalies
- Thousands of people currently earn living publishing such papers
- Question: if AI writes better papers than particle physicists, what happens?
Broader Perspective on Science
- Science as more than profession: way to understand world and solve problems
- Current human advantages in research completion and discovery
- Potential democratization vs quality concerns in research output
