AI summary by TwinMind
AI Gets Rapidly Smarter, And Makes Some Of Us Dumber
Sabine Hossenfelder
8m 9s
Grok 3 Release by xAI (February 18th)
- Elon Musk's xAI released Grok 3, described as "scary smart" by Musk
- Performance benchmarks:
- Outperformed most other AIs in mathematics, scientific reasoning, coding
- Set new record on El Marina crowdsourced evaluation platform
- Comparable to GPT-4 and DeepSeek in reasoning tasks
- Technical specifications:
- First general purpose AI model to exceed 10^25 FLOP in training compute
- Breaches EU AI Act threshold, requires additional safety tests
- EU law takes effect August 2024
- Capabilities and limitations:
- Cannot use both "think" and "deep search" modes simultaneously
- Can search X/Twitter (advantage over OpenAI and DeepSeek)
- Failed to find Musk's own tweet about quantum gravity query
- Example query: Generated 3 new ideas for gravity/force unification
- Suggested gravity as macroscopic shadow of quantum entanglement across higher dimensional structures
- Ongoing issues:
- Fails to pay attention to exact wording (confused 20 feathers vs 20 pounds of feathers)
- Easy to jailbreak, provides bomb instructions
Google AI Co-Scientist Launch
- AI super agent designed to help scientists discover research hypotheses and grant proposal topics
- Architecture: 1 supervising agent + 6 sub-agents with specialized roles
- Idea generation agent
- Criticism agent
- Modification agent
- Rating agent
- Not yet publicly available
- Case study results:
- Tested with bacterial antimicrobial resistance research group
- AI reproduced same hypothesis as unpublished research project
- Completed analysis in minutes vs months for human researchers
- Lead author response: "We were shocked... you have access to my computer?"
OpenAI GPT Development Timeline
- Sam Altman announced February 12th:
- GPT 4.5 release planned soon
- GPT-5 to follow within weeks/months
Persistent AI Model Issues
- All large language models continue basic problems:
- Attention to exact query wording failures
- Hallucination issues across models (GPT-4, Gemini Advanced)
- Financial literature reference invention with double-digit probability
- OpenAI's Deep Research tool invents data tables
- Jailbreaking vulnerabilities remain
Corporate AI Adoption Reality
- HSBC AI integration (Edward Akner): "Much of AI talk is success theater"
- Airbnb CEO Brian Chesky: "Not flowing to fundamental step change in productivity yet"
- Results described as not "Earth shattering"
Microsoft AI Impact Study
- Tracked ~300 people across different AI tasks (content generation, advice seeking)
- Two behavioral camps identified:
- High AI confidence users: Less critical thinking
- High self-confidence users: More critical thinking
- Historical context: Radio, TV, video games, internet, Google all previously accused of making people stupid
Regulatory Landscape
- EU AI Act implementation challenges
- "America innovates faster than Europe can regulate"
- Safety testing requirements for high-compute models
