AI summary by TwinMind
Language or Vision - What's Harder? (Ilya Sutskever) | AI Podcast Clips
Lex Fridman
9m 59s
Machine Learning Unity and Commonalities
- ML field has "huge amount of unity" - overlap of ideas and principles
- Only 1-3 very simple principles apply almost identically across modalities/problems
- Optimization improvements in vision automatically improve NLP and RL applications
- Computer vision and NLP are "very similar to each other today"
Current Architectural Differences
- Different architectures currently used:
- Transformers in NLP
- Convolutional neural networks in vision
- Historical fragmentation being reduced:
- Past: Every tiny NLP problem had own architecture
- Present: Single transformer handles all different NLP tasks
- Even earlier: More fragmentation, every AI problem had specialized sub-field
Future Unification Predictions
- Vision and natural language expected to become unified
- Eventual broader unification between RL and supervised learning predicted
- Vision of "one big black box" - shovel everything in, it figures out what to do
- Uncertainty about timeline due to computational efficiency concerns
Reinforcement Learning Distinctions
- RL requires slightly different techniques:
- Need for action-taking
- Exploration requirements
- Much higher variance
- RL operates in fundamentally non-stationary world:
- Actions change what you see
- Experience world differently as behavior changes
- Contrasts with traditional static problems with fixed distributions
Technical Commonalities Across Domains
- Same core techniques used:
- Gradient-based optimization
- Neural networks
- Adam optimizer
- RL has additional variance reduction tools for gradients
- Differences are "small" but "not completely insignificant"
Problem Difficulty Philosophy
- Questioning what "hard" means for AI problems
- Traditional benchmark approach: effort required to reach human-level performance
- "Once you solve a problem it stops being hard" - relative to current tools
- Current assessment: Both human-level language understanding and visual perception are hard
- Cannot be solved completely "in next three months"
Language vs Vision Complexity Debate
- Chomsky perspective: Language fundamental to everything, underlies all cognition
- Boundary question: Where does vision end and language begin?
- Example: Reading text from images - vision or language problem?
- Two possibilities for deep understanding:
- Impossible to achieve without same kind of system for both
- Getting one domain gives you the other "for free"
- Definitions matter significantly for determining difficulty
Human-Level Performance and Continuous Engagement
- Systems impress briefly, then expectations adjust
- Humans continue to provide surprises, humor, wit, insights
- Subjective tests like humor as measure of intelligence
- AI systems as of January 2020 not achieving sustained impressive performance
