AI summary by TwinMind

Language or Vision - What's Harder? (Ilya Sutskever) | AI Podcast Clips

Lex Fridman

9m 59s

Machine Learning Unity and Commonalities

  • ML field has "huge amount of unity" - overlap of ideas and principles
  • Only 1-3 very simple principles apply almost identically across modalities/problems
  • Optimization improvements in vision automatically improve NLP and RL applications
  • Computer vision and NLP are "very similar to each other today"

Current Architectural Differences

  • Different architectures currently used:
    • Transformers in NLP
    • Convolutional neural networks in vision
  • Historical fragmentation being reduced:
    • Past: Every tiny NLP problem had own architecture
    • Present: Single transformer handles all different NLP tasks
    • Even earlier: More fragmentation, every AI problem had specialized sub-field

Future Unification Predictions

  • Vision and natural language expected to become unified
  • Eventual broader unification between RL and supervised learning predicted
  • Vision of "one big black box" - shovel everything in, it figures out what to do
  • Uncertainty about timeline due to computational efficiency concerns

Reinforcement Learning Distinctions

  • RL requires slightly different techniques:
    • Need for action-taking
    • Exploration requirements
    • Much higher variance
  • RL operates in fundamentally non-stationary world:
    • Actions change what you see
    • Experience world differently as behavior changes
  • Contrasts with traditional static problems with fixed distributions

Technical Commonalities Across Domains

  • Same core techniques used:
    • Gradient-based optimization
    • Neural networks
    • Adam optimizer
  • RL has additional variance reduction tools for gradients
  • Differences are "small" but "not completely insignificant"

Problem Difficulty Philosophy

  • Questioning what "hard" means for AI problems
  • Traditional benchmark approach: effort required to reach human-level performance
  • "Once you solve a problem it stops being hard" - relative to current tools
  • Current assessment: Both human-level language understanding and visual perception are hard
    • Cannot be solved completely "in next three months"

Language vs Vision Complexity Debate

  • Chomsky perspective: Language fundamental to everything, underlies all cognition
  • Boundary question: Where does vision end and language begin?
    • Example: Reading text from images - vision or language problem?
  • Two possibilities for deep understanding:
    • Impossible to achieve without same kind of system for both
    • Getting one domain gives you the other "for free"
  • Definitions matter significantly for determining difficulty

Human-Level Performance and Continuous Engagement

  • Systems impress briefly, then expectations adjust
  • Humans continue to provide surprises, humor, wit, insights
  • Subjective tests like humor as measure of intelligence
  • AI systems as of January 2020 not achieving sustained impressive performance

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome