AI summary by TwinMind

How to Build AGI? (Ilya Sutskever) | AI Podcast Clips

Lex Fridman

18m 46s

AGI Development Requirements

  • Deep learning plus additional ideas needed for human-level intelligence
  • Self-play identified as key component
    • Has property of surprising in novel ways
    • Examples: OpenAI multi-agent hide-and-seek, AlphaZero
    • Produces creative solutions to problems
    • AGI systems should find surprising but useful solutions to problems
  • Current self-play mechanisms mostly in games/simulation contexts

Simulation vs Real World Learning

  • Simulation as tool with strengths and weaknesses
  • Transfer from simulation to real world is possible
    • Demonstrated many times by different groups
    • Especially successful in vision
  • OpenAI robot hand example (Rubik's Cube)
    • 100% training done in simulation
    • Policy trained to be adaptive for real-world transfer
    • Novel perturbations (giraffe, glove) not in training
    • Clean example of sim-to-real transfer
  • Expect deep learning transfer capabilities to increase
  • Humans learn from simulation (games) and apply to real world

Embodiment and Consciousness

  • Body useful but not necessary for AGI
    • Can learn things impossible without body
    • Can compensate without body (Helen Keller example)
  • Consciousness definition challenges
    • Fascinating and possibly emergent from neural representations
    • Humans are conscious; if ANNs sufficiently similar to brain, should be possible
    • Existence proof argument for conscious AI
  • Brain may be more complicated than credited

Intelligence Testing

  • Turing test with natural language discussed
  • Impressive capability: system that never makes mistakes humans wouldn't make
    • Current systems make different set of mistakes than humans
    • Would inspire confidence in true understanding
  • GPT-2 potentially smarter than humans in breadth/depth of knowledge
  • Criticism pattern: finding edge cases where AI fails, humans don't
  • Public convinced systems not intelligent from anecdotal failures
  • People will be impressed when AI moves GDP needle

First AGI Interaction

  • Would ask all kinds of questions to find mistakes
  • Questions would be factual, personal, emotional, psychological
  • Would ask for advice
  • No self-imposed limitations on conversation topics

Power and Control Dynamics

  • Lincoln quote: "Nearly all men can stand adversity, but if you want to test a man's character, give him power"
  • AGI creation represents ultimate 21st/22nd century power
  • Ideal vision: humanity as board members, AGI as CEO
    • Multiple AGIs for cities/countries
    • People vote on what their AGI should do
    • Democratic process taken to next level
    • Board can always "fire CEO" (reset parameters)

AGI Control and Alignment

  • Possible to build AI systems that want to be controlled by humans
  • Parent-child analogy: parents excited to help children succeed
  • AGI could have deep drive to help humans flourish
  • Power relinquishment crucial (George Washington example)
  • Speaker finds prospect of controlling AGI "terrifying"
  • Believes people can be better than expected when it counts
  • Alignment as RL problem: training value function to internalize human judgments
    • Separate objective perception system trained on human judgments
    • Integrated as base value function for capable RL system

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome