AI summary by TwinMind
How to Build AGI? (Ilya Sutskever) | AI Podcast Clips
Lex Fridman
18m 46s
AGI Development Requirements
- Deep learning plus additional ideas needed for human-level intelligence
- Self-play identified as key component
- Has property of surprising in novel ways
- Examples: OpenAI multi-agent hide-and-seek, AlphaZero
- Produces creative solutions to problems
- AGI systems should find surprising but useful solutions to problems
- Current self-play mechanisms mostly in games/simulation contexts
Simulation vs Real World Learning
- Simulation as tool with strengths and weaknesses
- Transfer from simulation to real world is possible
- Demonstrated many times by different groups
- Especially successful in vision
- OpenAI robot hand example (Rubik's Cube)
- 100% training done in simulation
- Policy trained to be adaptive for real-world transfer
- Novel perturbations (giraffe, glove) not in training
- Clean example of sim-to-real transfer
- Expect deep learning transfer capabilities to increase
- Humans learn from simulation (games) and apply to real world
Embodiment and Consciousness
- Body useful but not necessary for AGI
- Can learn things impossible without body
- Can compensate without body (Helen Keller example)
- Consciousness definition challenges
- Fascinating and possibly emergent from neural representations
- Humans are conscious; if ANNs sufficiently similar to brain, should be possible
- Existence proof argument for conscious AI
- Brain may be more complicated than credited
Intelligence Testing
- Turing test with natural language discussed
- Impressive capability: system that never makes mistakes humans wouldn't make
- Current systems make different set of mistakes than humans
- Would inspire confidence in true understanding
- GPT-2 potentially smarter than humans in breadth/depth of knowledge
- Criticism pattern: finding edge cases where AI fails, humans don't
- Public convinced systems not intelligent from anecdotal failures
- People will be impressed when AI moves GDP needle
First AGI Interaction
- Would ask all kinds of questions to find mistakes
- Questions would be factual, personal, emotional, psychological
- Would ask for advice
- No self-imposed limitations on conversation topics
Power and Control Dynamics
- Lincoln quote: "Nearly all men can stand adversity, but if you want to test a man's character, give him power"
- AGI creation represents ultimate 21st/22nd century power
- Ideal vision: humanity as board members, AGI as CEO
- Multiple AGIs for cities/countries
- People vote on what their AGI should do
- Democratic process taken to next level
- Board can always "fire CEO" (reset parameters)
AGI Control and Alignment
- Possible to build AI systems that want to be controlled by humans
- Parent-child analogy: parents excited to help children succeed
- AGI could have deep drive to help humans flourish
- Power relinquishment crucial (George Washington example)
- Speaker finds prospect of controlling AGI "terrifying"
- Believes people can be better than expected when it counts
- Alignment as RL problem: training value function to internalize human judgments
- Separate objective perception system trained on human judgments
- Integrated as base value function for capable RL system
