AI summary by TwinMind
Matt Botvinick: Neuroscience, Psychology, and AI at DeepMind | Lex Fridman Podcast #106
Lex Fridman
2h 0m 32s
Current Understanding of the Human Brain
- Weird moment in neuroscience history - understand brain at high functional level but very coarse
- High level = what brain computes, what behaviors need explaining
- Incredible progress in technologies to see single unit/dendritic level activity
- Yawning gap between high-level cognitive science and neuronal-level neurobiology
- Brain's purpose: producing adaptive behavior from perceptual inputs to behavioral outputs
- Psychology studies structure of that function; neuroscience studies mechanistic implementation
- Gap between electrical signals and behavior much larger than kidney analogy
Prefrontal Cortex and Cognitive Control
- Prefrontal cortex = anatomical region in front of motor cortex areas
- WWI brain damage studies revealed difficulty pinpointing frontal damage deficits
- Russian neuropsychologist Luria identified frontal cortex role in behavioral flexibility
- Distinction: controlled vs automatic behavior, goal-directed vs habitual behavior
- Prefrontal cortex enables overriding habits, switching between contexts
- Example: elbow bump vs handshake during COVID - prefrontal cortex allows context-appropriate behavior
- Cross-species question: mice have prefrontal cortex but debate over cognitive flexibility capabilities
- Brain shows functional differentiation but with graded rather than discrete boundaries
Neural Communication and Information Processing
- Rate code paradigm: spike frequency as primary information carrier
- Standard neuroscience: neurotransmitters → voltage changes → spikes → axonal conduction
- Deep learning networks use spike rate approximation (activity levels)
- Alternative theories: precise spike timing, dendritic computations, mechanical signals
- Circumstantial evidence from AI-neuroscience comparisons suggests rate code validity
- Primary visual cortex carries unexpected information about behavior and reward context
Meta-Learning and the Prefrontal Cortex Paper
- Meta-learning: learning algorithm gives rise to another learning algorithm
- Training recurrent neural networks on sequences of related tasks
- Slow synaptic learning shapes network dynamics into fast learning algorithms
- Example: two-button reward probability tasks in sequence
- Network dynamics become learning algorithm even with frozen weights
- Prefrontal cortex connection: highly recurrent, working memory, activity-based learning
- Dopamine-mediated synaptic learning may shape prefrontal activity dynamics
- Emergent phenomenon - happens automatically with memory + RL + task variety
- Real world saturated with "endless variety with endless redundancy"
Distributional Reinforcement Learning and Dopamine
- Traditional RL: single number value representation (expected reward)
- Distributional RL: full probability distribution over possible rewards
- Accelerates learning, enables richer representation learning
- Separates situations with same expected value but different distributions
- Dopamine traditionally viewed as reward prediction error signal
- New hypothesis: dopamine uses distributional coding for surprise signals
- Collaboration with Naoshige Uchida at Harvard
- Paper: "A Distributional Code for Value in Dopamine-Based Reinforcement Learning"
- Predictions confirmed in experimental data
- Suggests brain respects variety of future outcomes rather than single average
Human-AI Interaction Research
- Shift from traditional cognitive psychology to human-agent interaction
- Making world better for humans requires understanding human preferences
- Learning systems must interact with humans, not just observe
- Complex questions: conflicting human preferences, cultural values, social choice theory
- Two dimensions of human evaluation: capability and warmth
- AI focuses on capability; warmth dimension largely unexplored
- Ultimate Turing test: AI system humans consider "a good guy"
- Humans learn warmth skills from each other
- Possible to build systems that learn gratifying human interaction
Future Directions
- Neuroscience: explosion in new technologies, virtual reality for animal behavior
- Rebirth of interest in behavior structure with AI-informed computational mechanisms
- AI challenges: flexibility, multi-task competence, rapid task switching
- Focus on abstraction, cognitive control, prefrontal cortex functions
- Virtuous circle between AI and neuroscience
- Currently AI ahead due to innovation burst
- Inevitable future: neuroscience will provide insights back to AI
- Community benefit from intellectual diversity across spectrum
