AI summary by TwinMind

Matt Botvinick: Neuroscience, Psychology, and AI at DeepMind | Lex Fridman Podcast #106

Lex Fridman

2h 0m 32s

Current Understanding of the Human Brain

  • Weird moment in neuroscience history - understand brain at high functional level but very coarse
  • High level = what brain computes, what behaviors need explaining
  • Incredible progress in technologies to see single unit/dendritic level activity
  • Yawning gap between high-level cognitive science and neuronal-level neurobiology
  • Brain's purpose: producing adaptive behavior from perceptual inputs to behavioral outputs
  • Psychology studies structure of that function; neuroscience studies mechanistic implementation
  • Gap between electrical signals and behavior much larger than kidney analogy

Prefrontal Cortex and Cognitive Control

  • Prefrontal cortex = anatomical region in front of motor cortex areas
  • WWI brain damage studies revealed difficulty pinpointing frontal damage deficits
  • Russian neuropsychologist Luria identified frontal cortex role in behavioral flexibility
  • Distinction: controlled vs automatic behavior, goal-directed vs habitual behavior
  • Prefrontal cortex enables overriding habits, switching between contexts
  • Example: elbow bump vs handshake during COVID - prefrontal cortex allows context-appropriate behavior
  • Cross-species question: mice have prefrontal cortex but debate over cognitive flexibility capabilities
  • Brain shows functional differentiation but with graded rather than discrete boundaries

Neural Communication and Information Processing

  • Rate code paradigm: spike frequency as primary information carrier
  • Standard neuroscience: neurotransmitters → voltage changes → spikes → axonal conduction
  • Deep learning networks use spike rate approximation (activity levels)
  • Alternative theories: precise spike timing, dendritic computations, mechanical signals
  • Circumstantial evidence from AI-neuroscience comparisons suggests rate code validity
  • Primary visual cortex carries unexpected information about behavior and reward context

Meta-Learning and the Prefrontal Cortex Paper

  • Meta-learning: learning algorithm gives rise to another learning algorithm
  • Training recurrent neural networks on sequences of related tasks
  • Slow synaptic learning shapes network dynamics into fast learning algorithms
  • Example: two-button reward probability tasks in sequence
  • Network dynamics become learning algorithm even with frozen weights
  • Prefrontal cortex connection: highly recurrent, working memory, activity-based learning
  • Dopamine-mediated synaptic learning may shape prefrontal activity dynamics
  • Emergent phenomenon - happens automatically with memory + RL + task variety
  • Real world saturated with "endless variety with endless redundancy"

Distributional Reinforcement Learning and Dopamine

  • Traditional RL: single number value representation (expected reward)
  • Distributional RL: full probability distribution over possible rewards
  • Accelerates learning, enables richer representation learning
  • Separates situations with same expected value but different distributions
  • Dopamine traditionally viewed as reward prediction error signal
  • New hypothesis: dopamine uses distributional coding for surprise signals
  • Collaboration with Naoshige Uchida at Harvard
  • Paper: "A Distributional Code for Value in Dopamine-Based Reinforcement Learning"
  • Predictions confirmed in experimental data
  • Suggests brain respects variety of future outcomes rather than single average

Human-AI Interaction Research

  • Shift from traditional cognitive psychology to human-agent interaction
  • Making world better for humans requires understanding human preferences
  • Learning systems must interact with humans, not just observe
  • Complex questions: conflicting human preferences, cultural values, social choice theory
  • Two dimensions of human evaluation: capability and warmth
  • AI focuses on capability; warmth dimension largely unexplored
  • Ultimate Turing test: AI system humans consider "a good guy"
  • Humans learn warmth skills from each other
  • Possible to build systems that learn gratifying human interaction

Future Directions

  • Neuroscience: explosion in new technologies, virtual reality for animal behavior
  • Rebirth of interest in behavior structure with AI-informed computational mechanisms
  • AI challenges: flexibility, multi-task competence, rapid task switching
  • Focus on abstraction, cognitive control, prefrontal cortex functions
  • Virtuous circle between AI and neuroscience
  • Currently AI ahead due to innovation burst
  • Inevitable future: neuroscience will provide insights back to AI
  • Community benefit from intellectual diversity across spectrum

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome