AI summary by TwinMind

Jeff Dean: AI isn't as smart as you think -- but it could be | TED

TED

18m 31s

Jeff Dean Background & AI Progress Overview

  • Jeff Dean, AI Research & Health lead at Google, 20+ years at company
  • Started in tiny office above T-Mobile store in downtown Palo Alto
  • Last decade: tremendous AI progress in computer vision, language understanding, speech
  • Computer vision transformation: 10 years ago computers couldn't see, now they can
  • Applications: flood prediction, 100+ language translation, disease prediction/diagnosis

Neural Networks & Computational Power Foundation

  • Two key components underlying AI progress:
    • Neural networks: breakthrough approach, shining last 15 years (not new idea)
    • Computational power: required to make neural networks effective
  • Neural networks loosely based on real neural systems
  • Ideas from 1960s-70s
  • Individual neuron: set of inputs with associated weights, output = function of inputs × weights
  • Learning: repeatedly making tiny adjustments to weight values, strengthening influences
  • Can train for complex tasks: translation, object detection

Historical Context & Early Challenges

  • Since computing beginning: people wanted computers to see, understand language/speech
  • Early approaches: hand-coded algorithms, didn't work well
  • Jeff's 1990 undergraduate experience:
    • Took neural networks class, excited about potential
    • University of Minnesota had 32-processor machine
    • Senior thesis: parallel training of neural networks
    • Was wrong about computational needs
  • Required ~1 million times more computational power than available in 1990
  • 2005: Moore's law provided necessary computing power
  • Universities worldwide started seeing neural network successes

Google's Neural Network Breakthrough

  • Google team started project to train very large neural networks
  • Trained system with 10 million randomly selected YouTube video frames
  • System developed object recognition capabilities
  • Notably: learned to recognize cats without being told what cats were
  • Used patterns in data to develop cat concept independently
  • Beginning of decade-long string of successes at Google and elsewhere
  • Applications: speech recognition, search quality, maps geographic understanding

TPU Development

  • Built hardware tailored for neural network computations
  • Neural networks have two special properties:
    • Tolerant of reduced precision (couple significant digits vs 6-7)
    • Algorithms composed of matrix and vector operations sequences
  • TPU v1 (Tensor Processing Unit): specialized for low-precision matrix/vector operations
  • Used behind every Google search, translation, DeepMind AlphaGo matches
  • Lee Sedol and Ke Jie competed against racks of TPU cards
  • Built subsequent improved TPU versions

Three Key Problems with Current AI

Problem 1: Single-Task Models

  • Most neural networks trained for one thing only
  • Heavyweight process: curate dataset, decide architecture, initialize weights, apply computation
  • Results in thousands of separate models for different tasks
  • Unlike human learning: people leverage existing knowledge for new skills
  • Example: Jeff's hydroponic gardening - didn't need to relearn basic plant knowledge
  • Current approach like "forgetting entire education every time you try something new"

Problem 2: Single Modality Focus

  • Most models deal with only one data type: images OR text OR speech
  • Humans continuously use all senses to learn, react, take actions
  • Should build models that process text, images, speech simultaneously
  • Fuse modalities so "leopard" concept triggered whether seeing word, video, or hearing spoken word
  • Can extend to non-human inputs: genetic sequences, 3D point clouds

Problem 3: Dense Models

  • Single model fully activated for every task/example
  • Same activation whether simple or complicated task
  • Unlike human brains: different parts good at different things
  • Example: watching garbage truck back up - Shakespearean sonnet brain part inactive
  • Should use sparsely activated models: call upon relevant parts for specific tasks
  • Advantage: high-capacity but efficient, only using needed parts

Pathways System Solution

  • Building system enabling new approaches called "Pathways"
  • Capabilities:
    • Handle thousands/millions of different tasks
    • Incrementally add new tasks
    • Deal with all modalities simultaneously
    • Learn new tasks as needed
    • Call upon relevant model bits for different examples/tasks
  • Expected to be step forward in AI system building

Responsible AI Considerations

  • Need to ensure powerful AI systems benefit everyone
  • Important questions: fairness, interpretability, privacy, security
  • Training on thousands/millions of tasks requires large data amounts
  • Data must be thoughtfully collected, representative of different communities/situations worldwide
  • 2018: Google published AI principles for technology development
  • Guides research approach and product AI usage
  • Principles updated as learning continues
  • Hundreds of people working on related research issues

Future Applications & Impact

  • Move from single-purpose pattern recognition to general-purpose intelligent systems
  • Deeper world understanding enables tackling greatest humanity problems:
    • Diagnose more diseases
    • Engineer better medicines (chemistry/physics knowledge infusion)
    • Advance educational systems (individualized tutoring)
    • Tackle climate change, clean energy solutions
  • Requires multidisciplinary expertise worldwide
  • AI potential to help billions of people vs current millions

Q&A Session with Chris Anderson

AI Pattern Recognition Evolution

  • Traditional AI: computers recognize information patterns, get better than humans
  • New approach: richer-layered concepts including various components (leopard example)
  • Grand challenge: generalize from known tasks to new tasks easily
  • Current separate models need lots of data, learning everything from nothing
  • Pathways approach: systems infused with thousands/millions of tasks knowledge
  • Can teach new tasks with relatively few examples (5 vs 10,000-100,000)

Potential Risks & Safeguards

  • AI can be powerful force for good or negative consequences depending on application
  • Importance of principles for evaluating AI uses, careful/thoughtful application consideration
  • Concern: AI learning from world "as it is" vs "as it should be"
  • Research focus: adapt/augment data to reflect desired values vs observed world values
  • Hundreds of people working on fairness, interpretability research topics

Google's Research Independence

  • Research group collaborates with Ads, Search, Maps groups
  • Also conducts basic research published openly
  • 1,000+ papers published last year on various topics including fairness, interpretability
  • Need to advance state of art for safe, responsible model development
  • Importance of showing world this work creates better future for Google and everyone

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome