AI summary by TwinMind

Sam Altman: OpenAI CEO on GPT-4, ChatGPT, and the Future of AI | Lex Fridman Podcast #367

Lex Fridman

2h 23m 56s

Open AI's Early Days and Mockery

  • Open AI announced AGI goal end of 2015, people thought they were "batshit insane"
  • Eminent AI scientist at large industrial lab DMed reporters saying Open AI wasn't very good
  • Only Open AI and DeepMind were brave enough to talk about AGI despite field mockery
  • Now they don't get mocked as much

GPT-4 Overview and Architecture

  • GPT-4 will be viewed as very early AI in retrospect - slow, buggy, doesn't do many things well
  • Similar to early computers that pointed path to something important
  • Progress viewed as continual exponential curve rather than single breakthrough moment
  • ChatGPT (not underlying model) was the pivotal moment due to usability via RLHF and interface
  • RLHF (Reinforcement Learning with Human Feedback) - show two outputs, ask which is better, feed back with reinforcement learning
  • Remarkably little data needed for RLHF to make model much more useful
  • RLHF aligns model to what humans want it to do

Training Data and Model Development

  • Pre-training dataset pulled from many sources: open source databases, partnerships, internet content
  • Includes Reddit, newspapers, general web - "more content than most people think"
  • Task is filtering rather than finding content
  • Multiple complex components: algorithms, architecture, neural network size, data selection, human supervision
  • GPT-3 to GPT-4 involved hundreds of small technical improvements multiplied together
  • Open AI good at finding small wins and multiplying them together

Model Size and Scaling

  • Sam clarifies he didn't predict GPT-4 would have 100 trillion parameters (journalist misinterpretation)
  • Parameter count race similar to gigahertz race in processors
  • What matters is performance, not parameter count
  • Focus on doing whatever makes best performance rather than most elegant solution

Capabilities and Reasoning

  • GPT-4 can do "some kind of reasoning" for some definition of reasoning
  • Remarkable that reasoning capability emerges from ingesting human knowledge
  • Can compress huge amount of web content into organized "human wisdom"
  • Distinction between using model as database vs reasoning engine
  • System can struggle with tasks that seem obvious (like counting characters) due to architecture

Safety and Alignment Process

  • Finished GPT-4 training summer 2022, immediately started safety work
  • Gave to people for red teaming, internal safety evals, built new alignment methods
  • Goal: degree of alignment increases faster than rate of capability progress
  • GPT-4 most capable and most aligned model they've released
  • Current RLHF works for current systems but may not scale to super powerful systems
  • Alignment and capability are closely linked - better alignment techniques lead to better capabilities

System Message and Steerability

  • System message allows users to specify behavior (e.g., "respond as Shakespeare")
  • GPT-4 tuned to treat system message with high authority
  • Still vulnerable to jailbreaks but improving
  • Goal to give users personalized, granular control over time

Bias and Content Moderation

  • GPT-4 much less biased than GPT-3.5 according to critics
  • No single model will ever be viewed as unbiased by everyone
  • Need broad societal agreement on bounds, then different countries/users can have different versions
  • Moderation includes systems to refuse to answer certain questions
  • Sam dislikes feeling "scolded by a computer" - wants to treat users like adults
  • GPT-4 has enough nuance to explore controversial topics without being preachy

Programming and Code Generation

  • GPT-4 significantly changing programming within days of release
  • Enables iterative dialogue for code debugging and modification
  • Amazing leverage for programmers to do creative work better
  • May lead to much more code being written rather than fewer programmers

Economic and Job Impact

  • Customer service likely category for significant job displacement
  • Every technological revolution eliminates some jobs, enhances others, creates new ones
  • Society confused about whether people want to work more or less
  • Goal to move people to better, more fulfilling jobs
  • Universal Basic Income as component of solution during transition
  • Economic transformation will drive political transformation

AI Safety Concerns

  • Current worries: disinformation problems, economic shocks beyond what we're prepared for
  • Concern about LLMs directing social media discourse without detection
  • Soon will be many capable open source LLMs with few safety controls
  • Need to try regulatory approaches, use more powerful AIs to detect problems
  • Fear of fast AI takeoff scenarios
  • Prefers slow takeoff with short timelines as safest scenario

Consciousness and AGI Timeline

  • Doesn't think GPT-4 is conscious, refers to systems as "it" not "him/her"
  • Believes AI can be conscious but GPT-4 doesn't feel close to AGI
  • Important to treat current systems as tools, not creatures
  • Ilya's consciousness test: train model without consciousness concepts, see if it recognizes subjective experience when described

Microsoft Partnership

  • Microsoft understood Open AI's need for unusual control provisions
  • Satya Nadella both great leader and great manager - rare combination
  • Partnership works well despite complexity of integrating with large company
  • Control provisions help ensure capitalist imperatives don't override AI development priorities

Open AI Structure and Culture

  • Started as nonprofit, created capped profit subsidiary when couldn't raise enough as nonprofit
  • Nonprofit maintains voting control, can cancel equity, make non-standard decisions
  • Cap is 100x for Open AI (much lower for new investors)
  • Sam spends 1/3 of time on hiring, approves every hire
  • High bar for team members, hard work, high trust and autonomy
  • Building in public philosophy despite PR risks

Silicon Valley Bank Crisis

  • SVB horribly mismanaged - bought long-dated instruments secured by short-term deposits
  • Example of incentive misalignment and how fast things can change with social media/mobile banking
  • Speed of bank run much faster than 2008 due to technology
  • Preview of shifts AGI will bring to institutions
  • Need higher FDIC insurance limits or full deposit guarantees

Future Conversations with AGI

  • Excited to ask about physics, theory of everything, faster-than-light travel
  • First question might be about other intelligent alien civilizations
  • AGI might not answer directly but could tell us what experiments to run or detectors to build
  • If told aliens already here, would just continue normal life

Advice for Young People

  • Wrote blog post "How to be successful" with principles like compound yourself, self-belief, think independently
  • But advises caution about taking too much advice from others
  • What worked for him may not work for others
  • Mostly got what he wanted by ignoring advice
  • Focus on what brings joy, fulfillment, and usefulness
  • Life often feels like "going along with the current"

Meaning and Philosophy

  • Views AGI development as culmination of all human effort, not work of small group
  • Everything from transistor discovery to current chips involved millions of people
  • Humanity on exponential curve of progress
  • Wonders how many other intelligent civilizations exist
  • Believes they're "in this together" as human civilization
  • Optimistic about working hard to ensure good outcomes

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome