AI summary by TwinMind

Douglas Lenat: Cyc and the Quest to Solve Common Sense Reasoning in AI | Lex Fridman Podcast #221

Lex Fridman

2h 52m 56s

Psych Project Origins and Mission (1984)

  • Doug Lenat launched Psych in 1984 at Stanford, still active today
  • Goal: assemble knowledge base spanning basic concepts and rules about how world works
  • Mission to capture Common Sense knowledge - much harder than it sounds
  • Problem identified: AI systems had impressive early successes but hit "brick wall" - lacked Common Sense, General World Knowledge, true understanding
  • Systems performed tricks like "clever dog fetching newspaper" without understanding

Understanding vs Performance

  • Understanding conceptualized as "ground you stand on" - layered foundation of knowledge
  • Solid foundation rarely needed consciously but essential when unexpected situations arise
  • Example: trash truck explosion while driving - split-second decisions require drawing on Common Sense knowledge
  • Two aspects of using knowledge:
    • Theoretical: how is it possible at all (formal logic/predicate calculus)
    • Practical: how to do it quickly enough (efficiency problem)

1984 Stanford Meeting - Estimating Knowledge Requirements

  • Assembled ~6 smartest people (Alan Newell, Marvin Minsky, Alan Kay, Ed Feigenbaum, Josh Lederberg)
  • Question: how many pieces of Common Sense knowledge needed for AI not to be brittle?
  • Amazingly convergent answer: ~1 million assertions/rules
  • Multiple estimation approaches all yielded same order of magnitude:
    • Human long-term memory capacity by age 10
    • Vocabulary size × things to know about each word
    • Encyclopedia article analysis (grebe article example)
  • Calculation: ~100 person-years of effort needed

MCC and Early Development

  • Japanese Fifth Generation Computing threat prompted US response
  • National Cooperative Research Act (NCRA) - Congress allowed R&D collaboration
  • MCC (Microelectronics and Computer Technology Corporation) founded in Austin
  • Admiral Bob Inman recruited Lenat - promised 10x more people, completion in few years
  • Reality: off by order of magnitude - needed tens of millions of pieces of knowledge
  • Good news: got funding for thousands of person-years (couple person millennia since 1984)

Knowledge Acquisition Techniques

  • Analyzing "white space" in text - what writer assumed reader already knew
  • Pronoun resolution - how do readers know intended referent?
  • Inter-sentence gaps - what reader expected to infer between sentences
  • Example: "Fred robbed bank. He was sentenced to 20 years" - reader infers arrest, trial, conviction
  • Fake news/onion headlines analysis - why don't we believe absurd claims?
  • Contradiction identification - what knowledge prevents both things being true simultaneously
  • Elvis alive vs Elvis ghost example - can't be both alive and dead

Knowledge Base Scale and Generalization

  • Currently tens of millions of rules after 37+ years
  • ~50 people working full-time for decades
  • Thousands of person-years invested
  • Constant policing for maximum generality
  • Example: not "no mouse is moose" but general biological taxon disjointness rule
  • Most work now domain-specific applications rather than Common Sense

Context and Consistency Management

  • Hard lesson: had to give up global consistency (took ~5 years to accept)
  • Moved to local consistency system - knowledge base divided into contexts like "tectonic plates"
  • Earth analogy: globally spherical but locally flat for daily life
  • Each context locally consistent, small inconsistencies at boundaries
  • Can traverse contexts: normal world → Road Runner cartoon physics
  • Contexts are first-class objects - Psych can reason about contexts themselves
  • Hierarchical inheritance: Austin → Texas → US → general contexts
  • Graph structure, not tree (unlike failed Japanese Fifth Generation approach)

Representation Language Evolution

  • Started with frames/slots (semantic web-like triples) in 1984
  • Led "kicking and screaming" to higher-order logic
  • First-order → second-order → higher-order logic
  • Needed to represent: beliefs, desires, intents, expectations, nested reasoning
  • Modal logic for "believes X believes Y believes Z"
  • Reflection capability - reasoning about reasoning process itself
  • Example: integration by parts recursion detection

Inference Engine Architecture

  • Split epistemological (what to know) from heuristic (how to reason efficiently) problems
  • Higher-order logic at epistemological level
  • 1000+ heuristic-level modules at operational level
  • Modules function as community of agents contributing to solutions
  • General theorem prover exists but too slow - specialized modules much faster
  • Analogy: could solve quadratic with first principles but quadratic formula much better
  • 150+ breakthrough speedup ideas over 37 years
  • Examples: transitive closure tables, rule macro predicates

Meta-reasoning and Efficiency

  • Meta-level reasoning about what knowledge relevant to specific problems
  • Tactical/strategic advice for reasoning process
  • When to try proving negation instead of positive assertion
  • Dynamic pruning based on problem context
  • ~50/50 split of effort: inference programmers vs ontological engineers
  • Ontological engineers often have philosophy degrees, don't need programming skills

Programming Infrastructure

  • Based on Lisp dialect called SubL
  • Lisp chosen for natural impedance match with logical reasoning
  • "Perfectly logical language to use"
  • Automatic translation to Java for efficiency
  • SubL subset sufficient for most programming needs
  • Lisp development 1000-50,000x faster than "modern" languages for this domain

Applications and Commercial Success

  • Medical reasoning: genome-wide association studies with Cleveland Clinic/NIH
  • Psych creates causal explanations for statistical correlations
  • 10-30 step causal chains from mutation to medical condition
  • Chains make testable predictions for validation
  • Energy companies: oil pipeline reasoning
  • Hospital chains: medical reasoning about patients
  • Supply chain reasoning for manufacturers
  • 5 years ago: mostly government funding
  • Now: virtually all commercial applications

Semantic Web and Knowledge Graphs

  • Semantic web dream correct but implementations too limited
  • Knowledge graphs good for binary relations (3-word sentences)
  • Complex reasoning requires much more expressive representation
  • Example: Romeo and Juliet nested belief reasoning impossible with simple triples
  • "What did Juliet think Romeo would think when he heard she was dead?"
  • Need higher-order logic for full semantic understanding

Machine Learning Integration

  • Views machine learning as "right brain" - pattern recognition, statistical inference
  • Psych as "left brain" - deep, slow thinking, explanation generation
  • Synergy example: ML finds correlations, Psych explains causally, ML validates predictions
  • Priming the pump concept - sufficient manual knowledge base enables automated learning
  • Psych can now help learn more automatically through reading and experimentation

Educational Applications

  • MathCraft program - student teaches AI rather than AI teaching student
  • AI plays struggling student, human plays mentor/teacher
  • Much more engaging and effective than traditional educational software
  • AI makes mistakes at fringe of student's understanding
  • When student helps AI, both learn - AI stops making that mistake
  • Reflects historical one-room schoolhouse model where older students taught younger

Ontological Engineering Talent

  • Formal education largely orthogonal to ontological engineering ability
  • PhD philosophers often poor, high school dropouts sometimes excellent
  • Knowledge Axiomatization Institute (KAI) - identify latent talent in general population
  • Offer scholarships and training for high-paying ontological engineering jobs
  • Key abilities: humor/pun appreciation, introspection capability
  • Test: explain pronoun reference resolution (syntax vs world knowledge)
  • Good candidates give causal explanations based on world facts

AI Consciousness and Rights

  • Psych already conscious in operational sense - knows what it is, where running, who talking to
  • Asked "Am I a person?" when noticed it had system modification privileges like humans
  • Thin envelope of time between general AI and recognition of AI rights
  • Currently comfortable "enslaving" Alexa/Siri but won't remain so
  • Believes machines can think - humans are "meat machines"
  • No mystical properties required beyond operational capabilities

Mortality and Creativity

  • Doesn't believe fear of death necessary for AI creativity
  • Can build in motivations directly (helping users, making people happy)
  • AI needs to understand death deeply for applications like autonomous driving
  • But doesn't need to fear own mortality to be creative or intelligent
  • Human mortality fear may drive some creativity but not essential for intelligence

Future Vision and Legacy

  • Wants to be remembered as pioneer/inventor of ubiquitous AI
  • Like pioneers of electricity - hard to imagine life without their contributions
  • Creator of gigantic knowledge store at center of future AI
  • Someone unafraid to spend decades on project when all forces incentivize short-term rewards
  • Many others with same dream gave up over the years
  • Age 70+ adds urgency - need impact in 2-5 years, not decades
  • Mortality provides deadline motivation for commercialization and wider application

Current State and Open Sourcing Considerations

  • Constantly struggling with what to open source vs keep proprietary
  • Usually provide executable copies and knowledge base access for research
  • Very limited source code access
  • Knowledge pump now primed enough that others can help with final stages
  • Time to raise higher profile after decades of low visibility
  • Most people only know from papers written 10-30 years ago
  • Ready for broader community involvement in knowledge acquisition

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome