AI summary by TwinMind

The First Neural Networks

Asianometry

18m 52s

Origins of Neural Networks (1943-1949)

  • Warren McCulloch (neuroscientist/philosopher) & Walter Pitts (math prodigy) published 1943 paper on neuron networks
    • McCulloch searched for "least psychic event" - settled on neuron's binary firing behavior
    • Inspired by Turing's 1936 paper on logical machines
    • Argued neural networks could perform logical operations, arithmetic, data management
  • McCulloch-Pitts neuron model: weighted sum of inputs, threshold activation, binary output (1 or 0)
    • Very simple compared to biological neurons
    • Lacked transmission delays, synchronized firing, frequency modulation
  • Donald Hebb (Canadian psychologist) proposed learning theory in 1949 book "The Organization of Behavior"
    • "Cells that fire together wire together" - repeated co-firing strengthens connections
    • Introduced concept of connection weights
  • Connectionism movement emerged from this work

AI Field Emergence (1950s)

  • Late 1940s/early 1950s: exciting computer developments
    • EDVAC stored program computer (John Von Neumann consultation)
  • 1955: John McCarthy (Dartmouth professor) proposed summer workshop on thinking machines
    • November 1955: McCarthy, Marvin Minsky, Claude Shannon wrote proposal to Rockefeller Institute
    • Coined term "artificial intelligence"
  • Dartmouth Workshop (June-August 1956)
    • Herbert Simon & Alan Newell presented Logic Theorist program
    • Proved 38 theorems from Principia Mathematica
    • Introduced Physical Symbol System theory
  • Arthur Samuel (IBM) presented checkers program (1952)
    • Programmed IBM 701 in machine assembly (pre-Fortran)
    • 1955: added self-play capability with heuristic editing
    • Coined term "machine learning"
  • Alan Turing (1948) wrote about simulating entire neural networks
    • Predated capable computers, received little discussion

The Perceptron Era (1957-1960)

  • Frank Rosenblatt (psychologist, Minsky's high school classmate) created perceptron
    • Cornell aeronautical laboratory, Office of Naval Research sponsorship
    • Studied pilot landing psychology, applied probability theory to human memory/perception
  • Perceptron Mark 1 (1957): statistical model in IBM 704 computer
  • July 1958: premature press conference before hardware completion
    • New York Times: "first electronic device to think as human brain"
    • Could "perceive, recognize, identify surroundings without human training"
    • Navy officers called it "human being without life"
  • Mark 1 perceptron machine architecture:
    • S units (sensory system): binary light/sound inputs
    • A units (association system): single layer McCulloch-Pitts neurons with weighted sums
    • R units (response system): motor output as lights for operator review
  • Rosenblatt's training methodology:
    • Random initial weights (like baby's brain)
    • Show stimuli (e.g., letter X), get response
    • Adjust weights based on correct/incorrect feedback
    • Claimed finite training cycles for any learnable classification task

Perceptron Movement Growth (Late 1950s-1960s)

  • Rosenblatt visited Stanford Research Institute (now SRI International)
  • Bernard Widrow & student Ted Hoff created Adaline and Madaline devices
  • Public impact: magazine articles asking if human brains would be replaced
  • Rosenblatt's bold claims: perceptron "will be conscious of its own existence," "capable of having original ideas"
  • 1960: Mark 1 machine completed and demonstrated
  • 1962: Rosenblatt published "Principles of Neurodynamics" textbook
    • Acknowledged failings of 1958 introduction
    • Regretted impact of early rhetoric
    • Clarified perceptron as class of models, not single thinking machine

Symbolic AI Opposition (1960s)

  • Marvin Minsky & Seymour Papert became leading symbolic AI advocates
  • Minsky argued "insurmountable limitations" of perceptron
  • Motivations: personal slight over press, competition for research funding
  • Mathematical criticisms circulated as drafts through second half of 1960s
  • Core limitations identified:
    • Single layer perceptrons only handle linearly separable problems
    • XOR (exclusive or) problem: cannot solve with single linear separation
    • Computational complexity scaling issues for complex images
    • Need complete retraining for larger networks
  • Symbolic algorithms could easily solve these problems

The Perceptron Book and AI Winter (1969-1971)

  • 1969: Minsky & Papert published "Perceptrons" book
  • Myth: book ended connectionism movement
  • Reality: single layer limitations already known, multi-layer solutions existed
  • Core problem: no training algorithm for multi-layer networks
    • Rosenblatt's algorithm only worked for single layer
    • Hidden layer training problem unsolved
    • Rosenblatt experimented with frozen first layer, trained second layer
  • Momentum stalled, funding shifted to symbolic AI approaches
  • Stanford team abandoned connectionism
  • Ted Hoff left Widrow's team for Intel, developed first microprocessor ideas

Rosenblatt's Later Work and Death (1968-1971)

  • Shifted focus to chemical transfer of learned behaviors
  • Experiments: trained flatworms in maze, ground up, fed chemicals to naive flatworms
  • Involved in anti-war and anti-racist movements at Cornell
  • Worked with student activists for university representation
  • Democratic party political involvement
  • 1971: died in tragic boating accident at age 43
  • Three congressmen made tribute to his memory
  • Minsky dedicated perceptron book to his memory

Legacy and Future Developments

  • Government funding redirected from neural networks to symbolic AI
  • Neural network research entered decade-long winter
  • Later developments mentioned:
    • Sigmoid output functions replaced binary outputs
    • 1980s: backpropagation algorithm emerged for multi-layer training
    • Rosenblatt coined "back propagating error correction" in 1962 (different from modern algorithm)
    • Neural networks remained in shadows for decades until hardware/problems aligned

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome