AI summary by TwinMind
Future Computers Will Be Radically Different (Analog Computing)
Veritasium
21m 42s
Historical Context of Analog vs Digital Computing
- Analog computers dominated for hundreds of years
- Predicted eclipses, tides, guided anti-aircraft guns
- Digital computers overtook with solid-state transistors
- Today: perfect storm creating analog resurgence
Analog Computer Demonstration
- Physical analog computer demonstration
- Programmed by connecting wires in specific ways
- Solves differential equations (damped mass oscillating on spring)
- Oscilloscope shows position over time
- Parameters adjustable: damping, spring constant, mass
- Lorenz system implementation
- Models atmospheric convection
- Famous chaos example with butterfly attractor
- Real-time parameter adjustment possible
Analog vs Digital Computing Comparison
Analog Advantages
- Incredibly powerful and fast
- Low power consumption
- Simple operations:
- Adding 8-bit numbers: 2 wires vs 50 transistors (digital)
- Multiplication: current through resistor (V = I×R) vs ~1000 transistors (digital)
Analog Disadvantages
- Not general-purpose (can't run Microsoft Word)
- Continuous inputs/outputs prevent exact values
- Non-repeatable calculations
- Manufacturing variations in components (resistors, capacitors)
- ~1% error rate expected
- Single-purpose, non-repeatable, inexact
Early AI Development
Perceptron (1958)
- Created by Cornell psychologist Frank Rosenblatt
- Mimicked brain neuron firing
- 400 photocells in 20×20 pixel grid
- Single output neuron with adjustable weights
- Training algorithm: add/subtract input activations based on correctness
- Could distinguish shapes (rectangles vs triangles, letters)
- Media hype: "embryo of electronic computer...walk, talk, see, write, reproduce itself"
AI Winter (1969)
- MIT's Minsky and Papert critique
- Led to first AI winter
- Rosenblatt drowned at age 43 in Chesapeake Bay
AI Resurgence (1980s)
ALVN Self-Driving Car
- Carnegie Mellon creation
- Neural network with hidden layer (4 neurons)
- Input: 30×32 pixel road images
- 32 output neurons for steering angles
- Training: human driver provided correct steering
- Back-propagation weight adjustment
- Top speed: 1-2 km/h (limited by matrix multiplication speed)
Modern AI Breakthrough
ImageNet Dataset (2006-2009)
- Fei-Fei Li's initiative
- 1.2 million human-labeled images
- Largest labeled dataset at the time
- Annual competition (2010-2017): ImageNet Large Scale Visual Recognition Challenge
- 1000 categories including 90 dog breeds
Performance Evolution
- 2010: 28.2% top-5 error rate
- 2011: 25.8% error rate
- 2012: AlexNet breakthrough - 16.4% error rate
- 8 layers, 500,000 neurons
- 60 million weights/biases
- 700 million operations per image
- Pioneered GPU usage for training
- Paper cited 100,000+ times
- 2015: 3.6% error rate (better than human performance)
- 100-layer neural network
Current AI Challenges
Energy Consumption
- Training neural network = yearly consumption of 3 households
Von Neumann Bottleneck
- Digital computers store data in memory, access over bus
- Matrix multiplications: most time/energy spent fetching weights vs computing
Moore's Law Limitations
- Transistor doubling every 2 years slowing
- Transistor size approaching atomic scale
- Fundamental physical miniaturization challenges
Analog Computing Renaissance
Perfect Storm Factors
- Digital computers reaching limits
- Neural networks exploding in popularity
- Core task: matrix multiplication
- Neural networks don't need digital precision (96% vs 98% confidence acceptable)
- Component variability tolerable
Mythic AI Implementation
- Texas startup creating analog AI chips
- Repurposed digital flash storage cells as variable resistors
- Flash cell operation:
- Normal: store 1 or 0 via electron trapping
- Mythic approach: variable electron count = variable resistance
- Voltage × conductance = multiplication
- Performance: 25 trillion operations/second at 3 watts
- Digital comparison: 25-100 trillion ops/sec at 50-100 watts (larger systems)
Applications
- Augmented/virtual reality (pose capture, rendering)
- Depth estimation from single webcam
- Security cameras
- Autonomous systems
- Manufacturing inspection (Frito-Lay chip quality control)
- Smart home speakers (wake word detection)
Remaining Challenges
- Signal distortion over multiple operations
- Solution: analog-to-digital conversion between processing blocks
- 50 sequential matrix multiplies would distort beyond usability in pure analog
Historical Parallel
- Rosenblatt originally used digital IBM computer for perceptron
- Found it too slow, built custom analog computer with variable resistors and motors
- His neural network concept proved correct - analog approach may also be right
Future Perspective
- Digital dominated last 50 years (music, pictures, video)
- Analog may be better suited for current computing tasks
- Brain analogy: neurons fire digitally (on/off) but thinking is analog (parallel, everywhere)
- True AI may require analog computing power
