AI summary by TwinMind

The Secret Language Scaling WhatsApp and Discord

Theo - t3․gg

28m 31s

Introduction to Erlang/Elixir for Scaling

  • Big companies use Rust/Go but secret weapon: Erlang from 1980s
  • Built for phone companies, now powers real-time apps
  • Elixir: modern syntax on top of Erlang VM (BEAM)
  • Author excited about Elixir Conference August 27-30 Orlando

Origins and History

  • Erlang developed late 1980s by Ericsson (Swedish telecom)
  • Goal: massive concurrency + high availability for telecom systems
  • Pre-internet era - designed for phone companies
  • Lightweight processes, message passing, fault tolerance
  • BEAM (Bogdan/Bjorn Erlang Abstract Machine) - virtual machine concept
  • Revolutionary at time - abstract machine vs direct hardware compilation

Elixir Creation Story

  • Created 2011 by Jose Valim (Ruby developer)
  • Built OAuth implementation for Ruby
  • Frustrated with Ruby's single-threaded scaling limitations
  • Load balancing invented largely to handle Ruby's scaling issues
  • Loved Erlang capabilities but hated syntax
  • Combined Erlang power with Ruby-like beautiful syntax
  • Added functional programming features

Five Core Features Making BEAM Powerful

1. Fault Tolerance

  • "Let it crash" philosophy
  • Individual process failures don't crash entire system
  • Supervisors automatically restart failed processes
  • Processes isolated - failures contained
  • Hot code swapping possible - change code live without downtime

2. Scalability

  • Handles inter-node communication seamlessly
  • Simple node connection: Node.connect("node_two@ip")
  • Run code on different machines: Node.spawn(function)
  • GenServer for distributed computing
  • Chunks arrays across multiple nodes automatically
  • No complex distributed systems like Kafka/RabbitMQ needed

3. Distribution

  • Treats local/remote processes identically
  • Easy to add nodes to cluster
  • No complex load balancers or sharding strategies
  • Beam handles distribution automatically
  • Same source code assumed available on all nodes

4. Responsiveness

  • Preemptive scheduler with execution windows
  • Max 500 function calls per process before switching
  • Similar to JavaScript event loop but multi-node
  • IO operations delegated to separate threads
  • No process blocking from long-running operations

5. Live Code Updates

  • Hot code swapping without restarting application
  • Critical for real-time apps requiring continuous operation
  • Update WebSocket servers without disconnecting users
  • Fix bugs/add features without service interruption

Performance Philosophy Discussion

  • Performance != just raw computation speed
  • System performance more important than individual function speed
  • Rust faster for single functions, Elixir better for distributed systems
  • Elixir makes scaling to multiple servers trivial
  • Parallelization easier than micro-optimizations
  • When need more performance: spin up another server

Case Study 1: WhatsApp

  • Founded 2009 by Brian Acton & John Koum
  • Chose Erlang for fault tolerance, scalability, distribution
  • Based on ejabberd (open source XMPP server in Erlang)
  • Exponential growth to 1+ billion daily active users by 2018
  • At acquisition: 32 engineers total, ~6 on infrastructure
  • $19 billion acquisition with 6 engineers serving 450M users
  • Handled millions messages/day with minimal downtime
  • Anton Lavrik (WhatsApp engineer): concurrency Erlang's best attribute

Case Study 2: Discord

  • Founded 2015 by Jason Citron & Stanislav Vishnevsky
  • Originally for gamers, chose Python + Elixir
  • Core real-time chat still powered by Elixir
  • Popular servers (Fortnite, Minecraft) handle hundreds of thousands concurrent users
  • 2019 milestone: 11+ million concurrent users
  • Used Rust for member list via NIFs (Native Implemented Functions)
  • Combined Rust performance with Elixir scalability/productivity

Case Study 3: Food Focus App

  • Author's personal project: live cooking video + chat
  • Smartwatch integration with synchronized recipe timers
  • Tech requirements: real-time features, ready-to-use tools, good DX
  • Chose Elixir/Phoenix backend (Flutter frontend - author regrets)
  • Key tools used:
    • Phoenix Channels (WebSocket-based real-time API)
    • ETS (Erlang Term Storage) - built-in key-value store
    • Ecto (database operations)
    • Oban (background jobs)
    • Electric SQL (local-first queries)
  • Delivered on time and budget
  • Less time configuring third-party libraries, more time building features

Other Companies Using Erlang/Elixir

  • Spotify, Pinterest, PepsiCo, Financial Times, Heroku
  • Fly.io (built entirely on Elixir)
  • Twitch (used for marathons)
  • All achieve high concurrency, fault tolerance, scalability

Comparison with Other Languages

  • Gleam: same benefits when using Erlang backend
  • JavaScript: similar event loop concepts but single-node
  • Go: easier parallelization than Rust
  • Rust: great for single-core performance, struggles with async/concurrency
  • Java: similar VM concept but not used for distribution

Technical Details

  • Lightweight processes: thousands/millions concurrent without performance degradation
  • Message passing without shared memory
  • Node discovery and orchestration built-in
  • Pipe operator for functional programming
  • Pattern matching (one of earliest languages with it, after ML)
  • RPC calls across nodes trivial

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome