AI summary by TwinMind

Tim Sweeney: Fortnite, Unreal Engine, and the Future of Gaming | Lex Fridman Podcast #467

Lex Fridman

4h 25m 19s

Computer Graphics and Human Rendering Challenges

  • Humans hardest part of computer graphics due to millions of years of evolution
    • Dedicated brain systems to detect patterns, faces, infer emotions/intent
    • Cavemen needed to determine if strangers were friendly or threatening
  • Extraordinarily detailed expectations of faces, notice imperfections from CG limitations
  • Advanced hardware for capturing humans
    • Capture sphere with dozens of cameras taking high resolution, high frame rate video
    • Range of motions, facial muscle/tendon/fat interactions for expressions
    • Not just shape but entire range of motion possibilities
  • Hair rendering extremely difficult
    • Can't render every hair (billion times too slow)
    • Need approximations capturing net effect without calculating every light interaction
  • Subsurface scattering critical for realistic skin
    • Skin not completely opaque, light travels through it
    • Huge impact on appearance
    • Why painted mannequins never look realistic
  • Oiliness creates specific nuanced complex reflections
  • Light absorbed and travels through skin creating textures human eyes perceive
  • Subtle expressions: squinting, smile formation, difference between real/fake smiles
  • Getting faces right requires dozens of different systems interplaying
    • If any one wrong, eye completely drawn to it (uncanny valley)

Tim Sweeney's Programming Origins and Early Epic Games

  • First programming experience age 11 visiting brother Steve (16 years older) in California
  • Brother had IBM PC, learned BASIC in few days
  • Vivid memory of first programs: asterisk-catching game in text mode
  • 10-15,000 hours coding age 10-20 before shipping first commercial program
  • Built databases, programming language/compiler (didn't know where to buy one)
  • Bulletin board systems before internet
    • Used modem to dial local phone numbers
    • Every town had hundreds of bulletin boards with different themes
  • ZZT origin story (1991)
    • Started as text editor project
    • Added smiley face cursor, different characters performed gameplay actions
    • Evolved into game editor with ~20-30 objects
    • User testing with neighbors of all ages
    • Released first game free, sold sequels for $30 via mail order
    • Making $100/day as 20-year-old
  • Shareware model precedent to free-to-play
  • Always included editor tools with games - core Epic philosophy

Unreal Engine Development Journey

  • Wolfenstein/Doom breakthrough moment - future was 3D
  • Initially depressed by Carmack's technical wizardry, stopped programming 6 months
  • Michael Abrash's articles explained 3D texture mapping techniques
  • 3.5 years development, constantly thinking 6 months from shipping
  • Team grew to ~20 people, nearly bankrupted Epic multiple times
  • Early engine licensing deals (Microprose, GT Interactive) provided crucial funding
  • Technical innovations:
    • Constructive solid geometry in real-time (30-hour coding session)
    • Dynamic colored lighting with shadows
    • Volumetric fog (inspired by Finnish GPU company screenshot)
    • Light mapping techniques for performance

Unreal Engine Evolution (Versions 1-5)

  • UE1: Software rendering, later Voodoo GPU support
  • UE2: GPU hardware acceleration, vehicles
  • UE3: DirectX9 programmable shaders, real-time lighting, multi-threading
  • UE4: Continued improvements, more artist options
  • UE5: Major leap with Nanite (micropolygon geometry) and Lumen (global illumination)
  • Some systems unchanged since 1998 (file management, networking)
  • Single-threaded game simulation major limitation (using 1 core of 16-core CPU)
  • UE6 plans: Address core limitations, multi-threading, few years away

Nanite and Lumen Technical Details

  • Nanite: Virtualized micropolygon geometry system
    • Handles billions of polygons efficiently
    • Nyquist sampling theorem: 2 triangles per pixel indistinguishable from thousands
    • Bypasses GPU triangle rasterization hardware for small triangles
    • Uses pixel shaders and ray tracing for better results
  • Lumen: Global illumination system
    • Calculates light bouncing throughout entire scene
    • Multiple scales from kilometers to millimeters
    • Seamless integration across all levels
    • Dynamic lighting that "just works" for artists

Metahuman Technology

  • Vlad Mastilovic leading digital humans effort
  • Capture sphere hardware with dozens of cameras
  • Thousands of hours processing per human capture
  • Dataset goal: encompass entire range of human faces across all cultures/ages
  • Ability to reconstruct arbitrary faces from iPhone photos using learned data
  • Metahuman Creator: Tool for creating faces with adjustable parameters
  • Metahuman Animator: Facial capture from devices like iPhone
  • Translation between actor face shape and character face shape

Fortnite Development and Success

  • Origin: 2011 game jam, build forts by day, defend against zombies by night
  • 7 years development, multiple pivots
  • Art style change from realistic to Pixar-inspired stylized
  • Save the World mode launched early 2017 (moderate success)
  • Battle Royale: 4-week development in war room with 30 people
  • Explosive growth: 300 to thousands of employees, hundreds of millions to billions revenue
  • Technical scaling: 40,000 to 15 million concurrent users
  • Microservices architecture, AWS cloud hosting crucial
  • Cross-platform breakthrough with Sony (initially resistant, eventually opened up)

Verse Programming Language

  • Functional logic language for large-scale simulation programming
  • Zero, one, or multiple values from expressions (failure, success, iteration)
  • Eliminates boolean variables in favor of success/failure patterns
  • SQL-like queries in conditions and loops
  • Type system enables mathematical proofs (Curry-Howard correspondence)
  • Backwards compatibility checking through type system
  • Transactional memory for concurrency
    • Speculative execution of transactions
    • Conflict detection and retry for conflicting operations
    • Moving from programmer problem to language problem
  • Currently shipping in UEFn (Unreal Editor for Fortnite)
  • Will be central to UE6

Metaverse Vision and Standards

  • Current problems: Fragmented ecosystems
    • Different friends systems across platforms
    • Different usernames, controls, purchases don't transfer
    • No social cohesion between games
  • Solutions needed:
    • Federated social systems with @ usernames
    • Unified economies for cosmetic items across games
    • Standards bodies for portable outfits/items
    • Revenue sharing based on engagement patterns
  • Programming language standards crucial (like HTML/JavaScript for web)
  • Need for distributed collaborative content creation

Epic vs Apple Legal Battle

  • Apple's 30% fees economically ruinous, more than most game companies' profits
  • Gatekeeping prevents competition, innovation
  • Epic uniquely positioned to fight: majority users on PC/console, could survive iOS loss
  • Apple's soft power: slowing updates, app store manipulation
  • Other companies terrified of Apple destroying their business
  • Apple's anti-competitive techniques:
    • Blocking web browser improvements on iOS
    • Preventing competing stores from integrating social features
    • Complex barriers even where legally required (EU DMA)
  • Philosophical difference: Apple II was completely open platform vs current closed ecosystem

Epic Games Store and Competition

  • 12% revenue cut vs Steam's 30%
  • Launcher acknowledged as clunky, needs improvement
  • Exclusive deals as competitive necessity when price competition blocked
  • Steam's soft power prevents developers from offering better prices on Epic
  • Google's "Project Hug" paid hundreds of millions to prevent exclusive deals
  • All-in cost of operating stores ~6% of revenue (Google Play data)
  • Goal: Cross-platform social features, unified friend systems

Future of Gaming Industry

  • Trend toward large-scale multiplayer social experiences
  • Metcalfe's Law: Value quadratic with percentage of real-world friends connected
  • Consolidation into fewer, bigger games with massive reinvestment
  • Two main opportunities:
    • Big multiplayer experiences that retain players long-term
    • Efficient single-player games as "vacations" from main games
  • Winner-take-all dynamics emerging
  • Fun remains the core principle - industry loses sight of this at its peril

TwinMind

Get summaries like this for your own meetings

TwinMind takes notes in the background during meetings, lectures, and calls, then writes the summary for you.

Free plan to get started

No bot joins your calls

Phone, desktop & Chrome apps

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome

Your next meeting, summarized like this

TwinMind listens in the background and turns meetings, lectures, and calls into searchable notes you can chat with. Start free.

Available on phone, desktop & Chrome