AI summary by TwinMind
Dan Kokotov: Speech Recognition with AI and Humans | Lex Fridman Podcast #151
Lex Fridman
1h 28m 46s
Introduction and Background
- Conversation with Dan Kokotov, VP of Engineering at Rev.ai
- Rev.ai considered best speech-to-text AI engine by many metrics
- Rev is company doing captioning/transcription by humans and AI
- Host has been using Rev services for couple years, planning to add captions/transcripts to podcast episodes
- Sponsors mentioned: Athletic Greens, Blinkist, Business Wars podcast, Cash App
Science Fiction Discussion
- Greatest sci-fi novel: Dune by Frank Herbert
- Second greatest: Children of Dune, third: God Emperor of Dune
- God Emperor of Dune philosophical concept: Leto becomes tyrant/god emperor merged with sandworm
- Oppresses people to break through civilization stagnation
- Forces humanity to explode outward and colonize new worlds
- Philosophy: need pressure/suffering to make progress, avoid getting too comfortable
Rev Company Origins and Model
- Founded to improve on Upwork freelancer marketplace model
- CEO Jason was early Upwork employee, familiar with pain points
- Original Upwork problems:
- Customers had to browse/choose freelancers without expertise
- Potential for regret with bad choices
- Freelancers spent time on profile optimization vs actual work
- Rev solution: Remove barriers, pick standardizable verticals
- Started with translation, added audio transcription
- Hide complexity from users - simple file upload, get results back
- Originally 48 hours turnaround, made progressively shorter
- Rush processing now available
Service Categories and Pricing
- Focus on work-from-home, computer-based, standardizable tasks
- Translation: Birth certificates, legal documents (fairly well-defined)
- Audio transcription: Per audio minute pricing
- Current pricing: $1.25 per minute (was $1.00), automated version $0.25/minute
- Rev.com for human/end-user services, Rev.ai for ASR API services
Freelancer Demographics and Experience
- Two-sided marketplace: customers and "Revvers" (freelancers)
- Challenge: Balancing supply/demand of work and workers
- Demographics: Mostly US-based (English transcription requirement)
- Wide variety: Work-from-home moms, students, people with social anxiety, career break professionals
- Freelancers often knowledgeable about technical concepts, do research during work
- Many enjoy learning while transcribing (documentaries, lectures)
Transcription Process and Tools
- Freelancers log into workspace, see available audio list
- Tools to filter by preferences: length, subject, region
- Specialized editor built for transcription efficiency
- Start with ASR (speech recognition) draft, then correct
- Quality depends on audio: good audio (like podcasts) vs poor audio (back of auditorium)
- Speed: Real-time transcription very difficult, maybe 2-3x real-time for good audio
Automatic Speech Recognition (ASR) Technology
- ASR = Automatic Speech Recognition, audio-to-text conversion
- Rev believes they have world's best ASR for conversational speech
- Different from voice assistants (Siri) - limited vocabulary vs unstructured conversation
- Current accuracy: ~14% word error rate on test suite
- Word error rate calculation: insertions + deletions + substitutions
- Human accuracy estimated at 2-3% word error rate
- Compete against Google, Amazon, Microsoft
- Beat competitors in internal testing
Data Advantage and Machine Learning
- Biggest factor in ASR success: high-quality labeled data
- Rev's business model = being paid to annotate data ("magical flywheel")
- Have final transcripts, original audio, AND edit sequences from Revvers
- Early stages of utilizing all available data signals
- Edit sequence data potentially valuable for training improvement models
Rev.ai Platform and Vision
- Rev.ai brings ASR technology to broader market
- Evolution: Human-only → Temi (consumer ASR) → Human transcriptionists using ASR → Rev.ai API
- Vision: Enable new applications with highly accurate speech-to-text
- Goal: Make all audio/meetings as accessible as written notes
- Potential for indexed, archived, searchable meeting content
- Impact on remote work and collaboration
Podcast Industry and Transcription
- Current podcast discovery limited to title/description keyword search
- Transcription could enable content-based search within episodes
- Examples: Finding product recommendations, specific quotes
- Reporter use cases for pulling quotes and references
- Host planning to implement transcripts on website for better accessibility
Technical Challenges and Competition
- Competing with major tech companies (Google, YouTube auto-captions)
- Working toward 3% error rate goal (human-level performance)
- RSS feed integration proposals for standard transcript inclusion
- Criticism of exclusive platform deals vs open podcast ecosystem
Management Transition and Leadership Philosophy
- Dan's transition from programmer to VP of Engineering
- Loss of direct creation dopamine, replaced by team success satisfaction
- Management as "programming humans" - understanding different motivations
- People respond differently: some need criticism/push, others need constant encouragement
- "Manage by exception" philosophy - tailor approach to each individual
- Reference: "First Break All the Rules" management book
Book Recommendations
- Dune series (6 main books by Frank Herbert, skip son's additions)
- Brave New World by Aldous Huxley - prescient about genetic/social sorting
- Brazil (movie) - dystopian vision of authoritarian incompetence vs efficient evil dictatorship
- Creative Selection by Ken Cocienda - Apple engineer's perspective on working with Steve Jobs
- Discussion of various dystopian works and their relevance to current technology/society
Philosophy and Meaning of Life
- Meaning found in contributing to humanity as evolving organism
- Creation as core drive - from simple HTML buttons to complex systems
- Parallel between human creativity and religious concept of being created "in God's image"
- Living, creating, raising unique children as fundamental purposes
- Programming as god-like act of bringing non-existent things to life
