AI summary by TwinMind
How To Keep Your Users | Startup School
Y Combinator
29m 22s
Introduction & Background
- David Lee, YC Group Partner, discussing cohort retention as best quantitative method to answer "did we make something people want?"
- Personal background: YC Summer 2009 with Bump (100M+ users, ultimately failed as business), pivoted to photo sharing, acquired by Google, worked on Google Photos (1B+ users)
- Most founders don't understand cohort retention despite widespread analytics tool availability
Core Concept of Cohort Retention
- Definition: tracking what fraction of new users keep using product over time
- Key insight: track individual groups (cohorts) of new users separately vs mixing entire user base
- Three components to define:
- Cohorts: how to group new users (typically by acquisition time - weekly/monthly)
- Action: what qualifies as "active user" in subsequent periods
- Time period: granularity for measuring retention (daily/weekly/monthly/quarterly)
Defining Cohorts
- Most common: group by first use date (weekly cohorts, monthly cohorts)
- Advanced segmentation: by country, acquisition channel, device, customer characteristics
- Start simple with time-based cohorts, then add dimensions
Selecting Actions for Active Users
- Avoid shallow metrics like "opened app" or "visited site"
- Choose actions correlated with actual value delivery
- Examples:
- Instagram: viewed 3+ posts (filters out accidental opens)
- Uber: completed a ride
- Google Photos: viewed photo full screen
- Best action = strongly correlated with user getting real value
- Avoid payment-only metrics (users stop using before stopping payment)
- Rubric: imagine watching customer use product - what makes you think "this is a good user"?
Time Period Selection
- Match intended usage frequency of product:
- Social/entertainment apps (Instagram, TikTok, YouTube): daily
- Utility products (Google Photos, Uber): weekly
- Travel apps (Airbnb): quarterly/semi-annually
- Avoid unconscious bias toward longer periods to inflate numbers
Triangle Chart Methodology
- Rows = cohorts by acquisition month
- Columns = subsequent months
- Each cell = count of users from that cohort active in that month
- Diagonal line = single calendar month showing cohort composition
- Convert to percentages by dividing by initial cohort size
- Visualize as line graph with each cohort as separate line
Critical Success Metric: Flat Curves
- Only thing that matters: whether cohort curves get flat, not absolute retention percentage
- Flat curves = sustainable user accumulation over time
- Non-flat curves = perpetual treadmill of acquiring new users while losing old ones
- Example: Google Photos had 20-40% weekly retention that flattened (80% churn but 20% stayed forever)
- Gave confidence to reach 20% of all humans (achieved 1B+ users in 4 years)
Common Mistakes That Fool Founders
1. Time Period Too Large
- Longer periods artificially improve retention numbers
- Bump example: weekly → monthly → quarterly cohorts to look better for investors
- Unconscious bias pushes toward wider cohorts
2. Action Too Easy/Shallow
- Counting notification clicks, passive exposure as "active"
- Google+ example: red notification bell views counted as active usage
- Gaming metrics through notifications leads to false confidence
3. Payment-Only Metrics
- Users stop using before stopping payment (Netflix example)
- Better: combine payment + actual product usage
4. Single Point vs Curve Shape Focus
- Founders cite "80% week-over-week retention" without context
- Must examine entire curve shape and trend, not isolated data points
5. Analytics Tool Misunderstanding
- Built-in cohort tools may not measure what founders think
- Rolling retention vs true cohort separation
- "Returned by date" vs "returned during period"
- Recommendation: build own cohort analysis first, then compare to tools
Improving Cohort Retention
1. Product Improvements
- New use cases, speed optimization, simplified flows
- Visible in newer cohorts performing better than older ones
- Example chart showing progression from poor early cohorts to flatter recent ones
2. Better User Acquisition
- Target right customer type for product
- Google Photos example: young users had poor retention (don't have many memories to reminisce)
- Mismatch between acquisition strategy and product purpose
3. Cohort Segmentation Analysis
- Slice cohorts by dimensions (country, company size, device)
- Identify high-performing segments vs poor ones
- Focus improvement efforts on insights from segmentation
4. First User Experience/Onboarding
- Often overlooked in favor of core product features
- Critical for B2B tools: teach usage, integrate into workflows
- Cheapest way to improve cohort performance
5. Network Effects
- Products where each new user improves experience for existing users
- Social networks, sharing platforms, messaging apps
- Should see cohort improvement over time as network density increases
Holy Grail: Upward Trending Curves
- Best case: cohort curves that get flat then increase over time
- Users stick around AND use product more frequently
- Indicates both retention and engagement growth
- Sign of truly exceptional product-market fit
Layer Cake Visualization
- Transform triangle chart to show absolute calendar time
- Each month's total users broken down by originating cohort
- Growing total with thick layers from old cohorts = beautiful startup chart
- Indicates sustainable growth from accumulated user base
Monitoring Frequency
- Don't check multiple times daily or even daily
- Refresh weekly or bi-weekly
- Early detection when things go wrong
- More likely: retention not good from beginning, need improvement
Conclusion & Philosophy
- Cohort retention = quantitative validation of qualitative user feedback
- Doesn't tell you what to change, but tells you if you're on right track
- Non-flat curves = certainty that you need to talk to customers and make something they want
- Goal: flat cohort retention curve for sustainable business growth
