17 KiB
17 KiB
Product Metrics Frameworks
A comprehensive guide to choosing and implementing product metrics frameworks for measuring success.
Table of Contents
- Overview
- AARRR (Pirate Metrics)
- HEART Framework
- North Star Metric
- OKRs (Objectives & Key Results)
- Product-Market Fit Metrics
- Engagement Metrics
- Choosing the Right Framework
Overview
Why Metrics Matter
Metrics help you:
- Measure feature success objectively
- Make data-driven decisions
- Align teams around shared goals
- Identify what's working and what's not
- Communicate impact to stakeholders
Types of Metrics
Leading Indicators: Predict future outcomes
- Example: Free trial sign-ups → Future revenue
Lagging Indicators: Measure past results
- Example: Monthly revenue, churn rate
Actionable vs. Vanity Metrics
- Actionable: Can influence through product changes (e.g., conversion rate)
- Vanity: Looks good but doesn't drive decisions (e.g., total registered users)
AARRR (Pirate Metrics)
Created by Dave McClure, this framework focuses on the customer lifecycle.
The Five Stages
Acquisition → Activation → Retention → Revenue → Referral
1. Acquisition
What: How do users find you?
Key Metrics:
- Website traffic
- App store impressions
- Click-through rate (CTR) from ads
- Cost per acquisition (CPA)
- Traffic sources (organic, paid, referral)
Example Targets:
- 10,000 monthly website visitors
- CAC < $50
- 5% CTR on paid ads
Questions to Answer:
- Which channels drive the most users?
- What's our cost per channel?
- Which campaigns convert best?
2. Activation
What: Do users have a great first experience?
Key Metrics:
- Sign-up completion rate
- Time to "aha moment"
- Percentage reaching key milestone
- Onboarding completion rate
- Feature adoption in first session
Example Targets:
- 60% sign-up completion
- 40% reach "aha moment" in first session
- 80% complete onboarding
Questions to Answer:
- What does a great first experience look like?
- Where do users drop off in onboarding?
- How quickly do users find value?
Example "Aha Moments":
- Slack: Send your first message
- Dropbox: Upload your first file
- Airbnb: Complete your first booking
3. Retention
What: Do users come back?
Key Metrics:
- Daily/Weekly/Monthly Active Users (DAU/WAU/MAU)
- Retention curves (Day 1, Day 7, Day 30)
- Churn rate
- Session frequency
- Feature usage over time
Example Targets:
- 40% Day 7 retention
- 25% Day 30 retention
- < 5% monthly churn
Cohort Analysis:
| Cohort | Week 1 | Week 2 | Week 3 | Week 4 |
|-----------|--------|--------|--------|--------|
| Jan 2024 | 100% | 45% | 30% | 25% |
| Feb 2024 | 100% | 50% | 35% | 28% |
Questions to Answer:
- What makes users come back?
- When do users churn?
- How can we re-engage inactive users?
4. Revenue
What: How do you monetize?
Key Metrics:
- Monthly Recurring Revenue (MRR)
- Average Revenue Per User (ARPU)
- Customer Lifetime Value (LTV)
- Conversion to paid rate
- Upsell/cross-sell rate
Example Targets:
- 5% free-to-paid conversion
- $50 ARPU
- LTV:CAC ratio > 3:1
Formulas:
LTV = ARPU × Average Customer Lifespan
LTV:CAC = Lifetime Value ÷ Customer Acquisition Cost
Churn Rate = Customers Lost ÷ Total Customers × 100
Questions to Answer:
- Which features drive conversions?
- What's our payback period?
- How can we increase ARPU?
5. Referral
What: Do users tell others?
Key Metrics:
- Viral coefficient (K-factor)
- Net Promoter Score (NPS)
- Referral rate
- Social shares
- Word-of-mouth attribution
Example Targets:
- 15% of users refer others
- NPS > 50
- K-factor > 1 (viral growth)
Formulas:
K-factor = (Number of Invites per User) × (Conversion Rate of Invites)
NPS = % Promoters - % Detractors
Questions to Answer:
- Why do users refer others?
- How can we incentivize referrals?
- What makes us shareable?
AARRR Example: SaaS Product
| Stage | Metric | Current | Target | Actions |
|---|---|---|---|---|
| Acquisition | Monthly visitors | 50,000 | 75,000 | SEO, content marketing |
| Activation | Trial sign-ups | 5% | 8% | Improve landing page |
| Retention | Day 30 retention | 20% | 30% | Onboarding improvements |
| Revenue | Free-to-paid conversion | 3% | 5% | Pricing page redesign |
| Referral | Users who refer | 8% | 15% | Referral program launch |
HEART Framework
Created by Google, focuses on user experience quality.
The Five Dimensions
HEART = Happiness + Engagement + Adoption + Retention + Task Success
1. Happiness
What: User satisfaction and perception
Metrics:
- Net Promoter Score (NPS)
- Customer Satisfaction (CSAT)
- User ratings/reviews
- Support ticket sentiment
- User feedback scores
Measurement Methods:
- Surveys (post-interaction, periodic)
- App store ratings
- In-app feedback forms
- Social media sentiment
Example:
Feature: New checkout flow
Happiness Metric: CSAT score
Target: > 4.5/5 average rating
Measurement: Post-purchase survey
2. Engagement
What: Level of user involvement
Metrics:
- Session duration
- Pages/screens per session
- Feature usage frequency
- Time spent in app
- Actions per session
Example:
Feature: News feed
Engagement Metric: Daily sessions per user
Current: 1.2 sessions/day
Target: 2.0 sessions/day
3. Adoption
What: New users or feature uptake
Metrics:
- New user sign-ups
- Feature adoption rate
- Time to first use
- Percentage of users trying new feature
Example:
Feature: Dark mode
Adoption Metric: % of users enabling dark mode
Target: 40% within 30 days of launch
4. Retention
What: Users returning over time
Metrics:
- DAU/WAU/MAU
- Retention curves
- Churn rate
- Repeat usage rate
Example:
Feature: Collaboration tools
Retention Metric: Week-over-week active teams
Target: 70% of teams active weekly
5. Task Success
What: Can users accomplish their goals?
Metrics:
- Task completion rate
- Error rate
- Time to complete task
- Search success rate
Example:
Feature: File upload
Task Success Metric: Upload completion rate
Current: 85%
Target: 95%
Error analysis: Large file timeouts
HEART Framework Template
| Dimension | Goals | Signals | Metrics |
|---|---|---|---|
| Happiness | Users love the feature | Positive feedback | NPS > 40 |
| Engagement | Users interact frequently | Daily active usage | 60% DAU/MAU |
| Adoption | Most users try it | Feature activation | 70% adoption |
| Retention | Users keep coming back | Weekly return rate | 50% W1 retention |
| Task Success | Users complete goals | Low error rate | 95% success rate |
North Star Metric
A single metric that best captures the core value you deliver to customers.
Characteristics of a Good North Star Metric
- Reflects value delivery to customers
- Measures progress toward your vision
- Actionable by the team
- Leading indicator of revenue
- Understandable by everyone
Examples by Company
| Company | North Star Metric | Why |
|---|---|---|
| Airbnb | Nights booked | Core value: successful stays |
| Spotify | Time spent listening | Core value: music enjoyment |
| Slack | Messages sent by teams | Core value: communication |
| Daily Active Users | Core value: social connection | |
| Netflix | Hours watched | Core value: entertainment |
| Uber | Rides completed | Core value: transportation |
| Medium | Total time reading | Core value: quality content |
Finding Your North Star Metric
Step 1: Define your value proposition
- What core value do you deliver?
- What's the "aha moment" for users?
Step 2: Identify the metric
- What measurement best captures that value?
- Is it a leading indicator of business success?
Step 3: Validate the metric
- Does it correlate with revenue?
- Can teams influence it?
- Is it understandable?
Step 4: Set targets and track
- What's the current baseline?
- What's the target growth rate?
- How will you measure progress?
North Star Metric Tree
Break down your North Star into contributing metrics:
North Star: Weekly Active Users
├── New User Acquisition
│ ├── Sign-ups
│ └── Onboarding completion
├── Activation
│ └── Users reaching "aha moment"
└── Retention
├── Week 1 retention
└── Week 4 retention
OKRs (Objectives & Key Results)
Goal-setting framework popularized by Google.
Structure
Objective: Qualitative, inspirational goal Key Results: Quantitative, measurable outcomes (3-5 per objective)
Writing Good OKRs
Objective Characteristics:
- Inspirational and motivating
- Qualitative
- Time-bound (quarterly or annual)
- Aligned with company strategy
Key Result Characteristics:
- Quantitative and measurable
- Specific with clear targets
- Ambitious but achievable
- 3-5 per objective
Examples
Example 1: Growth OKR
Objective: Become the go-to platform for small business invoicing
Key Results:
- Increase monthly active businesses from 10,000 to 25,000
- Achieve 40% month-over-month retention
- Reach NPS of 50+
- Generate $500K MRR
Example 2: Product Quality OKR
Objective: Deliver a world-class mobile experience
Key Results:
- Reduce app crash rate from 2.5% to <0.5%
- Achieve 4.5+ star rating on both app stores
- Improve app load time to <2 seconds (p95)
- Increase mobile DAU/MAU ratio from 30% to 45%
Example 3: Feature Launch OKR
Objective: Successfully launch team collaboration features
Key Results:
- 60% of active users try collaboration features within 30 days
- 25% of users become weekly active collaborators
- Collaboration features drive 15% increase in paid conversions
- Achieve CSAT score of 4.2/5 for collaboration features
OKR Template for PRDs
## OKRs
### Objective: [Inspirational goal]
**Key Results:**
1. [Metric 1]: Increase/decrease [current] to [target] by [date]
2. [Metric 2]: Achieve [target value] for [metric]
3. [Metric 3]: [Specific measurable outcome]
**Tracking:**
- Current status: [Progress report]
- Dashboard: [Link to metrics dashboard]
- Review cadence: [Weekly/bi-weekly]
Product-Market Fit Metrics
Measuring whether you've achieved product-market fit.
Sean Ellis Test
Survey question: "How would you feel if you could no longer use [product]?"
- Very disappointed
- Somewhat disappointed
- Not disappointed
PMF Threshold: 40%+ answer "Very disappointed"
Other PMF Indicators
Qualitative Signals:
- Users voluntarily refer others
- Organic growth without marketing
- High engagement and retention
- Users find creative use cases
- Positive unsolicited feedback
Quantitative Metrics:
- Retention: 40%+ month 1 retention
- NPS: Score > 50
- Growth: 10%+ month-over-month organic growth
- Engagement: High DAU/MAU ratio (>40%)
- LTV:CAC: Ratio > 3:1
Engagement Metrics
Deep dive into measuring user engagement.
DAU/WAU/MAU
Definitions:
- DAU: Daily Active Users (unique users in a day)
- WAU: Weekly Active Users (unique users in a week)
- MAU: Monthly Active Users (unique users in a month)
Ratios:
- DAU/MAU: Stickiness (how many monthly users come daily)
- DAU/WAU: Daily engagement intensity
Benchmarks:
- Excellent: DAU/MAU > 50% (e.g., messaging apps)
- Good: DAU/MAU = 20-50% (e.g., social media)
- Average: DAU/MAU = 10-20% (e.g., utilities)
Session Metrics
Key Measurements:
- Session duration: Time spent per session
- Session frequency: Sessions per user per day/week
- Session depth: Actions/pages per session
Example Targets:
- Session duration: > 5 minutes
- Session frequency: 2+ sessions/day
- Session depth: > 8 page views
Feature Engagement
Metrics:
- Adoption rate: % of users who try the feature
- Active usage: % of users actively using regularly
- Depth of use: Actions per user within feature
Example:
Feature: Document collaboration
- Adoption: 50% of users have collaborated at least once
- Active usage: 30% collaborate weekly
- Depth: Average 12 collaborative edits per week
Choosing the Right Framework
Decision Matrix
| Framework | Best For | Time Horizon | Complexity |
|---|---|---|---|
| AARRR | Growth-focused products, startups | Ongoing | Medium |
| HEART | UX quality, feature launches | Per feature | Low-Medium |
| North Star | Company alignment, focus | Ongoing | Low |
| OKRs | Goal setting, team alignment | Quarterly | Medium-High |
By Product Stage
Early Stage (Pre-PMF):
- Focus: Product-Market Fit metrics
- Framework: AARRR (Activation & Retention focus)
- North Star: Early engagement metric
Growth Stage (Post-PMF):
- Focus: Scaling user acquisition
- Framework: Full AARRR funnel
- North Star: Growth-oriented metric
Mature Stage:
- Focus: Optimization and expansion
- Framework: HEART for features, OKRs for goals
- North Star: Revenue or engagement metric
By Product Type
Consumer Apps:
- AARRR for growth funnel
- DAU/MAU for engagement
- Viral coefficient for referral
B2B SaaS:
- ARR/MRR for revenue
- Churn rate for retention
- Expansion revenue for growth
Marketplace:
- GMV (Gross Merchandise Value)
- Take rate (% of transaction)
- Liquidity (supply/demand balance)
Content Platforms:
- Time spent on platform
- Content creation rate
- Content consumption rate
Metrics Anti-Patterns
Common Mistakes
1. Too Many Metrics
- Problem: Tracking everything = focusing on nothing
- Solution: Choose 3-5 key metrics per initiative
2. Vanity Metrics
- Problem: Total users looks good but doesn't inform decisions
- Solution: Focus on active users, engagement, retention
3. Lagging Only
- Problem: Only tracking revenue = rear-view mirror
- Solution: Balance with leading indicators (activation, engagement)
4. No Targets
- Problem: Tracking without goals
- Solution: Set specific, time-bound targets
5. Not Segmenting
- Problem: Average metrics hide important patterns
- Solution: Segment by user type, cohort, feature usage
Metrics Template for PRDs
## Success Metrics
### North Star Metric
**Metric:** [Your single most important metric]
**Current:** [Baseline value]
**Target:** [Goal value by launch + X months]
**Why:** [Why this metric matters]
### Supporting Metrics
#### Acquisition
- **Metric 1:** [Name] - Current: [X], Target: [Y]
- **Metric 2:** [Name] - Current: [X], Target: [Y]
#### Activation
- **Metric 1:** [Name] - Current: [X], Target: [Y]
- **Metric 2:** [Name] - Current: [X], Target: [Y]
#### Retention
- **Metric 1:** [Name] - Current: [X], Target: [Y]
- **Metric 2:** [Name] - Current: [X], Target: [Y]
#### Revenue (if applicable)
- **Metric 1:** [Name] - Current: [X], Target: [Y]
### Counter-Metrics
[Metrics to ensure you're not sacrificing other areas]
- Example: Ensure support tickets don't increase > 10%
### Measurement Plan
- **Dashboard:** [Link]
- **Review Cadence:** [Weekly/bi-weekly]
- **Owner:** [Name]
Resources & Tools
Analytics Platforms
- Amplitude: Product analytics, retention analysis
- Mixpanel: Event tracking, funnel analysis
- Google Analytics: Web analytics
- Heap: Auto-capture analytics
Survey Tools
- Delighted: NPS surveys
- SurveyMonkey: Custom surveys
- Typeform: Engaging survey forms
Dashboard Tools
- Tableau: Data visualization
- Looker: Business intelligence
- Datadog: Infrastructure metrics
- Metabase: Open-source BI
Summary
Key Takeaways:
- Choose frameworks that match your product stage and goals
- Balance leading and lagging indicators
- Set specific targets with timelines
- Track counter-metrics to avoid unintended consequences
- Review regularly and iterate on what you measure
- Keep it simple - 3-5 key metrics per initiative
- Align metrics with business objectives
- Make metrics actionable - can the team influence them?
Remember: The best metric is one that drives the right behavior and aligns your team around what matters most to users and the business.