zhongwei/gh-jamesrochabrun-skills
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
gh-jamesrochabrun-skills/skills/technical-launch-planner/references/metrics_frameworks.md
Zhongwei Li f28999f19c Initial commit
2025-11-29 18:48:55 +08:00

11 KiB
Raw Permalink Blame History

Developer Product Metrics

Comprehensive guide to measuring success for technical products, developer tools, and APIs.

Why Developer Metrics Are Different

Developer products have unique characteristics:

  • High technical barriers to adoption
  • Longer evaluation periods
  • Community-driven growth
  • Usage-based pricing models
  • Quality over quantity (one great developer > 100 casual users)

Traditional B2B SaaS metrics don't always apply directly.

The Developer Funnel

Awareness → Interest → Evaluation → Activation → Engagement → Retention → Monetization

Each stage has specific metrics.

1. Awareness Metrics

Top-of-Funnel

Website Traffic:

  • Documentation page views
  • Landing page visits
  • Blog traffic
  • GitHub repository views

Search & Discovery:

  • Organic search rankings (for key terms)
  • GitHub stars
  • Stack Overflow mentions
  • Social media mentions

Community Presence:

  • Discord/Slack members
  • Reddit subscribers
  • Newsletter subscribers
  • Conference attendance

Targets (vary by company):

  • 50K monthly docs views (early stage)
  • 500K monthly docs views (growth stage)
  • 1K+ GitHub stars (open source component)

2. Interest Metrics

Consideration Stage

Engagement:

  • Time on documentation
  • Pages per session
  • Video views (tutorials)
  • GitHub README views

Content Consumption:

  • Blog post reads
  • Tutorial completion rate
  • Demo video watch time
  • Webinar registrations

Social Proof:

  • Case study views
  • Customer testimonials read
  • Comparison page visits

Targets:

  • 5+ minute average session duration
  • 4+ pages per session
  • 50%+ video completion rate

3. Evaluation Metrics

Trial/Sandbox Stage

Sign-Up:

  • Developer sign-ups
  • API key requests
  • Sandbox activations
  • Free tier activations

Time to Value:

  • Time to first API call (target: < 10 minutes)
  • Time to "Hello World" (target: < 15 minutes)
  • Time to integration (target: < 1 hour)

Documentation Engagement:

  • Getting started guide views
  • Code sample copies
  • SDK downloads
  • Postman collection imports

Targets:

  • 1K+ sign-ups per month (early stage)
  • 60% make first API call within 24 hours
  • 10 minutes median time to first call

4. Activation Metrics

First Value Realized

Critical Activation Events:

  • First successful API call
  • First integration deployed
  • First production request
  • SDK installed and used
  • Sandbox → production migration

Activation Rate:

Activation Rate = (Users who complete activation event) / (Total sign-ups)

Depth of Activation:

  • Features explored
  • Endpoints called
  • SDKs used
  • Integrations enabled

Targets:

  • 50-70% activation rate (first API call)
  • 30-40% activation rate (production deployment)
  • 80%+ complete getting started guide

5. Engagement Metrics

Active Usage

Daily/Weekly/Monthly Active Developers (DAD/WAD/MAD):

DAD = Unique developers making API calls daily
WAD = Unique developers active weekly
MAD = Unique developers active monthly

Stickiness:

Stickiness = DAD / MAD
  • Target: > 20% (good)
  • Target: > 40% (excellent)

API Usage:

  • Total API calls per day/week/month
  • API calls per developer
  • Endpoints used per developer
  • Error rate (target: < 1%)

Feature Adoption:

  • % of developers using key features
  • Time to feature adoption
  • Feature depth (how many features per user)

Targets:

  • 40%+ stickiness (DAD/MAD)
  • 1K+ API calls per active developer per month
  • < 1% error rate
  • 3+ features adopted per developer

6. Retention Metrics

Developer Retention

Cohort Retention:

Day 1 Retention = Developers active on Day 1 / Total sign-ups
Day 7 Retention = Developers active on Day 7 / Total sign-ups
Day 30 Retention = Developers active on Day 30 / Total sign-ups

Typical Developer Product Retention:

  • Day 1: 60-70%
  • Day 7: 30-50%
  • Day 30: 20-40%
  • Day 90: 15-30%

Why Developer Retention Is Lower:

  • Evaluation period (many are just testing)
  • Project-based usage (finish project, stop using)
  • This is normal and expected

Churn Rate:

Monthly Churn = Developers who stopped using / Active developers at month start

Resurrection Rate:

Resurrection = Churned developers who return / Total churned developers

Targets:

  • < 5% monthly churn (paid users)
  • 40%+ Day 7 retention
  • 25%+ Day 30 retention

7. Monetization Metrics

Revenue Metrics

Conversion Metrics:

Free → Paid Conversion Rate = Paid users / Total active users
  • Target: 3-10% (varies widely by product)

Revenue Metrics:

  • MRR (Monthly Recurring Revenue)
  • ARR (Annual Recurring Revenue)
  • ARPU (Average Revenue Per User)
  • Net Revenue Retention (NRR)

Usage-Based Pricing Metrics:

  • Average API calls per paid user
  • Tier distribution (how many in each pricing tier)
  • Upgrade rate (free → paid, basic → pro)
  • Expansion revenue (existing customers spending more)

Targets:

  • 5%+ free-to-paid conversion
  • 110%+ Net Revenue Retention
  • $50-$500 ARPU (varies by product)

Developer-Specific Metrics

Code Quality Metrics

SDK Quality:

  • Downloads per month
  • GitHub stars
  • Issues opened vs. closed
  • PR acceptance rate
  • Time to resolve issues

Documentation Quality:

  • Search success rate (did they find what they needed?)
  • Time on page (too short = unclear, too long = can't find)
  • Bounce rate on docs
  • Feedback (thumbs up/down on docs pages)

Targets:

  • 90%+ search success rate
  • < 40% bounce rate on docs
  • 80%+ positive feedback on docs

Developer Experience Metrics

Time-Based:

  • Time to first API call
  • Time to production
  • Time to integrate
  • Time to debug

Friction Points:

  • Authentication failures
  • API errors
  • SDK install issues
  • Documentation gaps

Support Metrics:

  • Support tickets per MAD
  • Time to first response
  • Time to resolution
  • Community forum response time

Targets:

  • < 5 minutes to first API call
  • < 1 support ticket per 100 MAD
  • < 4 hours first response time
  • 90%+ questions answered by community

Developer Satisfaction

Net Promoter Score (NPS)

Survey question: "How likely are you to recommend [product] to other developers?"

Scale: 0-10

Calculation:

NPS = % Promoters (9-10) - % Detractors (0-6)

Developer Product Benchmarks:

  • Excellent: NPS > 50
  • Good: NPS 30-50
  • Needs Work: NPS < 30

Developer Sentiment

Qualitative Indicators:

  • Social media sentiment
  • Community forum tone
  • GitHub issue sentiment
  • Review site ratings (G2, Capterra)
  • Stack Overflow sentiment

Quantitative Tracking:

  • Positive vs. negative mentions
  • Sentiment score (automated analysis)
  • Review ratings (1-5 stars)

Launch-Specific Metrics

Launch Day Metrics

Day 1:

  • Sign-ups / API keys
  • First API calls
  • Documentation views
  • Blog post views
  • Social media impressions
  • Email open rate
  • Email click rate

Targets (Tier 1 launch):

  • 5K+ sign-ups
  • 50%+ activation rate (first call)
  • 100K+ docs views
  • 50K+ blog views

Week 1 Metrics

  • Total sign-ups
  • Day 7 retention rate
  • Active developers
  • API calls made
  • Support tickets
  • Community questions
  • Social mentions

Targets (Tier 1):

  • 10K+ total sign-ups
  • 40%+ Day 7 retention
  • 5K+ active developers
  • 1M+ API calls
  • < 50 support tickets

Month 1 Metrics

  • Monthly Active Developers (MAD)
  • Free → paid conversion
  • NPS score
  • Documentation coverage (no major gaps)
  • Community health
  • Feature adoption

Targets (Tier 1):

  • 25K+ MAD
  • 3-5% paid conversion
  • NPS > 40
  • 80%+ positive doc feedback

Metrics Dashboard Template

Executive Dashboard

Adoption:

  • Total Developers: [X]
  • MAD: [X]
  • Growth Rate: [X%]

Engagement:

  • DAD/MAD: [X%]
  • API Calls/Day: [X]
  • Error Rate: [X%]

Retention:

  • Day 7: [X%]
  • Day 30: [X%]
  • Churn: [X%]

Revenue:

  • MRR: $[X]
  • ARPU: $[X]
  • NRR: [X%]

Quality:

  • NPS: [X]
  • Support Tickets/MAD: [X]

Product Team Dashboard

This Week:

  • New Developers: [X]
  • Activation Rate: [X%]
  • Features Adopted: [X]
  • Top API Endpoints: [List]

Trends:

  • MAD (7-day trend): [Graph]
  • API Calls (7-day): [Graph]
  • Error Rate (7-day): [Graph]

Health:

  • Documentation Gaps: [Count]
  • Open Issues: [Count]
  • P0 Bugs: [Count]

Metric Collection

Where to Track

Product Analytics:

  • Amplitude
  • Mixpanel
  • Heap
  • PostHog

API Analytics:

  • Moesif
  • API metrics (custom)
  • CloudWatch / Datadog

Documentation Analytics:

  • Google Analytics
  • Readme.io analytics
  • GitBook analytics

Developer Feedback:

  • Intercom
  • Zendesk
  • Community forum analytics
  • Survey tools (Delighted, SurveyMonkey)

Setting Targets

Early Stage (0-1 year)

Focus on activation and engagement:

  • Sign-ups: 1K-10K/month
  • Activation: 50%+
  • MAD: 500-5K
  • Day 7 Retention: 30%+

Growth Stage (1-3 years)

Focus on scale and retention:

  • Sign-ups: 10K-50K/month
  • MAD: 10K-100K
  • Day 30 Retention: 25%+
  • Free→Paid: 5%+

Mature Stage (3+ years)

Focus on efficiency and expansion:

  • MAD: 100K+
  • NRR: 110%+
  • ARPU: Increasing
  • CAC Payback: < 12 months

Common Pitfalls

Vanity Metrics

Avoid:

  • Total registered users (most are inactive)
  • Total API calls (could be from one user)
  • GitHub stars alone (may not use product)

Focus on:

  • Active users (making API calls)
  • Retained users (coming back)
  • Engaged users (using multiple features)

Wrong Benchmarks

Don't compare developer product metrics to:

  • B2C social apps (much higher DAU/MAU)
  • Enterprise SaaS (lower volume, higher ACV)
  • E-commerce (transactional, not sustained use)

Use developer product benchmarks instead.

Summary: Key Metrics to Track

Must Track:

  1. Monthly Active Developers (MAD)
  2. Activation Rate (first API call)
  3. Day 7 & Day 30 Retention
  4. Stickiness (DAD/MAD)
  5. API Error Rate
  6. NPS

Should Track: 7. Free → Paid Conversion 8. Time to First API Call 9. Documentation Effectiveness 10. Support Ticket Volume

Nice to Have: 11. GitHub Stars/Activity 12. Community Engagement 13. Social Sentiment 14. Feature Adoption Depth

Start with the must-track metrics, then expand.

Developer Metric Formulas

Quick reference:

Activation Rate = Activated Users / Sign-ups
Stickiness = DAD / MAD
Churn Rate = Users Lost / Total Users
NRR = (MRR + Expansion - Churn) / Starting MRR
LTV = ARPU / Churn Rate
CAC Payback = CAC / (ARPU * Gross Margin)

Remember: Metrics should drive action, not just reporting. If a metric doesn't change behavior, don't track it.

Reference in New Issue View Git Blame Copy Permalink