gh-lyndonkl-claude/skills/metrics-tree/resources/methodology.md

Metrics Tree Methodology

When to use this methodology: You've used template.md and need advanced techniques for:

• Multi-sided marketplaces or platforms
• Complex metric interdependencies and feedback loops
• Counter-metrics and guardrail systems
• Network effects and viral growth
• Preventing metric gaming
• Seasonal adjustment and cohort aging effects
• Portfolio approach for different business stages

If your metrics tree is straightforward: Use template.md directly. This methodology is for complex metric systems.

---

Table of Contents

1. Multi-Sided Marketplace Metrics
2. Counter-Metrics & Guardrails
3. Network Effects & Viral Loops
4. Preventing Metric Gaming
5. Advanced Leading Indicators
6. Metric Interdependencies
7. Business Stage Metrics

---

1. Multi-Sided Marketplace Metrics

Challenge

Marketplaces have supply-side and demand-side that must be balanced. Optimizing one side can hurt the other.

Solution: Dual Tree Approach

Step 1: Identify constraint

• Supply-constrained: More demand than supply → Focus on supply-side metrics
• Demand-constrained: More supply than demand → Focus on demand-side metrics
• Balanced: Need both → Monitor ratio/balance metrics

Step 2: Create separate trees

Supply-Side Tree:

North Star: Active Suppliers (providing inventory)
├─ New supplier activation
├─ Retained suppliers (ongoing activity)
└─ Supplier quality/performance

Demand-Side Tree:

North Star: Successful Transactions
├─ New customer acquisition
├─ Repeat customer rate
└─ Customer satisfaction

Step 3: Define balance metrics

• Liquidity ratio: Supply utilization rate (% of inventory sold)
• Match rate: % of searches resulting in transaction
• Wait time: Time from demand signal to fulfillment

Example (Uber):

• Supply NS: Active drivers with >10 hours/week
• Demand NS: Completed rides
• Balance metric: Average wait time <5 minutes, driver utilization >60%

Multi-Sided Decomposition Template

Marketplace GMV = (Supply × Utilization) × (Demand × Conversion) × Average Transaction

Where:
- Supply: Available inventory/capacity
- Utilization: % of supply that gets used
- Demand: Potential buyers/searches
- Conversion: % of demand that transacts
- Average Transaction: $ per transaction

---

2. Counter-Metrics & Guardrails

Problem

Optimizing primary metrics can create negative externalities (quality drops, trust declines, user experience suffers).

Solution: Balanced Scorecard with Guardrails

Framework:

1. Primary metric (North Star): What you're optimizing
2. Counter-metrics: What could be harmed
3. Guardrail thresholds: Minimum acceptable levels

Example (Content Platform):

Primary: Content Views (maximize)

Counter-metrics with guardrails:
- Content quality score: Must stay ≥7/10 (current: 7.8)
- User satisfaction (NPS): Must stay ≥40 (current: 52)
- Creator retention: Must stay ≥70% (current: 75%)
- Time to harmful content takedown: Must be ≤2 hours (current: 1.5h)

Rule: If any guardrail is breached, pause optimization of primary metric

Common Counter-Metric Patterns

Primary Metric	Potential Harm	Counter-Metric
Pageviews	Clickbait, low quality	Time on page, bounce rate
Engagement time	Addictive dark patterns	User-reported wellbeing, voluntary sessions
Viral growth	Spam	Unsubscribe rate, report rate
Conversion rate	Aggressive upsells	Customer satisfaction, refund rate
Speed to market	Technical debt	Bug rate, system reliability

How to Set Guardrails

1. Historical baseline: Look at metric over past 6-12 months, set floor at 10th percentile
2. Competitive benchmark: Set floor at industry average
3. User feedback: Survey users on acceptable minimum
4. Regulatory: Compliance thresholds

---

3. Network Effects & Viral Loops

Measuring Network Effects

Network effect: Product value increases as more users join.

Metrics to track:

• Network density: Connections per user (higher = stronger network)
• Cross-side interactions: User A's action benefits User B
• Viral coefficient (K): New users generated per existing user
  • K > 1: Exponential growth (viral)
  • K < 1: Sub-viral (need paid acquisition)

Decomposition:

New Users = Existing Users × Invitation Rate × Invitation Acceptance Rate

Example:
100,000 users × 2 invites/user × 50% accept = 100,000 new users (K=1.0)

Viral Loop Metrics Tree

North Star: Viral Coefficient (K)

Decomposition:

K = (Invitations Sent / User) × (Acceptance Rate) × (Activation Rate)

Input metrics:
├─ Invitations per user
│  ├─ % users who send ≥1 invite
│  ├─ Average invites per sender
│  └─ Invitation prompts shown
├─ Invite acceptance rate
│  ├─ Invite message quality
│  ├─ Social proof (sender credibility)
│  └─ Landing page conversion
└─ New user activation rate
   ├─ Onboarding completion
   ├─ Value realization time
   └─ Early engagement actions

Network Density Metrics

Measure connectedness:

• Average connections per user
• % of users with ≥N connections
• Clustering coefficient (friends-of-friends)
• Active daily/weekly connections

Threshold effects:

• Users with 7+ friends have 10x retention (identify critical mass)
• Teams with 10+ members have 5x engagement (team size threshold)

---

4. Preventing Metric Gaming

Problem

Teams optimize for the letter of the metric, not the spirit, creating perverse incentives.

Gaming Detection Framework

Step 1: Anticipate gaming For each metric, ask: "How could someone game this?"

Example metric: Time on site

• Gaming: Auto-play videos, infinite scroll, fake engagement
• Intent: User finds value, willingly spends time

Step 2: Add quality signals Distinguish genuine value from gaming:

Time on site (primary)
+ Quality signals (guards against gaming):
  - Active engagement (clicks, scrolls, interactions) vs passive
  - Return visits (indicates genuine interest)
  - Completion rate (finished content vs bounced)
  - User satisfaction rating
  - Organic shares (not prompted)

Step 3: Test for gaming

• Spot check: Sample user sessions, review for patterns
• Anomaly detection: Flag outliers (10x normal behavior)
• User feedback: "Was this session valuable to you?"

Gaming Prevention Patterns

Pattern 1: Combination metrics Don't measure single metric; require multiple signals:

❌ Single: Pageviews
✓ Combined: Pageviews + Time on page >30s + Low bounce rate

Pattern 2: User-reported value Add subjective quality check:

Primary: Feature adoption rate
+ Counter: "Did this feature help you?" (must be >80% yes)

Pattern 3: Long-term outcome binding Tie short-term to long-term:

Primary: New user signups
+ Bound to: 30-day retention (signups only count if user retained)

Pattern 4: Peer comparison Normalize by cohort or segment:

Primary: Sales closed
+ Normalized: Sales closed / Sales qualified leads (prevents cherry-picking easy wins)

---

5. Advanced Leading Indicators

Technique 1: Propensity Scoring

Predict future behavior from early signals.

Method:

1. Collect historical data: New users + their 30-day outcomes
2. Identify features: Day 1 behaviors (actions, time spent, features used)
3. Build model: Logistic regression or decision tree predicting 30-day retention
4. Score new users: Probability of retention based on day 1 behavior
5. Threshold: Users with >70% propensity score are "likely retained"

Example (SaaS):

30-day retention model (R² = 0.78):
Retention = 0.1 + 0.35×(invited teammate) + 0.25×(completed 3 workflows) + 0.20×(time in app >20min)

Leading indicator: % of users with propensity score >0.7
Current: 45% → Target: 60% (predicts 15% retention increase)

Technique 2: Cohort Behavior Clustering

Find archetypes that predict outcomes.

Method:

1. Segment users by first-week behavior patterns
2. Measure long-term outcomes per segment
3. Identify high-value archetypes

Example:

Archetypes (first week):
- "Power user": 5+ days active, 20+ actions → 85% retain
- "Social": Invites 2+ people, comments 3+ times → 75% retain
- "Explorer": Views 10+ pages, low actions → 40% retain
- "Passive": <3 days active, <5 actions → 15% retain

Leading indicator: % of new users becoming "Power" or "Social" archetypes
Target: Move 30% → 45% into high-value archetypes

Technique 3: Inflection Point Analysis

Find tipping points where behavior changes sharply.

Method:

1. Plot outcome (retention) vs candidate metric (actions taken)
2. Find where curve steepens (inflection point)
3. Set that as leading indicator threshold

Example:

Retention by messages sent (first week):
- 0-2 messages: 20% retention (slow growth)
- 3-9 messages: 45% retention (moderate growth)
- 10+ messages: 80% retention (sharp jump)

Inflection point: 10 messages
Leading indicator: % of users hitting 10+ messages in first week

---

6. Metric Interdependencies

Problem

Metrics aren't independent; changing one affects others in complex ways.

Solution: Causal Diagram

Step 1: Map relationships Draw arrows showing how metrics affect each other:

[Acquisition] → [Active Users] → [Engagement] → [Retention]
     ↓                                              ↑
[Activation] ----------------------------------------

Step 2: Identify feedback loops

• Positive loop (reinforcing): A → B → A (exponential) Example: More users → more network value → more users
• Negative loop (balancing): A → B → ¬A (equilibrium) Example: More supply → lower prices → less supply

Step 3: Predict second-order effects If you increase metric X by 10%:

• Direct effect: Y increases 5%
• Indirect effect: Y affects Z, which feeds back to X
• Net effect: May be amplified or dampened

Example (Marketplace):

Increase driver supply +10%:
  → Wait time decreases -15%
  → Rider satisfaction increases +8%
  → Rider demand increases +5%
  → Driver earnings decrease -3% (more competition)
  → Driver churn increases +2%
  → Net driver supply increase: +10% -2% = +8%

Modeling Tradeoffs

Technique: Regression or experiments

Run A/B test increasing X
Measure all related metrics
Calculate elasticities:
  - If X increases 1%, Y changes by [elasticity]%
Build tradeoff matrix

Tradeoff Matrix Example:

If increase by 10%	Acquisition	Activation	Retention	Revenue
Acquisition	+10%	-2%	-1%	+6%
Activation	0%	+10%	+5%	+12%
Retention	0%	+3%	+10%	+15%

Interpretation: Investing in retention has best ROI (15% revenue lift vs 6% from acquisition).

---

7. Business Stage Metrics

Problem

Optimal metrics change as business matures. Early-stage metrics differ from growth or maturity stages.

Stage-Specific North Stars

Pre-Product/Market Fit (PMF):

• Focus: Finding PMF, not scaling
• North Star: Retention (evidence of value)
• Key metrics:
  • Week 1 → Week 2 retention (>40% = promising)
  • NPS or "very disappointed" survey (>40% = good signal)
  • Organic usage frequency (weekly+ = habit-forming)

Post-PMF, Pre-Scale:

• Focus: Unit economics and growth
• North Star: New activated users per week (acquisition + activation)
• Key metrics:
  • LTV/CAC ratio (target >3:1)
  • Payback period (target <12 months)
  • Month-over-month growth rate (target >10%)

Growth Stage:

• Focus: Efficient scaling
• North Star: Revenue or gross profit
• Key metrics:
  • Net revenue retention (target >100%)
  • Magic number (ARR growth / S&M spend, target >0.75)
  • Burn multiple (cash burned / ARR added, target <1.5)

Maturity Stage:

• Focus: Profitability and market share
• North Star: Free cash flow or EBITDA
• Key metrics:
  • Operating margin (target >20%)
  • Market share / competitive position
  • Customer lifetime value

Transition Triggers

When to change North Star:

PMF → Growth: When retention >40%, NPS >40, organic growth observed
Growth → Maturity: When growth rate <20% for 2+ quarters, market share >30%

Migration approach:

1. Track both old and new North Star for 2 quarters
2. Align teams on new metric
3. Deprecate old metric
4. Update dashboards and incentives

---

Quick Decision Trees

"Should I use counter-metrics?"

Is primary metric easy to game or has quality risk?
├─ YES → Add counter-metrics with guardrails
└─ NO → Is metric clearly aligned with user value?
    ├─ YES → Primary metric sufficient, monitor only
    └─ NO → Redesign metric to better capture value

"Do I have network effects?"

Does value increase as more users join?
├─ YES → Track network density, K-factor, measure at different scales
└─ NO → Does one user's action benefit others?
    ├─ YES → Measure cross-user interactions, content creation/consumption
    └─ NO → Standard metrics tree (no network effects)

"Should I segment my metrics tree?"

Do different user segments have different behavior patterns?
├─ YES → Do segments have different value to business?
    ├─ YES → Create separate trees per segment, track segment mix
    └─ NO → Single tree, annotate with segment breakdowns
└─ NO → Are there supply/demand sides?
    ├─ YES → Dual trees (Section 1)
    └─ NO → Single unified tree

---

Summary: Advanced Technique Selector

Scenario	Use This Technique	Section
Multi-sided marketplace	Dual tree + balance metrics	1
Risk of negative externalities	Counter-metrics + guardrails	2
Viral or network product	K-factor + network density	3
Metric gaming risk	Quality signals + combination metrics	4
Need better prediction	Propensity scoring + archetypes	5
Complex interdependencies	Causal diagram + elasticities	6
Changing business stage	Stage-appropriate North Star	7