zhongwei/gh-lyndonkl-claude
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-30 08:38:26 +08:00
commit 41d9f6b189
304 changed files with 98322 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

256
skills/estimation-fermi/resources/evaluators/rubric_estimation_fermi.json Normal file
View File

@@ -0,0 +1,256 @@
{
"criteria": [
{
"name": "Question Clarification",
"1": "Question vague, units missing, scope undefined, decision context unclear",
"3": "Question restated with units, scope bounded, decision context mentioned",
"5": "Question precise and unambiguous, units specified, scope clearly defined, decision context and required precision explicit"
},
{
"name": "Decomposition Quality",
"1": "No decomposition or single wild guess, components not estimable, unclear how they combine",
"3": "Logical decomposition into 2-3 levels, components mostly estimable, formula clear",
"5": "Elegant decomposition (3-5 levels), all components independently estimable from knowledge/experience, path from components to answer transparent"
},
{
"name": "Assumptions Explicit",
"1": "Assumptions unstated, black-box numbers with no justification, cannot be challenged",
"3": "Major assumptions stated, some justification provided, most can be identified and questioned",
"5": "All assumptions explicitly stated, fully justified with anchors/sources, sensitivity noted, easily challenged and refined"
},
{
"name": "Anchoring",
"1": "Components based on guesses not anchored in any known quantities, no sources cited",
"3": "Most components anchored in known data (population, benchmarks, personal experience), some sources mentioned",
"5": "All components grounded in credible anchors (data, benchmarks, analogies, first principles), sources cited, confidence levels assessed"
},
{
"name": "Bounding",
"1": "No bounds provided, single point estimate only, no sense of uncertainty range",
"3": "Upper and lower bounds calculated, range assessed (factor of X), some scenario analysis",
"5": "Rigorous bounds via optimistic/pessimistic scenarios, range quantified, decision sensitivity assessed (does decision change across range?), constraints checked"
},
{
"name": "Calculation Correctness",
"1": "Math errors, units don't match, formula incorrect, compounding mistakes",
"3": "Math generally correct, units mostly consistent, formula works, minor errors possible",
"5": "Math accurate, dimensional analysis verified (units cancel correctly), formula logic sound, calculation transparent and reproducible"
},
{
"name": "Sanity Checking",
"1": "No validation, answer not compared to reality, obvious errors not caught",
"3": "Some sanity checks (order of magnitude comparison, gut check), major errors would be caught",
"5": "Comprehensive validation (dimensional analysis, reality comparison, extreme case testing, derived metrics consistency, gut check), implausible results flagged and investigated"
},
{
"name": "Triangulation",
"1": "Single approach only, no alternate path, cannot validate estimate",
"3": "Attempted alternate decomposition, comparison made, discrepancies noted",
"5": "Multiple independent paths (top-down vs bottom-up, supply vs demand), estimates within factor of 3, reconciliation of differences, confidence increased by agreement"
},
{
"name": "Precision Appropriate",
"1": "False precision (8.372M when uncertainty is ±10×), or uselessly vague (\"a lot\"), wrong significant figures",
"3": "Rounded to 1-2 significant figures, order of magnitude clear, some uncertainty acknowledged",
"5": "Precision matches uncertainty (1 sig fig for ±3×, 2 sig figs for ±30%), expressed as range when appropriate, confidence intervals calibrated, avoids false precision trap"
},
{
"name": "Decision Actionability",
"1": "Estimate disconnected from decision, no recommendation, unclear how to use result",
"3": "Connection to decision mentioned, some implication for action, directionally useful",
"5": "Clear decision implication (go/no-go, prioritize/deprioritize), threshold analysis (if >X then Y), sensitivity to key assumptions identified, actionable recommendation based on estimate and uncertainty"
}
],
"guidance_by_type": {
"Market Sizing (TAM/SAM/SOM)": {
"target_score": 4.0,
"key_requirements": [
"Top-down decomposition (population → filters → addressable → price), components estimable from census/industry data",
"Anchors: Population figures, market penetration rates, pricing from comparables, industry reports",
"Bounds: Optimistic (high penetration, premium price) vs Pessimistic (low penetration, discount price)",
"Triangulation: Cross-check against public company revenues in space, VC market estimates, bottom-up from customer count",
"Sanity check: Compare to GDP (market can't exceed consumer spending), check per-capita figures"
],
"common_pitfalls": [
"Confusing TAM (total addressable) with SAM (serviceable) or SOM (obtainable)",
"Overestimating willingness to pay (assume most won't pay)",
"Not accounting for competition (you won't get 100% share)"
]
},
"Infrastructure Capacity": {
"target_score": 4.1,
"key_requirements": [
"Decomposition: Users → Actions per user → Resources per action → Overhead/utilization",
"Anchors: Similar systems (Instagram scale), known limits (AWS instance capacity), load testing data, utilization factors (70-80% not 100%)",
"Bounds: Peak load (Black Friday, viral event) vs Average, Growth trajectory (2× vs 10× per year)",
"Triangulation: Top-down from users vs Bottom-up from server capacity, cost validation (does $/user make sense?)",
"Sanity check: Cost per user < LTV, compare to public cloud bills of similar companies"
],
"common_pitfalls": [
"Assuming 100% utilization (real systems: 70-80% for headroom)",
"Forgetting overhead (databases, load balancers, redundancy)",
"Linear scaling assumptions (ignoring caching, batching gains)"
]
},
"Financial Projections": {
"target_score": 3.9,
"key_requirements": [
"Decomposition: Revenue (customers × conversion × ARPU), Costs (COGS + CAC + R&D + G&A)",
"Anchors: Cohort data (historical conversion), industry benchmarks (CAC/LTV ratios, SaaS metrics), comparable company metrics",
"Bounds: Bull case (high growth, efficient scaling) vs Bear case (slow growth, rising costs)",
"Triangulation: Build cohort model vs top-down market share model, cross-check margins with industry",
"Sanity check: Growth follows S-curve not exponential forever, margins approach industry norms at scale, rule of 40"
],
"common_pitfalls": [
"Assuming exponential growth continues indefinitely",
"Not accounting for churn (especially in SaaS)",
"Ignoring seasonality or cyclicality"
]
},
"Resource Planning (Headcount/Budget)": {
"target_score": 4.0,
"key_requirements": [
"Decomposition: Work to be done (features, tickets, customers) → Productivity per person → Overhead (meetings, vacation, ramp)",
"Anchors: Team velocity (story points/sprint), industry ratios (support agents per 1000 customers), hiring timelines",
"Bounds: Experienced team (high productivity, low ramp) vs New team (learning curve, attrition)",
"Triangulation: Bottom-up from roadmap vs Top-down from revenue per employee benchmark",
"Sanity check: Headcount vs revenue growth (should correlate), compare to peer companies at similar scale"
],
"common_pitfalls": [
"Not accounting for ramp time (new hires take 3-6 months to full productivity)",
"Ignoring overhead (meetings, hiring, training consume 20-30% of time)",
"Underestimating hiring pipeline (offer to start date 1-3 months)"
]
},
"Impact Assessment": {
"target_score": 3.8,
"key_requirements": [
"Decomposition: Total impact = Units affected × Impact per unit × Duration, account for baseline and counterfactual",
"Anchors: Emission factors (kg CO2/kWh), conversion rates (program → behavior change), precedent studies with measured effects",
"Bounds: Conservative (low adoption, small effect) vs Optimistic (high adoption, large effect)",
"Triangulation: Top-down (total population × penetration) vs Bottom-up (measured cases × scale factor)",
"Sanity check: Impact should scale linearly or sub-linearly (diminishing returns), compare to similar interventions"
],
"common_pitfalls": [
"Not accounting for counterfactual (what would have happened anyway)",
"Ignoring adoption rates (not everyone participates)",
"Linear extrapolation when effects have diminishing returns"
]
}
},
"guidance_by_complexity": {
"Simple Question (1-2 levels)": {
"target_score": 3.5,
"description": "Single decomposition (A × B), components directly estimable, low uncertainty",
"key_requirements": [
"Clear decomposition (2-3 components), formula transparent",
"Components anchored in known quantities or personal experience",
"Basic bounds (±2-3×), sanity check against reality",
"Single approach sufficient if well-validated"
],
"time_estimate": "3-5 minutes",
"examples": [
"Annual revenue from daily revenue (revenue/day × 365)",
"Customers served per year (customers/day × 250 workdays)",
"Storage needed (users × data per user)"
]
},
"Moderate Question (3-4 levels)": {
"target_score": 4.0,
"description": "Multi-level decomposition, some uncertainty in components, decision depends on order of magnitude",
"key_requirements": [
"Logical 3-4 level decomposition, all components independently estimable",
"Assumptions explicit, anchored in data or benchmarks",
"Bounds calculated (optimistic/pessimistic scenarios), range assessed",
"Triangulation via alternate path, estimates within factor of 3",
"Comprehensive sanity checks (units, reality comparison, derived metrics)"
],
"time_estimate": "10-15 minutes",
"examples": [
"Market sizing (population → segment → addressable → price)",
"Server capacity needed (users → requests → compute)",
"Headcount planning (work → productivity → overhead)"
]
},
"Complex Question (5+ levels)": {
"target_score": 4.3,
"description": "Deep decomposition tree, high uncertainty, decision sensitive to assumptions, requires triangulation",
"key_requirements": [
"Sophisticated decomposition (5+ levels or multiple parallel paths), components validated independently",
"All assumptions stated and justified, sensitivity analysis (which matter most?)",
"Rigorous bounding (scenario analysis, constraint checking), decision sensitivity assessed",
"Multiple triangulation paths (top-down/bottom-up, supply/demand), cross-validation with public data",
"Extensive sanity checking (dimensional analysis, extreme cases, internal consistency)",
"Uncertainty quantified (confidence intervals), precision matched to uncertainty"
],
"time_estimate": "20-30 minutes",
"examples": [
"Multi-year financial model (revenue streams × growth × costs × scenarios)",
"Climate impact assessment (emissions × multiple sources × counterfactual × time horizon)",
"Large infrastructure sizing (users × behavior × compute × storage × network × redundancy)"
]
}
},
"common_failure_modes": [
{
"failure": "Missing units or unit errors",
"symptom": "Mixing per-day with per-year, confusing millions and billions, currency not specified, units don't cancel in formula",
"detection": "Dimensional analysis fails (units don't match), or derived metrics nonsensical (revenue per employee = $5)",
"fix": "Always write units next to every number, verify dimensional analysis (units cancel to expected final unit), convert all to common time basis before combining"
},
{
"failure": "Unstated assumptions",
"symptom": "Numbers appear without justification, reader cannot challenge or refine estimate, black-box calculation",
"detection": "Ask 'Why this number?' and no answer is provided in documentation",
"fix": "For each component, explicitly state assumption and anchor (e.g., 'Assuming 250 workdays/year', 'Based on industry benchmark of 3% conversion')"
},
{
"failure": "False precision",
"symptom": "8.372M when uncertainty is ±5×, or $47,293 when components are rough guesses",
"detection": "More than 2 significant figures, or precision implies certainty mismatched to actual uncertainty",
"fix": "Round to 1-2 significant figures matching uncertainty (±3× → 1 sig fig, ±30% → 2 sig figs), express as range when uncertainty high"
},
{
"failure": "No bounds or sanity checks",
"symptom": "Single point estimate, no validation, implausible result not caught (market size exceeds GDP, growth >100%/year sustained)",
"detection": "Estimate seems wrong intuitively but no mechanism to validate, or obviously violates constraints",
"fix": "Always calculate bounds (optimistic/pessimistic), compare to known comparables, check extreme cases, verify doesn't violate physics/economics"
},
{
"failure": "Decomposition not estimable",
"symptom": "Components still too complex ('estimate market size' → 'estimate demand' is not progress), or circular (define A in terms of B, B in terms of A)",
"detection": "Ask 'Can I estimate this component from knowledge/data?' If no, decomposition incomplete",
"fix": "Decompose until each component answerable from: known data, personal experience, analogous comparison, or first principles"
},
{
"failure": "Anchoring bias",
"symptom": "Estimate heavily influenced by first number heard, even if irrelevant ('Is it $10M?' causes you to anchor near $10M)",
"detection": "Estimate changes significantly based on arbitrary starting point provided",
"fix": "Generate independent estimate before seeing any suggested numbers, then compare and justify any convergence"
},
{
"failure": "Double-counting",
"symptom": "Same quantity included twice in formula (counting businesses AND employees when businesses already includes employee count)",
"detection": "Draw decomposition tree visually - if same box appears twice in different branches, likely double-counted",
"fix": "Clearly define what each component includes and excludes, ensure mutually exclusive and collectively exhaustive (MECE)"
},
{
"failure": "No triangulation",
"symptom": "Single path only, cannot validate estimate, reader must trust decomposition is correct",
"detection": "Only one approach provided, no alternate path or comparison to known data",
"fix": "Re-estimate via different decomposition (top-down vs bottom-up, supply vs demand), check if within factor of 3, cross-validate with public data when available"
},
{
"failure": "Ignoring utilization/efficiency",
"symptom": "Assuming 100% capacity (servers always at max, people work 40hr with zero meetings/vacation, factories run 24/7 with no downtime)",
"detection": "Capacity estimates seem too high compared to real systems",
"fix": "Apply realistic utilization factors (servers 70-80%, people 60-70% after meetings/overhead, factories 80-90% after maintenance)"
},
{
"failure": "Linear extrapolation errors",
"symptom": "Assuming linear when exponential (tech adoption), or exponential when logistic (growth eventually saturates)",
"detection": "Projections violate constraints (market share >100%, growth continues at 100%/year for decades)",
"fix": "Check if relationship is truly linear, apply appropriate growth curve (exponential for early adoption, S-curve for saturation, linear for steady-state)"
}
]
}

398
skills/estimation-fermi/resources/methodology.md Normal file
View File

@@ -0,0 +1,398 @@
# Fermi Estimation Methodology
Advanced techniques for anchoring, bounding, triangulation, and calibration.
## Workflow
```
Fermi Estimation Progress:
- [ ] Step 1: Clarify the question and define metric
- [ ] Step 2: Decompose into estimable components
- [ ] Step 3: Estimate components using anchors
- [ ] Step 4: Bound with upper/lower limits
- [ ] Step 5: Calculate and sanity-check
- [ ] Step 6: Triangulate with alternate path
```
**Step 1: Clarify the question and define metric**
Define scope, units, decision context before decomposition.
**Step 2: Decompose into estimable components**
Choose decomposition strategy based on problem structure and available knowledge.
**Step 3: Estimate components using anchors**
Apply [1. Anchoring Techniques](#1-anchoring-techniques) to ground estimates in known quantities.
**Step 4: Bound with upper/lower limits**
Use [2. Bounding Techniques](#2-bounding-techniques) to bracket answer with constraints.
**Step 5: Calculate and sanity-check**
Validate using dimensional analysis, reality checks, and [3. Calibration Methods](#3-calibration-methods).
**Step 6: Triangulate with alternate path**
Apply [4. Triangulation Approaches](#4-triangulation-approaches) to estimate via different decomposition.
---
## 1. Anchoring Techniques
Methods for grounding component estimates in known quantities.
### Common Knowledge Anchors
**Demographics**: Population figures, household counts, labor force
- US population: ~330M (2020s), grows ~0.5%/year
- US households: ~130M, avg 2.5 people/household
- US labor force: ~165M, unemployment typically 3-6%
- Global: ~8B people, ~2B households, ~55% urban
**Economic**: GDP, spending, income
- US GDP: ~$25T, per capita ~$75k
- Median household income: ~$70k
- Consumer spending: ~70% of GDP
- Federal budget: ~$6T (~24% of GDP)
**Physical constants**: Time, space, energy
- Earth: ~510M km², ~70% water, ~150M km² land
- US land: ~10M km² (~3.8M sq mi)
- Day = 24 hours, year ≈ 365 days ≈ 52 weeks
- Typical human: 70kg, 2000 kcal/day, 8hr sleep, 16hr awake
**Business benchmarks**: SaaS metrics, retail, tech
- SaaS ARR per employee: $150-300k (mature companies)
- Retail revenue per sq ft: $300-500/year (varies by category)
- Tech company valuation: 5-15× revenue (growth stage), 2-5× (mature)
- CAC payback: <12 months good, <18 acceptable
### Data Lookup Strategies
**Quick reliable sources** (use for anchoring):
- Google: Population figures, company sizes, market data
- Wikipedia: Demographics, economic stats, physical data
- Public company filings: Revenue, employees, customers (10-K, investor decks)
- Industry reports: Gartner, Forrester, McKinsey (market sizing)
**Estimation from fragments**:
- Company has "thousands of employees" → Estimate 3,000-5,000
- Market is "multi-billion dollar" → Estimate $2-9B
- "Most people" do X → Estimate 60-80%
- "Rare" occurrence → Estimate <5%
### Personal Experience Anchors
**Observation-based**: Use your own data points
- How many apps on your phone? (extrapolate to avg user)
- How often do you buy X? (extrapolate to market)
- How long does task Y take? (estimate productivity)
**Analogous reasoning**: Scale from known to unknown
- If you know NYC subway ridership, estimate SF BART by population ratio
- If you know your company's churn, estimate competitor's by industry norms
- If you know local restaurant count, estimate city-wide by neighborhood scaling
**Bracketing intuition**: Use confidence ranges
- "I'm 80% confident the answer is between X and Y"
- Then use geometric mean: √(X×Y) as central estimate
- Example: Starbucks locations - probably between 10k and 50k → ~22k (actual ~16k)
---
## 2. Bounding Techniques
Methods for calculating upper/lower limits to bracket the answer.
### Constraint-Based Bounding
**Physical constraints**: Cannot exceed laws of nature
- Maximum speed: Speed of light (for data), sound (for physical travel), human limits (for manual tasks)
- Maximum density: People per area (fire code limits, standing room), data per volume (storage media)
- Maximum efficiency: Thermodynamic limits, conversion efficiency (solar ~20-25%, combustion ~35-45%)
**Economic constraints**: Cannot exceed available resources
- Market size bounded by: GDP, consumer spending, addressable population × willingness to pay
- Company revenue bounded by: Market size × maximum possible share (~20-30% typically)
- Headcount bounded by: Budget ÷ avg salary, or revenue ÷ revenue per employee benchmark
**Time constraints**: Cannot exceed available hours
- Work output bounded by: People × hours/person × productivity
- Example: "How many customers can support team handle?" → Agents × (40 hr/week - 10hr meetings) × tickets/hr
### Scenario-Based Bounding
**Optimistic scenario** (favorable assumptions):
- High adoption (80-100% of addressable market)
- Premium pricing (top quartile of range)
- High efficiency (best-in-class productivity)
- Fast growth (aggressive but plausible)
**Pessimistic scenario** (conservative assumptions):
- Low adoption (5-20% of addressable market)
- Discount pricing (bottom quartile of range)
- Low efficiency (industry average or below)
- Slow growth (cautious, accounts for setbacks)
**Example - Market sizing**:
- Optimistic: All 500k target businesses × $100/month × 12 = $600M
- Pessimistic: 5% penetration (25k) × $30/month × 12 = $9M
- Range: $9M - $600M (67× span → likely too wide, refine assumptions)
### Sensitivity Analysis
Identify which assumptions most affect result:
**Method**: Vary each component ±50%, measure impact on final estimate
**High sensitivity components**: Small change → large impact on result
- Focus refinement effort here
- Example: If CAC varies 50% and final ROI changes 40%, CAC is high sensitivity
**Low sensitivity components**: Large change → small impact on result
- Less critical to get precise
- Example: If office rent varies 50% but total costs change 5%, rent is low sensitivity
**Rule of thumb**: 80% of uncertainty often comes from 20% of assumptions. Find and refine those critical few.
---
## 3. Calibration Methods
Techniques for improving estimation accuracy through practice and feedback.
### Calibration Exercises
**Known problems**: Practice on verifiable questions, compare estimate to actual
**Exercise set 1 - Demographics**:
1. US population in 1950? (Estimate, then check: ~150M)
2. Number of countries in UN? (Estimate, then check: ~190)
3. World's tallest building height? (Estimate, then check: ~830m Burj Khalifa)
**Exercise set 2 - Business**:
1. Starbucks annual revenue? (Estimate, then check: ~$35B)
2. Google employees worldwide? (Estimate, then check: ~150k)
3. Average Netflix subscription price? (Estimate, then check: ~$15/month)
**Exercise set 3 - Consulting classics**:
1. Piano tuners in Chicago? (Estimate, then check: ~100)
2. Gas stations in US? (Estimate, then check: ~150k)
3. Golf balls fitting in school bus? (Estimate, then check: ~500k)
**Feedback loop**:
- Track your estimate vs actual ratio
- If consistently >3× off, identify bias (underestimate? overestimate?)
- Calibrate future estimates (if you typically 2× low, mentally adjust upward)
### Confidence Intervals
Express uncertainty quantitatively:
**90% confidence interval**: "I'm 90% sure the answer is between X and Y"
- Width reflects uncertainty (narrow = confident, wide = uncertain)
- Calibration: Of 10 estimates with 90% CI, ~9 should contain true value
**Common mistakes**:
- Too narrow (overconfidence): Only 50% of your "90% CIs" contain true value
- Too wide (useless): All your CIs span 3+ orders of magnitude
- Goal: Calibrated such that X% of your X% CIs are correct
**Practice calibration**:
1. Make 20 estimates with 80% CIs
2. Check how many actually contained true value
3. If <16 (80% of 20), you're overconfident → widen future CIs
4. If >18, you're underconfident → narrow future CIs
### Bias Identification
**Anchoring bias**: Over-relying on first number you hear
- Mitigation: Generate estimate independently before seeing any numbers
- Test: Estimate revenue before someone says "Is it $10M?" vs after
**Availability bias**: Overweighting recent/memorable events
- Mitigation: Seek base rates, historical averages, not just recent headline cases
- Example: Don't estimate startup success rate from TechCrunch unicorn coverage
**Optimism bias**: Tendency to assume favorable outcomes
- Mitigation: Explicitly calculate pessimistic scenario, force consideration of downsides
- Particularly important for: Timelines, costs, adoption rates
**Unit confusion**: Mixing millions/billions, per-day/per-year
- Mitigation: Always write units, check dimensional analysis
- Example: Company earns $10M/year = ~$27k/day (sanity check: does that seem right for scale?)
---
## 4. Triangulation Approaches
Estimating the same quantity via multiple independent paths to validate.
### Supply-Side vs Demand-Side
**Supply-side**: Count producers/capacity, estimate output
- Example (Uber drivers in SF):
- ~5,000 drivers (estimated from driver forum activity, Uber PR)
- ~30 rides/day per driver
- = 150,000 rides/day in SF
**Demand-side**: Count consumers/need, estimate consumption
- Example (Uber rides in SF):
- ~900k population
- ~5% use Uber daily (frequent users)
- ~3 rides per user-day
- = 135,000 rides/day in SF
**Triangulation**: 150k (supply) vs 135k (demand) → Within 10%, confidence high
### Top-Down vs Bottom-Up
**Top-down**: Start with large total, filter down
- Example (Restaurant revenue in city):
- 1M population
- ~$8k/person annual food spending
- ~40% spent at restaurants
- = $3.2B restaurant revenue
**Bottom-up**: Start with unit, scale up
- Example (Restaurant revenue in city):
- ~1,500 restaurants
- ~50 meals/day per restaurant
- ~$25 average check
- ~350 days/year
- = ~$650M restaurant revenue
**Discrepancy**: $3.2B vs $650M → 5× difference, investigate!
- Possible explanations: Underestimated restaurants (chains, cafes, food trucks), or overestimated per-capita spending
### Multiple Decomposition Paths
**Example - Estimating AWS revenue**:
**Path 1 - Customer-based**:
- ~1M active AWS customers (public statements)
- Average spend ~$10k/year (mix of startups $1k, enterprises $100k+)
- = $10B revenue
**Path 2 - Workload-based**:
- ~5M EC2 instances running globally (estimated from public data)
- ~$500/month avg per instance (mix of small/large)
- = $30B revenue
**Path 3 - Market share**:
- Cloud infrastructure market ~$200B (2023)
- AWS market share ~30%
- = $60B revenue
**Triangulation**: $10B vs $30B vs $60B → Within same order of magnitude, actual AWS revenue ~$85B (2023)
- All paths got within 3-10× (reasonable for Fermi), but spread suggests different assumptions need refinement
### Cross-Validation with Public Data
After Fermi estimate, quickly check if public data exists:
**Public company financials**: 10-K filings, investor presentations
- Validate: Revenue, employees, customers, margins
**Industry reports**: Gartner, IDC, Forrester market sizing
- Validate: TAM, growth rates, share
**Government data**: Census, BLS, FDA, EPA
- Validate: Population, employment, health, environment figures
**Academic studies**: Research papers on adoption, behavior, impact
- Validate: Penetration rates, usage patterns, effect sizes
**Purpose**: Not to replace Fermi process, but to:
1. Calibrate (am I within 10×? If not, what's wrong?)
2. Refine (can I improve assumptions with real data?)
3. Build confidence (multiple paths + public data all agree → high confidence)
---
## 5. Advanced Decomposition Patterns
### Conversion Funnel Decomposition
For estimating outcomes in multi-step processes:
**Structure**:
```
Starting population
× Conversion 1 (% that complete step 1)
× Conversion 2 (% that complete step 2 | completed 1)
× Conversion 3 (% that complete step 3 | completed 2)
= Final outcome
```
**Example - SaaS conversions**:
```
Website visitors: 100k/month
× Trial signup: 5% = 5k trials
× Activation: 60% = 3k activated
× Paid conversion: 20% = 600 new customers/month
```
### Cohort-Based Decomposition
For estimating aggregate from groups with different characteristics:
**Example - App revenue**:
```
Cohort 1 (Power users): 10k users × $20/month = $200k
Cohort 2 (Regular users): 100k users × $5/month = $500k
Cohort 3 (Free users): 1M users × $0 = $0
Total: $700k/month
```
### Dimensional Analysis
Using units to guide decomposition:
**Example**: Estimating data center power consumption (want kW)
```
Servers (count)
× Power per server (kW/server)
+ Cooling overhead (×1.5 for PUE)
= Total power (kW)
```
Units guide you: Need kW, have servers → must find kW/server, which is estimable
---
## 6. Common Estimation Pitfalls
**Compounding errors**: Errors multiply in chains
- 3 components each ±50% uncertain → final estimate ±300% uncertain
- Mitigation: Keep decomposition shallow (3-5 levels max), validate high-sensitivity components
**False precision**: Reporting 8.372M when uncertainty is ±3×
- Mitigation: Round to 1-2 significant figures (8M not 8.372M), express as range
**Linearity assumption**: Assuming proportional scaling when it's not
- Reality: Economies of scale (costs grow sub-linearly), network effects (value grows super-linearly)
- Mitigation: Check if relationship is truly linear or if power law applies
**Survivorship bias**: Estimating from successes only
- Example: Average startup revenue based on unicorns (ignoring 90% that failed)
- Mitigation: Include full distribution, weight by probability
**Time-period confusion**: Mixing annual, monthly, daily figures
- Example: "Company earns $1M" - per year? per month? Need clarity.
- Mitigation: Always specify time units, convert to common basis
**Outdated anchors**: Using pre-pandemic data for post-pandemic estimates
- Example: Office occupancy, remote work adoption, e-commerce penetration all shifted
- Mitigation: Check anchor recency, adjust for structural changes
**Ignoring constraints**: Estimates that violate physics/economics
- Example: Market size exceeding GDP, growth rate >100%/year sustained indefinitely
- Mitigation: Sanity-check against absolute limits
**Double-counting**: Including same quantity twice
- Example: Counting both "businesses" and "employees" when businesses already includes employee count
- Mitigation: Draw clear decomposition tree, check for overlap

345
skills/estimation-fermi/resources/template.md Normal file
View File

@@ -0,0 +1,345 @@
# Fermi Estimation Templates
Quick-start templates for decomposition, component estimation, bounding, and sanity-checking.
## Workflow
```
Fermi Estimation Progress:
- [ ] Step 1: Clarify the question and define metric
- [ ] Step 2: Decompose into estimable components
- [ ] Step 3: Estimate components using anchors
- [ ] Step 4: Bound with upper/lower limits
- [ ] Step 5: Calculate and sanity-check
- [ ] Step 6: Triangulate with alternate path
```
**Step 1: Clarify the question and define metric**
Use [Clarification Template](#clarification-template) to restate question precisely with units, scope, timeframe, and decision context.
**Step 2: Decompose into estimable components**
Select decomposition strategy using [Decomposition Strategies](#decomposition-strategies) and build estimation tree.
**Step 3: Estimate components using anchors**
Ground each component in known quantities using [Component Estimation Template](#component-estimation-template).
**Step 4: Bound with upper/lower limits**
Calculate optimistic and pessimistic bounds using [Bounding Template](#bounding-template).
**Step 5: Calculate and sanity-check**
Compute central estimate and validate using [Sanity-Check Template](#sanity-check-template).
**Step 6: Triangulate with alternate path**
Re-estimate via different decomposition using [Triangulation Template](#triangulation-template) to validate.
---
## Clarification Template
**Original Question**: [As stated]
**Clarified Question**:
- **What exactly are we estimating?** (metric name, definition)
- **Units?** (dollars, people, tons, requests/sec, etc.)
- **Geographic scope?** (US, global, city, region)
- **Time scope?** (annual, daily, lifetime, point-in-time)
- **Exclusions?** (what are we NOT counting?)
**Decision Context**:
- **What decision depends on this estimate?**
- **What precision is needed?** (order of magnitude sufficient? need ±2x? ±10%?)
- **How will estimate be used?** (go/no-go decision, budget planning, prioritization)
- **What threshold matters?** (if >X we do Y, if <Z we don't)
**Success Criteria**:
- [ ] Question is unambiguous (two people would estimate same thing)
- [ ] Units are specified
- [ ] Scope is bounded (not infinite)
- [ ] Decision context is clear
---
## Decomposition Strategies
Choose strategy based on question structure and available knowledge.
### Top-Down Filtering
**When to use**: Estimating subset of total population (market sizing, target audience)
**Structure**: Start with total, apply filters to narrow down
**Template**:
```
Total population
× Filter 1 (% that have characteristic A)
× Filter 2 (% that have characteristic B | A)
× Filter 3 (% that have characteristic C | A,B)
= Target population
```
**Example - TAM estimation**:
```
US population (330M)
× Adults 18-65 (65% = 215M)
× Urban/suburban (80% = 172M)
× Smartphone users (90% = 155M)
× Willing to pay for app (5% = 7.75M)
× Price point ($50/year)
= TAM: ~$390M/year
```
### Bottom-Up Scaling
**When to use**: Have data on units, want to estimate total (revenue, capacity, production)
**Structure**: Start with single unit, multiply by count
**Template**:
```
Output per unit
× Number of units
× Utilization rate (% of theoretical capacity actually used)
= Total output
```
**Example - Server capacity**:
```
1 server handles 10k requests/sec
× 100 servers in cluster
× 70% utilization (peak headroom, maintenance, failures)
= Cluster capacity: 700k requests/sec
```
### Rate × Time
**When to use**: Estimating accumulation over time (customers served, miles driven, content consumed)
**Structure**: Flow rate × Duration
**Template**:
```
Rate (per time unit)
× Time duration (in same units)
= Total quantity
```
**Example - Annual customers**:
```
50 customers/day
× 250 business days/year
= 12,500 customers/year
```
### Density × Area/Volume
**When to use**: Spatial estimation (people in stadium, cars on highway, storage capacity)
**Structure**: Concentration × Space
**Template**:
```
Density (per unit area/volume)
× Total area/volume
= Total quantity
```
**Example - Stadium capacity**:
```
4 people/sq meter (seated)
× 10,000 sq meters seating area
= 40,000 capacity
```
### Analogous Scaling
**When to use**: Have data on similar system, adjusting for size difference
**Structure**: Known comparable × Scaling factor
**Template**:
```
Comparable system value
× (Our scale / Their scale)^exponent
= Our estimate
Exponent = 1 for linear scaling, <1 for economies of scale, >1 for diseconomies
```
**Example - Competitor revenue**:
```
Competitor A revenue: $100M with 500k users
Our users: 50k
Estimate: $100M × (50k/500k) = $10M
(assumes linear revenue per user)
```
---
## Component Estimation Template
For each component in decomposition:
**Component**: [Name of quantity being estimated]
**Anchor Strategy** (how will we estimate this?):
- [ ] Known data (cite source):
- [ ] Personal experience/intuition (describe basis):
- [ ] Derived from other estimates (show calculation):
- [ ] Analogous comparison (similar known quantity):
**Estimate**: [Value with units]
**Confidence**: (How certain are we?)
- [ ] High (have data or strong knowledge)
- [ ] Medium (reasonable inference)
- [ ] Low (educated guess, wide range)
**Sensitivity**: (How much does final answer depend on this?)
- [ ] High (10% change here → >5% change in final)
- [ ] Medium (10% change here → 1-5% change in final)
- [ ] Low (final answer insensitive to this component)
**Assumption stated**: [Explicit statement of what we're assuming]
---
## Bounding Template
### Optimistic Bound (Upper Limit)
**Scenario**: [What favorable conditions would maximize the estimate?]
**Assumptions**:
- Component 1: [Optimistic value]
- Component 2: [Optimistic value]
- ...
**Calculation**: [Show work]
**Result**: [Upper bound with units]
### Pessimistic Bound (Lower Limit)
**Scenario**: [What unfavorable conditions would minimize the estimate?]
**Assumptions**:
- Component 1: [Pessimistic value]
- Component 2: [Pessimistic value]
- ...
**Calculation**: [Show work]
**Result**: [Lower bound with units]
### Range Assessment
**Bounds**: [Lower] to [Upper]
**Ratio**: Upper/Lower = [X]× range
**Decision Sensitivity**:
- [ ] Decision same across entire range → Estimate good enough
- [ ] Decision changes within range → Need more precision or different approach
- [ ] Range too wide (>10× span) → Decomposition may be flawed, revisit assumptions
---
## Sanity-Check Template
### Dimensional Analysis
**Units Check**: Do units cancel correctly in calculation?
- Calculation: [Show units through formula]
- Final units: [Should match expected units]
- [ ] Units are correct
### Reality Checks
**Check 1 - Order of magnitude comparison**:
- Our estimate: [X]
- Known comparable: [Y]
- Ratio: [X/Y]
- [ ] Within factor of 10? (If not, investigate why)
**Check 2 - Extreme case testing**:
- What if assumption taken to extreme? (e.g., 100% penetration, everyone participates)
- Result: [Calculate]
- [ ] Does it violate physical/economic constraints? (population limits, GDP constraints, physics)
**Check 3 - Internal consistency**:
- Do derived metrics make sense? (profit margins, growth rates, per-capita figures)
- Derived metric: [Calculate, e.g., revenue per employee, cost per user]
- [ ] Is it in reasonable range for industry/domain?
**Check 4 - Personal intuition**:
- Does this "feel right" given your experience?
- [ ] Passes gut check
- [ ] Feels too high (revise assumptions)
- [ ] Feels too low (revise assumptions)
### Common Failure Modes to Check
- [ ] Did I double-count anything?
- [ ] Did I mix units (per day vs per year, millions vs billions)?
- [ ] Am I extrapolating linearly when should be exponential (or vice versa)?
- [ ] Did I account for utilization/efficiency factors (not everything runs at 100%)?
- [ ] Did I consider survivor bias (basing estimate on successful cases only)?
---
## Triangulation Template
### Primary Estimate (from main decomposition)
**Method**: [Top-down, bottom-up, etc.]
**Decomposition**: [Brief formula or tree]
**Result**: [Value with units]
### Alternate Estimate (different approach)
**Method**: [Different strategy than primary]
**Decomposition**: [Brief formula or tree]
**Result**: [Value with units]
### Comparison
**Primary**: [X]
**Alternate**: [Y]
**Ratio**: [X/Y] = [Z]×
**Assessment**:
- [ ] Within factor of 3 (good agreement, increase confidence)
- [ ] Factor of 3-10 (moderate agreement, average or investigate difference)
- [ ] >10× difference (investigate: one decomposition is likely flawed)
**Reconciliation** (if estimates differ):
- Why do they differ? (which assumptions differ?)
- Which approach is more reliable?
- Final estimate: [Choice or average]
---
## Complete Estimation Template
Structure for full documentation:
1. **Clarification**: Original question, clarified with units/scope, decision context
2. **Decomposition**: Strategy (top-down/bottom-up/etc), formula, estimation tree
3. **Component Estimates**: For each component - estimate, anchor, assumption, confidence
4. **Calculation**: Formula with numbers, central estimate (1-2 sig figs)
5. **Bounds**: Optimistic/pessimistic scenarios, range assessment
6. **Sanity Checks**: Units, order of magnitude comparison, extreme cases, consistency, gut check
7. **Triangulation**: Alternate approach, comparison (within factor of 3?), reconciliation
8. **Final Estimate**: Point estimate, range, confidence, key assumptions, sensitivity, recommendation
9. **Next Steps**: Data collection, assumption testing, detailed modeling if precision needed