Initial commit
This commit is contained in:
235
skills/estimation-fermi/SKILL.md
Normal file
235
skills/estimation-fermi/SKILL.md
Normal file
@@ -0,0 +1,235 @@
|
||||
---
|
||||
name: estimation-fermi
|
||||
description: Use when making quick order-of-magnitude estimates under uncertainty (market sizing, resource planning, feasibility checks), decomposing complex quantities into estimable parts, bounding unknowns with upper/lower limits, sanity-checking strategic assumptions, or when user mentions Fermi estimation, back-of-envelope calculation, order of magnitude, ballpark estimate, triangulation, or needs to assess feasibility before detailed analysis.
|
||||
---
|
||||
# Fermi Estimation
|
||||
|
||||
## Table of Contents
|
||||
- [Purpose](#purpose)
|
||||
- [When to Use](#when-to-use)
|
||||
- [What Is It?](#what-is-it)
|
||||
- [Workflow](#workflow)
|
||||
- [Common Patterns](#common-patterns)
|
||||
- [Guardrails](#guardrails)
|
||||
- [Quick Reference](#quick-reference)
|
||||
|
||||
## Purpose
|
||||
|
||||
Fermi estimation provides rapid order-of-magnitude answers to seemingly impossible questions by decomposing them into smaller, estimable parts. This skill guides you through decomposition strategies, bounding techniques, sanity checks, and triangulation to make defensible estimates when data is scarce, time is limited, or precision is unnecessary for the decision at hand.
|
||||
|
||||
## When to Use
|
||||
|
||||
Use this skill when:
|
||||
|
||||
- **Market sizing**: Estimating TAM/SAM/SOM for product launch, addressable market for new feature, competitive market share
|
||||
- **Resource planning**: Infrastructure capacity (servers, storage, bandwidth), staffing needs, budget allocation, inventory requirements
|
||||
- **Feasibility checks**: Can we build this in 6 months? Will customers pay $X? Is this market big enough?
|
||||
- **Strategic decisions**: Build vs buy tradeoffs, enter new market assessment, fundraising/runway calculations, pricing validation
|
||||
- **Business metrics**: Revenue projections, customer acquisition costs, LTV estimates, unit economics, break-even analysis
|
||||
- **Impact assessment**: Carbon footprint, energy consumption, social reach, cost savings from initiative
|
||||
- **Interview questions**: Consulting case interviews (piano tuners in Chicago), product sense questions, analytical reasoning tests
|
||||
- **Quick validation**: Sanity-checking detailed models, pressure-testing assumptions, getting directional answer before investing in precision
|
||||
|
||||
Trigger phrases: "ballpark estimate", "order of magnitude", "back-of-envelope", "roughly how many", "feasibility check", "gut check", "triangulate", "sanity check"
|
||||
|
||||
## What Is It?
|
||||
|
||||
Fermi estimation (named after physicist Enrico Fermi) breaks down complex unknowns into simpler components that can be estimated using common knowledge, constraints, and reasoning. The goal is not precision but being "right to within a factor of 10" quickly.
|
||||
|
||||
**Quick example:**
|
||||
|
||||
**Question**: How many piano tuners are in Chicago?
|
||||
|
||||
**Fermi decomposition**:
|
||||
1. **Population**: Chicago ~3 million people
|
||||
2. **Households**: 3M people ÷ 3 people/household = 1M households
|
||||
3. **Pianos**: ~1 in 20 households has piano = 50,000 pianos
|
||||
4. **Tuning frequency**: Piano tuned once/year on average
|
||||
5. **Tunings needed**: 50,000 tunings/year
|
||||
6. **Tuner capacity**: Tuner works 250 days/year, 4 tunings/day = 1,000 tunings/year per tuner
|
||||
7. **Tuners needed**: 50,000 ÷ 1,000 = **~50 piano tuners**
|
||||
|
||||
**Actual**: ~80-100 piano tuners in Chicago (within order of magnitude ✓)
|
||||
|
||||
**Business example - Market sizing:**
|
||||
|
||||
**Question**: What's the TAM for a B2B sales automation SaaS in the US?
|
||||
|
||||
**Decomposition**:
|
||||
1. **Total businesses in US**: ~30M
|
||||
2. **With sales teams**: ~10% = 3M businesses
|
||||
3. **With >10 employees** (can afford SaaS): ~2M businesses
|
||||
4. **Addressable** (tech-savvy, not enterprise with custom solutions): ~500k businesses
|
||||
5. **Price point**: $500/month average
|
||||
6. **TAM**: 500k × $500/month × 12 = **$3B/year**
|
||||
|
||||
**Validation**: Quick search confirms B2B sales tech market ~$5-7B (same order of magnitude ✓)
|
||||
|
||||
## Workflow
|
||||
|
||||
Copy this checklist and track your progress:
|
||||
|
||||
```
|
||||
Fermi Estimation Progress:
|
||||
- [ ] Step 1: Clarify the question and define metric
|
||||
- [ ] Step 2: Decompose into estimable components
|
||||
- [ ] Step 3: Estimate components using anchors
|
||||
- [ ] Step 4: Bound with upper/lower limits
|
||||
- [ ] Step 5: Calculate and sanity-check
|
||||
- [ ] Step 6: Triangulate with alternate path
|
||||
```
|
||||
|
||||
**Step 1: Clarify the question and define metric**
|
||||
|
||||
Restate question precisely (units, scope, timeframe). Identify what decision hinges on estimate (directional answer sufficient? order of magnitude?). See [resources/template.md](resources/template.md#clarification-template) for question clarification framework.
|
||||
|
||||
**Step 2: Decompose into estimable components**
|
||||
|
||||
Break unknown into product/quotient of knowable parts. Choose decomposition strategy (top-down, bottom-up, dimensional analysis). See [resources/template.md](resources/template.md#decomposition-strategies) for decomposition patterns.
|
||||
|
||||
**Step 3: Estimate components using anchors**
|
||||
|
||||
Ground estimates in known quantities (population, physical constants, market sizes, personal experience). State assumptions explicitly. See [resources/methodology.md](resources/methodology.md#anchoring-techniques) for anchor sources and calibration.
|
||||
|
||||
**Step 4: Bound with upper/lower limits**
|
||||
|
||||
Calculate optimistic (upper) and pessimistic (lower) bounds to bracket answer. Check if decision changes across range. See [resources/methodology.md](resources/methodology.md#bounding-techniques) for constraint-based bounding.
|
||||
|
||||
**Step 5: Calculate and sanity-check**
|
||||
|
||||
Compute estimate, round to 1-2 significant figures. Sanity-check against reality (does answer pass smell test?). See [resources/template.md](resources/template.md#sanity-check-template) for validation criteria.
|
||||
|
||||
**Step 6: Triangulate with alternate path**
|
||||
|
||||
Re-estimate using different decomposition to validate. Check if both paths yield same order of magnitude. Validate using [resources/evaluators/rubric_estimation_fermi.json](resources/evaluators/rubric_estimation_fermi.json). **Minimum standard**: Average score ≥ 3.5.
|
||||
|
||||
## Common Patterns
|
||||
|
||||
**Pattern 1: Market Sizing (TAM/SAM/SOM)**
|
||||
- **Decomposition**: Total population → Target segment → Addressable → Reachable → Price point
|
||||
- **Anchors**: Census data, industry reports, analogous markets, penetration rates
|
||||
- **Bounds**: Optimistic (high penetration, premium pricing) vs Pessimistic (low penetration, discount pricing)
|
||||
- **Sanity check**: Compare to public company revenues in space, VC market size estimates
|
||||
- **Example**: E-commerce TAM = US population × online shopping % × avg spend/year
|
||||
|
||||
**Pattern 2: Infrastructure Capacity**
|
||||
- **Decomposition**: Users → Requests per user → Compute/storage per request → Overhead
|
||||
- **Anchors**: Similar services (Instagram, Twitter), known capacity (EC2 instance limits), load testing data
|
||||
- **Bounds**: Peak (Black Friday) vs Average load, Growth trajectory (2x/year vs 10x/year)
|
||||
- **Sanity check**: Cost per user should be < LTV, compare to public cloud bills of similar scale
|
||||
- **Example**: Servers needed = (DAU × requests/user × ms/request) ÷ (instance capacity × utilization)
|
||||
|
||||
**Pattern 3: Staffing/Headcount**
|
||||
- **Decomposition**: Work to be done (features, tickets, customers) → Productivity per person → Overhead (meetings, support)
|
||||
- **Anchors**: Industry benchmarks (engineer per X users, support agent per Y customers), team velocity, hiring timelines
|
||||
- **Bounds**: Experienced team (high productivity) vs New team (ramp time), Aggressive timeline (crunch) vs Sustainable pace
|
||||
- **Sanity check**: Headcount growth should match revenue growth curve, compare to peers at similar scale
|
||||
- **Example**: Engineers needed = (Story points in roadmap ÷ Velocity per engineer) + 20% overhead
|
||||
|
||||
**Pattern 4: Financial Projections**
|
||||
- **Decomposition**: Revenue = Users × Conversion rate × ARPU, Costs = COGS + Sales/Marketing + R&D + G&A
|
||||
- **Anchors**: Cohort data, industry CAC/LTV benchmarks, comparable company metrics, historical growth
|
||||
- **Bounds**: Bull case (high growth, efficient scaling) vs Bear case (slow growth, rising costs)
|
||||
- **Sanity check**: Margins should approach industry norms at scale, growth rate should follow S-curve not exponential forever
|
||||
- **Example**: Year 2 revenue = Year 1 revenue × (1 + growth rate) × (1 - churn)
|
||||
|
||||
**Pattern 5: Impact Assessment**
|
||||
- **Decomposition**: Total impact = Units affected × Impact per unit × Duration
|
||||
- **Anchors**: Emission factors (kg CO2/kWh), conversion rates (program → behavior change), precedent studies
|
||||
- **Bounds**: Conservative (low adoption, small effect) vs Optimistic (high adoption, large effect)
|
||||
- **Sanity check**: Impact should scale linearly or sub-linearly (diminishing returns), compare to similar interventions
|
||||
- **Example**: Carbon saved = (Users switching × Miles driven/year × Emissions/mile) - Baseline
|
||||
|
||||
## Guardrails
|
||||
|
||||
**Critical requirements:**
|
||||
|
||||
1. **State assumptions explicitly**: Every Fermi estimate rests on assumptions. Make them visible ("Assuming 250 workdays/year", "If conversion rate ~3%"). Allows others to challenge/refine. Unstated assumptions create false precision.
|
||||
|
||||
2. **Aim for order of magnitude, not precision**: Goal is 10^X, not X.XX. Round to 1-2 significant figures (50 not 47.3, 3M not 2,847,291). False precision wastes time and misleads. If decision needs precision, don't use Fermi—get real data.
|
||||
|
||||
3. **Decompose until components are estimable**: Break down until you reach quantities you can estimate from knowledge/experience. If a component is still "how would I know that?", decompose further. Avoid plugging in wild guesses for complex sub-problems.
|
||||
|
||||
4. **Use multiple paths (triangulation)**: Estimate same quantity via different decompositions (top-down vs bottom-up, supply-side vs demand-side). If paths agree within factor of 3, confidence increases. If they differ by 10x+, investigate which decomposition is flawed.
|
||||
|
||||
5. **Bound the answer**: Calculate optimistic and pessimistic cases to bracket reality. If decision is same across range (market is $1B or $10B, either way we enter), bounds matter less. If decision flips (profitable at $10M, not at $1M), need precision or better estimate.
|
||||
|
||||
6. **Sanity-check against reality**: Does answer pass smell test? Compare to known quantities (your estimate for Starbucks revenue should be within 10x of actual ~$35B). Use dimensional analysis (units should cancel correctly). Check extreme cases (what if everyone did X? does it break physics?).
|
||||
|
||||
7. **Calibrate on known problems**: Practice on questions with verifiable answers (US population, Disney World attendance, wheat production). Identify your biases (overestimate? underestimate? anchoring?). Improves future estimates.
|
||||
|
||||
8. **Acknowledge uncertainty ranges**: Express estimates as ranges or confidence intervals when appropriate ("10-100k users", "likely $1-5M"). Communicates epistemic humility. Avoids false precision trap.
|
||||
|
||||
**Common pitfalls:**
|
||||
|
||||
- ❌ **Anchoring on the wrong number**: Using irrelevant or biased starting point. If someone says "Is it 1 million?" you anchor there even if no reason to.
|
||||
- ❌ **Double-counting**: Including same quantity twice in decomposition (counting both businesses and employees when businesses already includes employees).
|
||||
- ❌ **Unit errors**: Mixing per-day and per-year, confusing millions and billions, wrong currency conversion. Always check units.
|
||||
- ❌ **Survivor bias**: Estimating based on successful cases (average startup revenue from unicorns, not including failures).
|
||||
- ❌ **Linear extrapolation**: Assuming linear growth when exponential (or vice versa). Growth rates change over time.
|
||||
- ❌ **Ignoring constraints**: Physical limits (can't exceed speed of light), economic limits (market can't grow faster than GDP forever).
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**Key resources:**
|
||||
|
||||
- **[resources/template.md](resources/template.md)**: Clarification framework, decomposition strategies, estimation template, sanity-check criteria
|
||||
- **[resources/methodology.md](resources/methodology.md)**: Anchoring techniques, bounding methods, triangulation approaches, calibration exercises
|
||||
- **[resources/evaluators/rubric_estimation_fermi.json](resources/evaluators/rubric_estimation_fermi.json)**: Quality criteria for decomposition, assumptions, bounds, sanity checks
|
||||
|
||||
**Common Anchors:**
|
||||
|
||||
**Demographics:**
|
||||
- US population: ~330M, Households: ~130M, Labor force: ~165M
|
||||
- World population: ~8B, Urban: ~55%, Internet users: ~5B
|
||||
|
||||
**Business:**
|
||||
- Fortune 500 revenue: $100k to $600B (median ~$30B)
|
||||
- Startup valuations: Seed ~$5-10M, Series A ~$30-50M, Unicorn >$1B
|
||||
- SaaS metrics: CAC ~$1-5k, LTV/CAC ratio >3, Churn <5%/year
|
||||
|
||||
**Technology:**
|
||||
- AWS EC2 instance: ~10k requests/sec, S3 storage: $0.023/GB/month
|
||||
- Mobile app: ~5-10 screens/day per user, 50-100 API calls/session
|
||||
- Website: ~2-3 pages/session, 1-2min session duration
|
||||
|
||||
**Physical:**
|
||||
- Person: ~70kg, 2000 kcal/day, 8 hours sleep
|
||||
- Car: ~25 mpg, 12k miles/year, $30k new, 200k mile lifetime
|
||||
- House: ~2000 sq ft, $300k median US, 30-year mortgage
|
||||
|
||||
**Conversion factors:**
|
||||
- 1 year ≈ 250 workdays ≈ 2000 work hours
|
||||
- 1 million seconds ≈ 11.5 days, 1 billion seconds ≈ 32 years
|
||||
- 1 mile ≈ 1.6 km, 1 kg ≈ 2.2 lbs, 1 gallon ≈ 3.8 liters
|
||||
|
||||
**Decomposition Strategies:**
|
||||
|
||||
- **Top-down**: Start with total population, filter down (US population → Car owners → EV buyers)
|
||||
- **Bottom-up**: Start with unit, scale up (1 store revenue × Number of stores)
|
||||
- **Rate × Time**: Flow rate × Duration (Customers/day × Days/year)
|
||||
- **Density × Area/Volume**: Concentration × Space (People/sq mile × City area)
|
||||
- **Analogous scaling**: Known similar system, adjust for size (Competitor revenue × Our market share)
|
||||
|
||||
**Typical estimation time:**
|
||||
- Simple question (1-2 levels of decomposition): 3-5 minutes
|
||||
- Market sizing (3-4 levels): 10-15 minutes
|
||||
- Complex business case (multiple metrics, triangulation): 20-30 minutes
|
||||
|
||||
**When to escalate:**
|
||||
|
||||
- Decision requires precision (< factor of 2 uncertainty)
|
||||
- Estimate spans >2 orders of magnitude even with bounds
|
||||
- No reasonable decomposition path (too many unknowns)
|
||||
- Stakeholders need confidence intervals and statistical rigor
|
||||
→ Invest in data collection, detailed modeling, expert consultation
|
||||
|
||||
**Inputs required:**
|
||||
|
||||
- **Question** (what are we estimating? units? scope?)
|
||||
- **Decision context** (what decision hinges on this estimate? required precision?)
|
||||
- **Known anchors** (what related quantities do we know?)
|
||||
|
||||
**Outputs produced:**
|
||||
|
||||
- `estimation-fermi.md`: Question, decomposition, assumptions, calculation, bounds, sanity check, triangulation, final estimate with confidence range
|
||||
@@ -0,0 +1,256 @@
|
||||
{
|
||||
"criteria": [
|
||||
{
|
||||
"name": "Question Clarification",
|
||||
"1": "Question vague, units missing, scope undefined, decision context unclear",
|
||||
"3": "Question restated with units, scope bounded, decision context mentioned",
|
||||
"5": "Question precise and unambiguous, units specified, scope clearly defined, decision context and required precision explicit"
|
||||
},
|
||||
{
|
||||
"name": "Decomposition Quality",
|
||||
"1": "No decomposition or single wild guess, components not estimable, unclear how they combine",
|
||||
"3": "Logical decomposition into 2-3 levels, components mostly estimable, formula clear",
|
||||
"5": "Elegant decomposition (3-5 levels), all components independently estimable from knowledge/experience, path from components to answer transparent"
|
||||
},
|
||||
{
|
||||
"name": "Assumptions Explicit",
|
||||
"1": "Assumptions unstated, black-box numbers with no justification, cannot be challenged",
|
||||
"3": "Major assumptions stated, some justification provided, most can be identified and questioned",
|
||||
"5": "All assumptions explicitly stated, fully justified with anchors/sources, sensitivity noted, easily challenged and refined"
|
||||
},
|
||||
{
|
||||
"name": "Anchoring",
|
||||
"1": "Components based on guesses not anchored in any known quantities, no sources cited",
|
||||
"3": "Most components anchored in known data (population, benchmarks, personal experience), some sources mentioned",
|
||||
"5": "All components grounded in credible anchors (data, benchmarks, analogies, first principles), sources cited, confidence levels assessed"
|
||||
},
|
||||
{
|
||||
"name": "Bounding",
|
||||
"1": "No bounds provided, single point estimate only, no sense of uncertainty range",
|
||||
"3": "Upper and lower bounds calculated, range assessed (factor of X), some scenario analysis",
|
||||
"5": "Rigorous bounds via optimistic/pessimistic scenarios, range quantified, decision sensitivity assessed (does decision change across range?), constraints checked"
|
||||
},
|
||||
{
|
||||
"name": "Calculation Correctness",
|
||||
"1": "Math errors, units don't match, formula incorrect, compounding mistakes",
|
||||
"3": "Math generally correct, units mostly consistent, formula works, minor errors possible",
|
||||
"5": "Math accurate, dimensional analysis verified (units cancel correctly), formula logic sound, calculation transparent and reproducible"
|
||||
},
|
||||
{
|
||||
"name": "Sanity Checking",
|
||||
"1": "No validation, answer not compared to reality, obvious errors not caught",
|
||||
"3": "Some sanity checks (order of magnitude comparison, gut check), major errors would be caught",
|
||||
"5": "Comprehensive validation (dimensional analysis, reality comparison, extreme case testing, derived metrics consistency, gut check), implausible results flagged and investigated"
|
||||
},
|
||||
{
|
||||
"name": "Triangulation",
|
||||
"1": "Single approach only, no alternate path, cannot validate estimate",
|
||||
"3": "Attempted alternate decomposition, comparison made, discrepancies noted",
|
||||
"5": "Multiple independent paths (top-down vs bottom-up, supply vs demand), estimates within factor of 3, reconciliation of differences, confidence increased by agreement"
|
||||
},
|
||||
{
|
||||
"name": "Precision Appropriate",
|
||||
"1": "False precision (8.372M when uncertainty is ±10×), or uselessly vague (\"a lot\"), wrong significant figures",
|
||||
"3": "Rounded to 1-2 significant figures, order of magnitude clear, some uncertainty acknowledged",
|
||||
"5": "Precision matches uncertainty (1 sig fig for ±3×, 2 sig figs for ±30%), expressed as range when appropriate, confidence intervals calibrated, avoids false precision trap"
|
||||
},
|
||||
{
|
||||
"name": "Decision Actionability",
|
||||
"1": "Estimate disconnected from decision, no recommendation, unclear how to use result",
|
||||
"3": "Connection to decision mentioned, some implication for action, directionally useful",
|
||||
"5": "Clear decision implication (go/no-go, prioritize/deprioritize), threshold analysis (if >X then Y), sensitivity to key assumptions identified, actionable recommendation based on estimate and uncertainty"
|
||||
}
|
||||
],
|
||||
"guidance_by_type": {
|
||||
"Market Sizing (TAM/SAM/SOM)": {
|
||||
"target_score": 4.0,
|
||||
"key_requirements": [
|
||||
"Top-down decomposition (population → filters → addressable → price), components estimable from census/industry data",
|
||||
"Anchors: Population figures, market penetration rates, pricing from comparables, industry reports",
|
||||
"Bounds: Optimistic (high penetration, premium price) vs Pessimistic (low penetration, discount price)",
|
||||
"Triangulation: Cross-check against public company revenues in space, VC market estimates, bottom-up from customer count",
|
||||
"Sanity check: Compare to GDP (market can't exceed consumer spending), check per-capita figures"
|
||||
],
|
||||
"common_pitfalls": [
|
||||
"Confusing TAM (total addressable) with SAM (serviceable) or SOM (obtainable)",
|
||||
"Overestimating willingness to pay (assume most won't pay)",
|
||||
"Not accounting for competition (you won't get 100% share)"
|
||||
]
|
||||
},
|
||||
"Infrastructure Capacity": {
|
||||
"target_score": 4.1,
|
||||
"key_requirements": [
|
||||
"Decomposition: Users → Actions per user → Resources per action → Overhead/utilization",
|
||||
"Anchors: Similar systems (Instagram scale), known limits (AWS instance capacity), load testing data, utilization factors (70-80% not 100%)",
|
||||
"Bounds: Peak load (Black Friday, viral event) vs Average, Growth trajectory (2× vs 10× per year)",
|
||||
"Triangulation: Top-down from users vs Bottom-up from server capacity, cost validation (does $/user make sense?)",
|
||||
"Sanity check: Cost per user < LTV, compare to public cloud bills of similar companies"
|
||||
],
|
||||
"common_pitfalls": [
|
||||
"Assuming 100% utilization (real systems: 70-80% for headroom)",
|
||||
"Forgetting overhead (databases, load balancers, redundancy)",
|
||||
"Linear scaling assumptions (ignoring caching, batching gains)"
|
||||
]
|
||||
},
|
||||
"Financial Projections": {
|
||||
"target_score": 3.9,
|
||||
"key_requirements": [
|
||||
"Decomposition: Revenue (customers × conversion × ARPU), Costs (COGS + CAC + R&D + G&A)",
|
||||
"Anchors: Cohort data (historical conversion), industry benchmarks (CAC/LTV ratios, SaaS metrics), comparable company metrics",
|
||||
"Bounds: Bull case (high growth, efficient scaling) vs Bear case (slow growth, rising costs)",
|
||||
"Triangulation: Build cohort model vs top-down market share model, cross-check margins with industry",
|
||||
"Sanity check: Growth follows S-curve not exponential forever, margins approach industry norms at scale, rule of 40"
|
||||
],
|
||||
"common_pitfalls": [
|
||||
"Assuming exponential growth continues indefinitely",
|
||||
"Not accounting for churn (especially in SaaS)",
|
||||
"Ignoring seasonality or cyclicality"
|
||||
]
|
||||
},
|
||||
"Resource Planning (Headcount/Budget)": {
|
||||
"target_score": 4.0,
|
||||
"key_requirements": [
|
||||
"Decomposition: Work to be done (features, tickets, customers) → Productivity per person → Overhead (meetings, vacation, ramp)",
|
||||
"Anchors: Team velocity (story points/sprint), industry ratios (support agents per 1000 customers), hiring timelines",
|
||||
"Bounds: Experienced team (high productivity, low ramp) vs New team (learning curve, attrition)",
|
||||
"Triangulation: Bottom-up from roadmap vs Top-down from revenue per employee benchmark",
|
||||
"Sanity check: Headcount vs revenue growth (should correlate), compare to peer companies at similar scale"
|
||||
],
|
||||
"common_pitfalls": [
|
||||
"Not accounting for ramp time (new hires take 3-6 months to full productivity)",
|
||||
"Ignoring overhead (meetings, hiring, training consume 20-30% of time)",
|
||||
"Underestimating hiring pipeline (offer to start date 1-3 months)"
|
||||
]
|
||||
},
|
||||
"Impact Assessment": {
|
||||
"target_score": 3.8,
|
||||
"key_requirements": [
|
||||
"Decomposition: Total impact = Units affected × Impact per unit × Duration, account for baseline and counterfactual",
|
||||
"Anchors: Emission factors (kg CO2/kWh), conversion rates (program → behavior change), precedent studies with measured effects",
|
||||
"Bounds: Conservative (low adoption, small effect) vs Optimistic (high adoption, large effect)",
|
||||
"Triangulation: Top-down (total population × penetration) vs Bottom-up (measured cases × scale factor)",
|
||||
"Sanity check: Impact should scale linearly or sub-linearly (diminishing returns), compare to similar interventions"
|
||||
],
|
||||
"common_pitfalls": [
|
||||
"Not accounting for counterfactual (what would have happened anyway)",
|
||||
"Ignoring adoption rates (not everyone participates)",
|
||||
"Linear extrapolation when effects have diminishing returns"
|
||||
]
|
||||
}
|
||||
},
|
||||
"guidance_by_complexity": {
|
||||
"Simple Question (1-2 levels)": {
|
||||
"target_score": 3.5,
|
||||
"description": "Single decomposition (A × B), components directly estimable, low uncertainty",
|
||||
"key_requirements": [
|
||||
"Clear decomposition (2-3 components), formula transparent",
|
||||
"Components anchored in known quantities or personal experience",
|
||||
"Basic bounds (±2-3×), sanity check against reality",
|
||||
"Single approach sufficient if well-validated"
|
||||
],
|
||||
"time_estimate": "3-5 minutes",
|
||||
"examples": [
|
||||
"Annual revenue from daily revenue (revenue/day × 365)",
|
||||
"Customers served per year (customers/day × 250 workdays)",
|
||||
"Storage needed (users × data per user)"
|
||||
]
|
||||
},
|
||||
"Moderate Question (3-4 levels)": {
|
||||
"target_score": 4.0,
|
||||
"description": "Multi-level decomposition, some uncertainty in components, decision depends on order of magnitude",
|
||||
"key_requirements": [
|
||||
"Logical 3-4 level decomposition, all components independently estimable",
|
||||
"Assumptions explicit, anchored in data or benchmarks",
|
||||
"Bounds calculated (optimistic/pessimistic scenarios), range assessed",
|
||||
"Triangulation via alternate path, estimates within factor of 3",
|
||||
"Comprehensive sanity checks (units, reality comparison, derived metrics)"
|
||||
],
|
||||
"time_estimate": "10-15 minutes",
|
||||
"examples": [
|
||||
"Market sizing (population → segment → addressable → price)",
|
||||
"Server capacity needed (users → requests → compute)",
|
||||
"Headcount planning (work → productivity → overhead)"
|
||||
]
|
||||
},
|
||||
"Complex Question (5+ levels)": {
|
||||
"target_score": 4.3,
|
||||
"description": "Deep decomposition tree, high uncertainty, decision sensitive to assumptions, requires triangulation",
|
||||
"key_requirements": [
|
||||
"Sophisticated decomposition (5+ levels or multiple parallel paths), components validated independently",
|
||||
"All assumptions stated and justified, sensitivity analysis (which matter most?)",
|
||||
"Rigorous bounding (scenario analysis, constraint checking), decision sensitivity assessed",
|
||||
"Multiple triangulation paths (top-down/bottom-up, supply/demand), cross-validation with public data",
|
||||
"Extensive sanity checking (dimensional analysis, extreme cases, internal consistency)",
|
||||
"Uncertainty quantified (confidence intervals), precision matched to uncertainty"
|
||||
],
|
||||
"time_estimate": "20-30 minutes",
|
||||
"examples": [
|
||||
"Multi-year financial model (revenue streams × growth × costs × scenarios)",
|
||||
"Climate impact assessment (emissions × multiple sources × counterfactual × time horizon)",
|
||||
"Large infrastructure sizing (users × behavior × compute × storage × network × redundancy)"
|
||||
]
|
||||
}
|
||||
},
|
||||
"common_failure_modes": [
|
||||
{
|
||||
"failure": "Missing units or unit errors",
|
||||
"symptom": "Mixing per-day with per-year, confusing millions and billions, currency not specified, units don't cancel in formula",
|
||||
"detection": "Dimensional analysis fails (units don't match), or derived metrics nonsensical (revenue per employee = $5)",
|
||||
"fix": "Always write units next to every number, verify dimensional analysis (units cancel to expected final unit), convert all to common time basis before combining"
|
||||
},
|
||||
{
|
||||
"failure": "Unstated assumptions",
|
||||
"symptom": "Numbers appear without justification, reader cannot challenge or refine estimate, black-box calculation",
|
||||
"detection": "Ask 'Why this number?' and no answer is provided in documentation",
|
||||
"fix": "For each component, explicitly state assumption and anchor (e.g., 'Assuming 250 workdays/year', 'Based on industry benchmark of 3% conversion')"
|
||||
},
|
||||
{
|
||||
"failure": "False precision",
|
||||
"symptom": "8.372M when uncertainty is ±5×, or $47,293 when components are rough guesses",
|
||||
"detection": "More than 2 significant figures, or precision implies certainty mismatched to actual uncertainty",
|
||||
"fix": "Round to 1-2 significant figures matching uncertainty (±3× → 1 sig fig, ±30% → 2 sig figs), express as range when uncertainty high"
|
||||
},
|
||||
{
|
||||
"failure": "No bounds or sanity checks",
|
||||
"symptom": "Single point estimate, no validation, implausible result not caught (market size exceeds GDP, growth >100%/year sustained)",
|
||||
"detection": "Estimate seems wrong intuitively but no mechanism to validate, or obviously violates constraints",
|
||||
"fix": "Always calculate bounds (optimistic/pessimistic), compare to known comparables, check extreme cases, verify doesn't violate physics/economics"
|
||||
},
|
||||
{
|
||||
"failure": "Decomposition not estimable",
|
||||
"symptom": "Components still too complex ('estimate market size' → 'estimate demand' is not progress), or circular (define A in terms of B, B in terms of A)",
|
||||
"detection": "Ask 'Can I estimate this component from knowledge/data?' If no, decomposition incomplete",
|
||||
"fix": "Decompose until each component answerable from: known data, personal experience, analogous comparison, or first principles"
|
||||
},
|
||||
{
|
||||
"failure": "Anchoring bias",
|
||||
"symptom": "Estimate heavily influenced by first number heard, even if irrelevant ('Is it $10M?' causes you to anchor near $10M)",
|
||||
"detection": "Estimate changes significantly based on arbitrary starting point provided",
|
||||
"fix": "Generate independent estimate before seeing any suggested numbers, then compare and justify any convergence"
|
||||
},
|
||||
{
|
||||
"failure": "Double-counting",
|
||||
"symptom": "Same quantity included twice in formula (counting businesses AND employees when businesses already includes employee count)",
|
||||
"detection": "Draw decomposition tree visually - if same box appears twice in different branches, likely double-counted",
|
||||
"fix": "Clearly define what each component includes and excludes, ensure mutually exclusive and collectively exhaustive (MECE)"
|
||||
},
|
||||
{
|
||||
"failure": "No triangulation",
|
||||
"symptom": "Single path only, cannot validate estimate, reader must trust decomposition is correct",
|
||||
"detection": "Only one approach provided, no alternate path or comparison to known data",
|
||||
"fix": "Re-estimate via different decomposition (top-down vs bottom-up, supply vs demand), check if within factor of 3, cross-validate with public data when available"
|
||||
},
|
||||
{
|
||||
"failure": "Ignoring utilization/efficiency",
|
||||
"symptom": "Assuming 100% capacity (servers always at max, people work 40hr with zero meetings/vacation, factories run 24/7 with no downtime)",
|
||||
"detection": "Capacity estimates seem too high compared to real systems",
|
||||
"fix": "Apply realistic utilization factors (servers 70-80%, people 60-70% after meetings/overhead, factories 80-90% after maintenance)"
|
||||
},
|
||||
{
|
||||
"failure": "Linear extrapolation errors",
|
||||
"symptom": "Assuming linear when exponential (tech adoption), or exponential when logistic (growth eventually saturates)",
|
||||
"detection": "Projections violate constraints (market share >100%, growth continues at 100%/year for decades)",
|
||||
"fix": "Check if relationship is truly linear, apply appropriate growth curve (exponential for early adoption, S-curve for saturation, linear for steady-state)"
|
||||
}
|
||||
]
|
||||
}
|
||||
398
skills/estimation-fermi/resources/methodology.md
Normal file
398
skills/estimation-fermi/resources/methodology.md
Normal file
@@ -0,0 +1,398 @@
|
||||
# Fermi Estimation Methodology
|
||||
|
||||
Advanced techniques for anchoring, bounding, triangulation, and calibration.
|
||||
|
||||
## Workflow
|
||||
|
||||
```
|
||||
Fermi Estimation Progress:
|
||||
- [ ] Step 1: Clarify the question and define metric
|
||||
- [ ] Step 2: Decompose into estimable components
|
||||
- [ ] Step 3: Estimate components using anchors
|
||||
- [ ] Step 4: Bound with upper/lower limits
|
||||
- [ ] Step 5: Calculate and sanity-check
|
||||
- [ ] Step 6: Triangulate with alternate path
|
||||
```
|
||||
|
||||
**Step 1: Clarify the question and define metric**
|
||||
|
||||
Define scope, units, decision context before decomposition.
|
||||
|
||||
**Step 2: Decompose into estimable components**
|
||||
|
||||
Choose decomposition strategy based on problem structure and available knowledge.
|
||||
|
||||
**Step 3: Estimate components using anchors**
|
||||
|
||||
Apply [1. Anchoring Techniques](#1-anchoring-techniques) to ground estimates in known quantities.
|
||||
|
||||
**Step 4: Bound with upper/lower limits**
|
||||
|
||||
Use [2. Bounding Techniques](#2-bounding-techniques) to bracket answer with constraints.
|
||||
|
||||
**Step 5: Calculate and sanity-check**
|
||||
|
||||
Validate using dimensional analysis, reality checks, and [3. Calibration Methods](#3-calibration-methods).
|
||||
|
||||
**Step 6: Triangulate with alternate path**
|
||||
|
||||
Apply [4. Triangulation Approaches](#4-triangulation-approaches) to estimate via different decomposition.
|
||||
|
||||
---
|
||||
|
||||
## 1. Anchoring Techniques
|
||||
|
||||
Methods for grounding component estimates in known quantities.
|
||||
|
||||
### Common Knowledge Anchors
|
||||
|
||||
**Demographics**: Population figures, household counts, labor force
|
||||
- US population: ~330M (2020s), grows ~0.5%/year
|
||||
- US households: ~130M, avg 2.5 people/household
|
||||
- US labor force: ~165M, unemployment typically 3-6%
|
||||
- Global: ~8B people, ~2B households, ~55% urban
|
||||
|
||||
**Economic**: GDP, spending, income
|
||||
- US GDP: ~$25T, per capita ~$75k
|
||||
- Median household income: ~$70k
|
||||
- Consumer spending: ~70% of GDP
|
||||
- Federal budget: ~$6T (~24% of GDP)
|
||||
|
||||
**Physical constants**: Time, space, energy
|
||||
- Earth: ~510M km², ~70% water, ~150M km² land
|
||||
- US land: ~10M km² (~3.8M sq mi)
|
||||
- Day = 24 hours, year ≈ 365 days ≈ 52 weeks
|
||||
- Typical human: 70kg, 2000 kcal/day, 8hr sleep, 16hr awake
|
||||
|
||||
**Business benchmarks**: SaaS metrics, retail, tech
|
||||
- SaaS ARR per employee: $150-300k (mature companies)
|
||||
- Retail revenue per sq ft: $300-500/year (varies by category)
|
||||
- Tech company valuation: 5-15× revenue (growth stage), 2-5× (mature)
|
||||
- CAC payback: <12 months good, <18 acceptable
|
||||
|
||||
### Data Lookup Strategies
|
||||
|
||||
**Quick reliable sources** (use for anchoring):
|
||||
- Google: Population figures, company sizes, market data
|
||||
- Wikipedia: Demographics, economic stats, physical data
|
||||
- Public company filings: Revenue, employees, customers (10-K, investor decks)
|
||||
- Industry reports: Gartner, Forrester, McKinsey (market sizing)
|
||||
|
||||
**Estimation from fragments**:
|
||||
- Company has "thousands of employees" → Estimate 3,000-5,000
|
||||
- Market is "multi-billion dollar" → Estimate $2-9B
|
||||
- "Most people" do X → Estimate 60-80%
|
||||
- "Rare" occurrence → Estimate <5%
|
||||
|
||||
### Personal Experience Anchors
|
||||
|
||||
**Observation-based**: Use your own data points
|
||||
- How many apps on your phone? (extrapolate to avg user)
|
||||
- How often do you buy X? (extrapolate to market)
|
||||
- How long does task Y take? (estimate productivity)
|
||||
|
||||
**Analogous reasoning**: Scale from known to unknown
|
||||
- If you know NYC subway ridership, estimate SF BART by population ratio
|
||||
- If you know your company's churn, estimate competitor's by industry norms
|
||||
- If you know local restaurant count, estimate city-wide by neighborhood scaling
|
||||
|
||||
**Bracketing intuition**: Use confidence ranges
|
||||
- "I'm 80% confident the answer is between X and Y"
|
||||
- Then use geometric mean: √(X×Y) as central estimate
|
||||
- Example: Starbucks locations - probably between 10k and 50k → ~22k (actual ~16k)
|
||||
|
||||
---
|
||||
|
||||
## 2. Bounding Techniques
|
||||
|
||||
Methods for calculating upper/lower limits to bracket the answer.
|
||||
|
||||
### Constraint-Based Bounding
|
||||
|
||||
**Physical constraints**: Cannot exceed laws of nature
|
||||
- Maximum speed: Speed of light (for data), sound (for physical travel), human limits (for manual tasks)
|
||||
- Maximum density: People per area (fire code limits, standing room), data per volume (storage media)
|
||||
- Maximum efficiency: Thermodynamic limits, conversion efficiency (solar ~20-25%, combustion ~35-45%)
|
||||
|
||||
**Economic constraints**: Cannot exceed available resources
|
||||
- Market size bounded by: GDP, consumer spending, addressable population × willingness to pay
|
||||
- Company revenue bounded by: Market size × maximum possible share (~20-30% typically)
|
||||
- Headcount bounded by: Budget ÷ avg salary, or revenue ÷ revenue per employee benchmark
|
||||
|
||||
**Time constraints**: Cannot exceed available hours
|
||||
- Work output bounded by: People × hours/person × productivity
|
||||
- Example: "How many customers can support team handle?" → Agents × (40 hr/week - 10hr meetings) × tickets/hr
|
||||
|
||||
### Scenario-Based Bounding
|
||||
|
||||
**Optimistic scenario** (favorable assumptions):
|
||||
- High adoption (80-100% of addressable market)
|
||||
- Premium pricing (top quartile of range)
|
||||
- High efficiency (best-in-class productivity)
|
||||
- Fast growth (aggressive but plausible)
|
||||
|
||||
**Pessimistic scenario** (conservative assumptions):
|
||||
- Low adoption (5-20% of addressable market)
|
||||
- Discount pricing (bottom quartile of range)
|
||||
- Low efficiency (industry average or below)
|
||||
- Slow growth (cautious, accounts for setbacks)
|
||||
|
||||
**Example - Market sizing**:
|
||||
- Optimistic: All 500k target businesses × $100/month × 12 = $600M
|
||||
- Pessimistic: 5% penetration (25k) × $30/month × 12 = $9M
|
||||
- Range: $9M - $600M (67× span → likely too wide, refine assumptions)
|
||||
|
||||
### Sensitivity Analysis
|
||||
|
||||
Identify which assumptions most affect result:
|
||||
|
||||
**Method**: Vary each component ±50%, measure impact on final estimate
|
||||
|
||||
**High sensitivity components**: Small change → large impact on result
|
||||
- Focus refinement effort here
|
||||
- Example: If CAC varies 50% and final ROI changes 40%, CAC is high sensitivity
|
||||
|
||||
**Low sensitivity components**: Large change → small impact on result
|
||||
- Less critical to get precise
|
||||
- Example: If office rent varies 50% but total costs change 5%, rent is low sensitivity
|
||||
|
||||
**Rule of thumb**: 80% of uncertainty often comes from 20% of assumptions. Find and refine those critical few.
|
||||
|
||||
---
|
||||
|
||||
## 3. Calibration Methods
|
||||
|
||||
Techniques for improving estimation accuracy through practice and feedback.
|
||||
|
||||
### Calibration Exercises
|
||||
|
||||
**Known problems**: Practice on verifiable questions, compare estimate to actual
|
||||
|
||||
**Exercise set 1 - Demographics**:
|
||||
1. US population in 1950? (Estimate, then check: ~150M)
|
||||
2. Number of countries in UN? (Estimate, then check: ~190)
|
||||
3. World's tallest building height? (Estimate, then check: ~830m Burj Khalifa)
|
||||
|
||||
**Exercise set 2 - Business**:
|
||||
1. Starbucks annual revenue? (Estimate, then check: ~$35B)
|
||||
2. Google employees worldwide? (Estimate, then check: ~150k)
|
||||
3. Average Netflix subscription price? (Estimate, then check: ~$15/month)
|
||||
|
||||
**Exercise set 3 - Consulting classics**:
|
||||
1. Piano tuners in Chicago? (Estimate, then check: ~100)
|
||||
2. Gas stations in US? (Estimate, then check: ~150k)
|
||||
3. Golf balls fitting in school bus? (Estimate, then check: ~500k)
|
||||
|
||||
**Feedback loop**:
|
||||
- Track your estimate vs actual ratio
|
||||
- If consistently >3× off, identify bias (underestimate? overestimate?)
|
||||
- Calibrate future estimates (if you typically 2× low, mentally adjust upward)
|
||||
|
||||
### Confidence Intervals
|
||||
|
||||
Express uncertainty quantitatively:
|
||||
|
||||
**90% confidence interval**: "I'm 90% sure the answer is between X and Y"
|
||||
- Width reflects uncertainty (narrow = confident, wide = uncertain)
|
||||
- Calibration: Of 10 estimates with 90% CI, ~9 should contain true value
|
||||
|
||||
**Common mistakes**:
|
||||
- Too narrow (overconfidence): Only 50% of your "90% CIs" contain true value
|
||||
- Too wide (useless): All your CIs span 3+ orders of magnitude
|
||||
- Goal: Calibrated such that X% of your X% CIs are correct
|
||||
|
||||
**Practice calibration**:
|
||||
1. Make 20 estimates with 80% CIs
|
||||
2. Check how many actually contained true value
|
||||
3. If <16 (80% of 20), you're overconfident → widen future CIs
|
||||
4. If >18, you're underconfident → narrow future CIs
|
||||
|
||||
### Bias Identification
|
||||
|
||||
**Anchoring bias**: Over-relying on first number you hear
|
||||
- Mitigation: Generate estimate independently before seeing any numbers
|
||||
- Test: Estimate revenue before someone says "Is it $10M?" vs after
|
||||
|
||||
**Availability bias**: Overweighting recent/memorable events
|
||||
- Mitigation: Seek base rates, historical averages, not just recent headline cases
|
||||
- Example: Don't estimate startup success rate from TechCrunch unicorn coverage
|
||||
|
||||
**Optimism bias**: Tendency to assume favorable outcomes
|
||||
- Mitigation: Explicitly calculate pessimistic scenario, force consideration of downsides
|
||||
- Particularly important for: Timelines, costs, adoption rates
|
||||
|
||||
**Unit confusion**: Mixing millions/billions, per-day/per-year
|
||||
- Mitigation: Always write units, check dimensional analysis
|
||||
- Example: Company earns $10M/year = ~$27k/day (sanity check: does that seem right for scale?)
|
||||
|
||||
---
|
||||
|
||||
## 4. Triangulation Approaches
|
||||
|
||||
Estimating the same quantity via multiple independent paths to validate.
|
||||
|
||||
### Supply-Side vs Demand-Side
|
||||
|
||||
**Supply-side**: Count producers/capacity, estimate output
|
||||
- Example (Uber drivers in SF):
|
||||
- ~5,000 drivers (estimated from driver forum activity, Uber PR)
|
||||
- ~30 rides/day per driver
|
||||
- = 150,000 rides/day in SF
|
||||
|
||||
**Demand-side**: Count consumers/need, estimate consumption
|
||||
- Example (Uber rides in SF):
|
||||
- ~900k population
|
||||
- ~5% use Uber daily (frequent users)
|
||||
- ~3 rides per user-day
|
||||
- = 135,000 rides/day in SF
|
||||
|
||||
**Triangulation**: 150k (supply) vs 135k (demand) → Within 10%, confidence high
|
||||
|
||||
### Top-Down vs Bottom-Up
|
||||
|
||||
**Top-down**: Start with large total, filter down
|
||||
- Example (Restaurant revenue in city):
|
||||
- 1M population
|
||||
- ~$8k/person annual food spending
|
||||
- ~40% spent at restaurants
|
||||
- = $3.2B restaurant revenue
|
||||
|
||||
**Bottom-up**: Start with unit, scale up
|
||||
- Example (Restaurant revenue in city):
|
||||
- ~1,500 restaurants
|
||||
- ~50 meals/day per restaurant
|
||||
- ~$25 average check
|
||||
- ~350 days/year
|
||||
- = ~$650M restaurant revenue
|
||||
|
||||
**Discrepancy**: $3.2B vs $650M → 5× difference, investigate!
|
||||
- Possible explanations: Underestimated restaurants (chains, cafes, food trucks), or overestimated per-capita spending
|
||||
|
||||
### Multiple Decomposition Paths
|
||||
|
||||
**Example - Estimating AWS revenue**:
|
||||
|
||||
**Path 1 - Customer-based**:
|
||||
- ~1M active AWS customers (public statements)
|
||||
- Average spend ~$10k/year (mix of startups $1k, enterprises $100k+)
|
||||
- = $10B revenue
|
||||
|
||||
**Path 2 - Workload-based**:
|
||||
- ~5M EC2 instances running globally (estimated from public data)
|
||||
- ~$500/month avg per instance (mix of small/large)
|
||||
- = $30B revenue
|
||||
|
||||
**Path 3 - Market share**:
|
||||
- Cloud infrastructure market ~$200B (2023)
|
||||
- AWS market share ~30%
|
||||
- = $60B revenue
|
||||
|
||||
**Triangulation**: $10B vs $30B vs $60B → Within same order of magnitude, actual AWS revenue ~$85B (2023)
|
||||
- All paths got within 3-10× (reasonable for Fermi), but spread suggests different assumptions need refinement
|
||||
|
||||
### Cross-Validation with Public Data
|
||||
|
||||
After Fermi estimate, quickly check if public data exists:
|
||||
|
||||
**Public company financials**: 10-K filings, investor presentations
|
||||
- Validate: Revenue, employees, customers, margins
|
||||
|
||||
**Industry reports**: Gartner, IDC, Forrester market sizing
|
||||
- Validate: TAM, growth rates, share
|
||||
|
||||
**Government data**: Census, BLS, FDA, EPA
|
||||
- Validate: Population, employment, health, environment figures
|
||||
|
||||
**Academic studies**: Research papers on adoption, behavior, impact
|
||||
- Validate: Penetration rates, usage patterns, effect sizes
|
||||
|
||||
**Purpose**: Not to replace Fermi process, but to:
|
||||
1. Calibrate (am I within 10×? If not, what's wrong?)
|
||||
2. Refine (can I improve assumptions with real data?)
|
||||
3. Build confidence (multiple paths + public data all agree → high confidence)
|
||||
|
||||
---
|
||||
|
||||
## 5. Advanced Decomposition Patterns
|
||||
|
||||
### Conversion Funnel Decomposition
|
||||
|
||||
For estimating outcomes in multi-step processes:
|
||||
|
||||
**Structure**:
|
||||
```
|
||||
Starting population
|
||||
× Conversion 1 (% that complete step 1)
|
||||
× Conversion 2 (% that complete step 2 | completed 1)
|
||||
× Conversion 3 (% that complete step 3 | completed 2)
|
||||
= Final outcome
|
||||
```
|
||||
|
||||
**Example - SaaS conversions**:
|
||||
```
|
||||
Website visitors: 100k/month
|
||||
× Trial signup: 5% = 5k trials
|
||||
× Activation: 60% = 3k activated
|
||||
× Paid conversion: 20% = 600 new customers/month
|
||||
```
|
||||
|
||||
### Cohort-Based Decomposition
|
||||
|
||||
For estimating aggregate from groups with different characteristics:
|
||||
|
||||
**Example - App revenue**:
|
||||
```
|
||||
Cohort 1 (Power users): 10k users × $20/month = $200k
|
||||
Cohort 2 (Regular users): 100k users × $5/month = $500k
|
||||
Cohort 3 (Free users): 1M users × $0 = $0
|
||||
Total: $700k/month
|
||||
```
|
||||
|
||||
### Dimensional Analysis
|
||||
|
||||
Using units to guide decomposition:
|
||||
|
||||
**Example**: Estimating data center power consumption (want kW)
|
||||
```
|
||||
Servers (count)
|
||||
× Power per server (kW/server)
|
||||
+ Cooling overhead (×1.5 for PUE)
|
||||
= Total power (kW)
|
||||
```
|
||||
|
||||
Units guide you: Need kW, have servers → must find kW/server, which is estimable
|
||||
|
||||
---
|
||||
|
||||
## 6. Common Estimation Pitfalls
|
||||
|
||||
**Compounding errors**: Errors multiply in chains
|
||||
- 3 components each ±50% uncertain → final estimate ±300% uncertain
|
||||
- Mitigation: Keep decomposition shallow (3-5 levels max), validate high-sensitivity components
|
||||
|
||||
**False precision**: Reporting 8.372M when uncertainty is ±3×
|
||||
- Mitigation: Round to 1-2 significant figures (8M not 8.372M), express as range
|
||||
|
||||
**Linearity assumption**: Assuming proportional scaling when it's not
|
||||
- Reality: Economies of scale (costs grow sub-linearly), network effects (value grows super-linearly)
|
||||
- Mitigation: Check if relationship is truly linear or if power law applies
|
||||
|
||||
**Survivorship bias**: Estimating from successes only
|
||||
- Example: Average startup revenue based on unicorns (ignoring 90% that failed)
|
||||
- Mitigation: Include full distribution, weight by probability
|
||||
|
||||
**Time-period confusion**: Mixing annual, monthly, daily figures
|
||||
- Example: "Company earns $1M" - per year? per month? Need clarity.
|
||||
- Mitigation: Always specify time units, convert to common basis
|
||||
|
||||
**Outdated anchors**: Using pre-pandemic data for post-pandemic estimates
|
||||
- Example: Office occupancy, remote work adoption, e-commerce penetration all shifted
|
||||
- Mitigation: Check anchor recency, adjust for structural changes
|
||||
|
||||
**Ignoring constraints**: Estimates that violate physics/economics
|
||||
- Example: Market size exceeding GDP, growth rate >100%/year sustained indefinitely
|
||||
- Mitigation: Sanity-check against absolute limits
|
||||
|
||||
**Double-counting**: Including same quantity twice
|
||||
- Example: Counting both "businesses" and "employees" when businesses already includes employee count
|
||||
- Mitigation: Draw clear decomposition tree, check for overlap
|
||||
345
skills/estimation-fermi/resources/template.md
Normal file
345
skills/estimation-fermi/resources/template.md
Normal file
@@ -0,0 +1,345 @@
|
||||
# Fermi Estimation Templates
|
||||
|
||||
Quick-start templates for decomposition, component estimation, bounding, and sanity-checking.
|
||||
|
||||
## Workflow
|
||||
|
||||
```
|
||||
Fermi Estimation Progress:
|
||||
- [ ] Step 1: Clarify the question and define metric
|
||||
- [ ] Step 2: Decompose into estimable components
|
||||
- [ ] Step 3: Estimate components using anchors
|
||||
- [ ] Step 4: Bound with upper/lower limits
|
||||
- [ ] Step 5: Calculate and sanity-check
|
||||
- [ ] Step 6: Triangulate with alternate path
|
||||
```
|
||||
|
||||
**Step 1: Clarify the question and define metric**
|
||||
|
||||
Use [Clarification Template](#clarification-template) to restate question precisely with units, scope, timeframe, and decision context.
|
||||
|
||||
**Step 2: Decompose into estimable components**
|
||||
|
||||
Select decomposition strategy using [Decomposition Strategies](#decomposition-strategies) and build estimation tree.
|
||||
|
||||
**Step 3: Estimate components using anchors**
|
||||
|
||||
Ground each component in known quantities using [Component Estimation Template](#component-estimation-template).
|
||||
|
||||
**Step 4: Bound with upper/lower limits**
|
||||
|
||||
Calculate optimistic and pessimistic bounds using [Bounding Template](#bounding-template).
|
||||
|
||||
**Step 5: Calculate and sanity-check**
|
||||
|
||||
Compute central estimate and validate using [Sanity-Check Template](#sanity-check-template).
|
||||
|
||||
**Step 6: Triangulate with alternate path**
|
||||
|
||||
Re-estimate via different decomposition using [Triangulation Template](#triangulation-template) to validate.
|
||||
|
||||
---
|
||||
|
||||
## Clarification Template
|
||||
|
||||
**Original Question**: [As stated]
|
||||
|
||||
**Clarified Question**:
|
||||
- **What exactly are we estimating?** (metric name, definition)
|
||||
- **Units?** (dollars, people, tons, requests/sec, etc.)
|
||||
- **Geographic scope?** (US, global, city, region)
|
||||
- **Time scope?** (annual, daily, lifetime, point-in-time)
|
||||
- **Exclusions?** (what are we NOT counting?)
|
||||
|
||||
**Decision Context**:
|
||||
- **What decision depends on this estimate?**
|
||||
- **What precision is needed?** (order of magnitude sufficient? need ±2x? ±10%?)
|
||||
- **How will estimate be used?** (go/no-go decision, budget planning, prioritization)
|
||||
- **What threshold matters?** (if >X we do Y, if <Z we don't)
|
||||
|
||||
**Success Criteria**:
|
||||
- [ ] Question is unambiguous (two people would estimate same thing)
|
||||
- [ ] Units are specified
|
||||
- [ ] Scope is bounded (not infinite)
|
||||
- [ ] Decision context is clear
|
||||
|
||||
---
|
||||
|
||||
## Decomposition Strategies
|
||||
|
||||
Choose strategy based on question structure and available knowledge.
|
||||
|
||||
### Top-Down Filtering
|
||||
|
||||
**When to use**: Estimating subset of total population (market sizing, target audience)
|
||||
|
||||
**Structure**: Start with total, apply filters to narrow down
|
||||
|
||||
**Template**:
|
||||
```
|
||||
Total population
|
||||
× Filter 1 (% that have characteristic A)
|
||||
× Filter 2 (% that have characteristic B | A)
|
||||
× Filter 3 (% that have characteristic C | A,B)
|
||||
= Target population
|
||||
```
|
||||
|
||||
**Example - TAM estimation**:
|
||||
```
|
||||
US population (330M)
|
||||
× Adults 18-65 (65% = 215M)
|
||||
× Urban/suburban (80% = 172M)
|
||||
× Smartphone users (90% = 155M)
|
||||
× Willing to pay for app (5% = 7.75M)
|
||||
× Price point ($50/year)
|
||||
= TAM: ~$390M/year
|
||||
```
|
||||
|
||||
### Bottom-Up Scaling
|
||||
|
||||
**When to use**: Have data on units, want to estimate total (revenue, capacity, production)
|
||||
|
||||
**Structure**: Start with single unit, multiply by count
|
||||
|
||||
**Template**:
|
||||
```
|
||||
Output per unit
|
||||
× Number of units
|
||||
× Utilization rate (% of theoretical capacity actually used)
|
||||
= Total output
|
||||
```
|
||||
|
||||
**Example - Server capacity**:
|
||||
```
|
||||
1 server handles 10k requests/sec
|
||||
× 100 servers in cluster
|
||||
× 70% utilization (peak headroom, maintenance, failures)
|
||||
= Cluster capacity: 700k requests/sec
|
||||
```
|
||||
|
||||
### Rate × Time
|
||||
|
||||
**When to use**: Estimating accumulation over time (customers served, miles driven, content consumed)
|
||||
|
||||
**Structure**: Flow rate × Duration
|
||||
|
||||
**Template**:
|
||||
```
|
||||
Rate (per time unit)
|
||||
× Time duration (in same units)
|
||||
= Total quantity
|
||||
```
|
||||
|
||||
**Example - Annual customers**:
|
||||
```
|
||||
50 customers/day
|
||||
× 250 business days/year
|
||||
= 12,500 customers/year
|
||||
```
|
||||
|
||||
### Density × Area/Volume
|
||||
|
||||
**When to use**: Spatial estimation (people in stadium, cars on highway, storage capacity)
|
||||
|
||||
**Structure**: Concentration × Space
|
||||
|
||||
**Template**:
|
||||
```
|
||||
Density (per unit area/volume)
|
||||
× Total area/volume
|
||||
= Total quantity
|
||||
```
|
||||
|
||||
**Example - Stadium capacity**:
|
||||
```
|
||||
4 people/sq meter (seated)
|
||||
× 10,000 sq meters seating area
|
||||
= 40,000 capacity
|
||||
```
|
||||
|
||||
### Analogous Scaling
|
||||
|
||||
**When to use**: Have data on similar system, adjusting for size difference
|
||||
|
||||
**Structure**: Known comparable × Scaling factor
|
||||
|
||||
**Template**:
|
||||
```
|
||||
Comparable system value
|
||||
× (Our scale / Their scale)^exponent
|
||||
= Our estimate
|
||||
|
||||
Exponent = 1 for linear scaling, <1 for economies of scale, >1 for diseconomies
|
||||
```
|
||||
|
||||
**Example - Competitor revenue**:
|
||||
```
|
||||
Competitor A revenue: $100M with 500k users
|
||||
Our users: 50k
|
||||
Estimate: $100M × (50k/500k) = $10M
|
||||
(assumes linear revenue per user)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Component Estimation Template
|
||||
|
||||
For each component in decomposition:
|
||||
|
||||
**Component**: [Name of quantity being estimated]
|
||||
|
||||
**Anchor Strategy** (how will we estimate this?):
|
||||
- [ ] Known data (cite source):
|
||||
- [ ] Personal experience/intuition (describe basis):
|
||||
- [ ] Derived from other estimates (show calculation):
|
||||
- [ ] Analogous comparison (similar known quantity):
|
||||
|
||||
**Estimate**: [Value with units]
|
||||
|
||||
**Confidence**: (How certain are we?)
|
||||
- [ ] High (have data or strong knowledge)
|
||||
- [ ] Medium (reasonable inference)
|
||||
- [ ] Low (educated guess, wide range)
|
||||
|
||||
**Sensitivity**: (How much does final answer depend on this?)
|
||||
- [ ] High (10% change here → >5% change in final)
|
||||
- [ ] Medium (10% change here → 1-5% change in final)
|
||||
- [ ] Low (final answer insensitive to this component)
|
||||
|
||||
**Assumption stated**: [Explicit statement of what we're assuming]
|
||||
|
||||
---
|
||||
|
||||
## Bounding Template
|
||||
|
||||
### Optimistic Bound (Upper Limit)
|
||||
|
||||
**Scenario**: [What favorable conditions would maximize the estimate?]
|
||||
|
||||
**Assumptions**:
|
||||
- Component 1: [Optimistic value]
|
||||
- Component 2: [Optimistic value]
|
||||
- ...
|
||||
|
||||
**Calculation**: [Show work]
|
||||
|
||||
**Result**: [Upper bound with units]
|
||||
|
||||
### Pessimistic Bound (Lower Limit)
|
||||
|
||||
**Scenario**: [What unfavorable conditions would minimize the estimate?]
|
||||
|
||||
**Assumptions**:
|
||||
- Component 1: [Pessimistic value]
|
||||
- Component 2: [Pessimistic value]
|
||||
- ...
|
||||
|
||||
**Calculation**: [Show work]
|
||||
|
||||
**Result**: [Lower bound with units]
|
||||
|
||||
### Range Assessment
|
||||
|
||||
**Bounds**: [Lower] to [Upper]
|
||||
|
||||
**Ratio**: Upper/Lower = [X]× range
|
||||
|
||||
**Decision Sensitivity**:
|
||||
- [ ] Decision same across entire range → Estimate good enough
|
||||
- [ ] Decision changes within range → Need more precision or different approach
|
||||
- [ ] Range too wide (>10× span) → Decomposition may be flawed, revisit assumptions
|
||||
|
||||
---
|
||||
|
||||
## Sanity-Check Template
|
||||
|
||||
### Dimensional Analysis
|
||||
|
||||
**Units Check**: Do units cancel correctly in calculation?
|
||||
- Calculation: [Show units through formula]
|
||||
- Final units: [Should match expected units]
|
||||
- [ ] Units are correct
|
||||
|
||||
### Reality Checks
|
||||
|
||||
**Check 1 - Order of magnitude comparison**:
|
||||
- Our estimate: [X]
|
||||
- Known comparable: [Y]
|
||||
- Ratio: [X/Y]
|
||||
- [ ] Within factor of 10? (If not, investigate why)
|
||||
|
||||
**Check 2 - Extreme case testing**:
|
||||
- What if assumption taken to extreme? (e.g., 100% penetration, everyone participates)
|
||||
- Result: [Calculate]
|
||||
- [ ] Does it violate physical/economic constraints? (population limits, GDP constraints, physics)
|
||||
|
||||
**Check 3 - Internal consistency**:
|
||||
- Do derived metrics make sense? (profit margins, growth rates, per-capita figures)
|
||||
- Derived metric: [Calculate, e.g., revenue per employee, cost per user]
|
||||
- [ ] Is it in reasonable range for industry/domain?
|
||||
|
||||
**Check 4 - Personal intuition**:
|
||||
- Does this "feel right" given your experience?
|
||||
- [ ] Passes gut check
|
||||
- [ ] Feels too high (revise assumptions)
|
||||
- [ ] Feels too low (revise assumptions)
|
||||
|
||||
### Common Failure Modes to Check
|
||||
|
||||
- [ ] Did I double-count anything?
|
||||
- [ ] Did I mix units (per day vs per year, millions vs billions)?
|
||||
- [ ] Am I extrapolating linearly when should be exponential (or vice versa)?
|
||||
- [ ] Did I account for utilization/efficiency factors (not everything runs at 100%)?
|
||||
- [ ] Did I consider survivor bias (basing estimate on successful cases only)?
|
||||
|
||||
---
|
||||
|
||||
## Triangulation Template
|
||||
|
||||
### Primary Estimate (from main decomposition)
|
||||
|
||||
**Method**: [Top-down, bottom-up, etc.]
|
||||
|
||||
**Decomposition**: [Brief formula or tree]
|
||||
|
||||
**Result**: [Value with units]
|
||||
|
||||
### Alternate Estimate (different approach)
|
||||
|
||||
**Method**: [Different strategy than primary]
|
||||
|
||||
**Decomposition**: [Brief formula or tree]
|
||||
|
||||
**Result**: [Value with units]
|
||||
|
||||
### Comparison
|
||||
|
||||
**Primary**: [X]
|
||||
**Alternate**: [Y]
|
||||
**Ratio**: [X/Y] = [Z]×
|
||||
|
||||
**Assessment**:
|
||||
- [ ] Within factor of 3 (good agreement, increase confidence)
|
||||
- [ ] Factor of 3-10 (moderate agreement, average or investigate difference)
|
||||
- [ ] >10× difference (investigate: one decomposition is likely flawed)
|
||||
|
||||
**Reconciliation** (if estimates differ):
|
||||
- Why do they differ? (which assumptions differ?)
|
||||
- Which approach is more reliable?
|
||||
- Final estimate: [Choice or average]
|
||||
|
||||
---
|
||||
|
||||
## Complete Estimation Template
|
||||
|
||||
Structure for full documentation:
|
||||
|
||||
1. **Clarification**: Original question, clarified with units/scope, decision context
|
||||
2. **Decomposition**: Strategy (top-down/bottom-up/etc), formula, estimation tree
|
||||
3. **Component Estimates**: For each component - estimate, anchor, assumption, confidence
|
||||
4. **Calculation**: Formula with numbers, central estimate (1-2 sig figs)
|
||||
5. **Bounds**: Optimistic/pessimistic scenarios, range assessment
|
||||
6. **Sanity Checks**: Units, order of magnitude comparison, extreme cases, consistency, gut check
|
||||
7. **Triangulation**: Alternate approach, comparison (within factor of 3?), reconciliation
|
||||
8. **Final Estimate**: Point estimate, range, confidence, key assumptions, sensitivity, recommendation
|
||||
9. **Next Steps**: Data collection, assumption testing, detailed modeling if precision needed
|
||||
Reference in New Issue
Block a user