Initial commit

skills/validation-frameworks/SKILL.md

@@ -0,0 +1,595 @@
+---
+name: validation-frameworks
+description: Problem and solution validation methodologies, assumption testing, and MVP validation experiments
+---
+
+# Validation Frameworks
+
+Frameworks for validating problems, solutions, and product assumptions before committing to full development.
+
+## When to Use This Skill
+
+**Auto-loaded by agents**:
+- `research-ops` - For problem/solution validation and assumption testing
+
+**Use when you need**:
+- Validating product ideas before building
+- Testing assumptions and hypotheses
+- De-risking product decisions
+- Running MVP experiments
+- Validating problem-solution fit
+- Testing willingness to pay
+- Evaluating technical feasibility
+
+## Core Principle: Reduce Risk Through Learning
+
+Building the wrong thing is expensive. Validation reduces risk by answering critical questions before major investment:
+
+- **Problem validation**: Is this a real problem worth solving?
+- **Solution validation**: Does our solution actually solve the problem?
+- **Market validation**: Will people pay for this?
+- **Usability validation**: Can people use it?
+- **Technical validation**: Can we build it?
+
+## The Validation Spectrum
+
+Validation isn't binary (validated vs. not validated). It's a spectrum of confidence:
+
+```
+Wild Guess → Hypothesis → Validated Hypothesis → High Confidence → Proven
+```
+
+Early stage: Cheap, fast tests (low confidence gain)
+Later stage: More expensive tests (high confidence gain)
+
+**Example progression**:
+1. **Assumption**: "Busy parents struggle to plan healthy meals"
+2. **Interview 5 parents** → Some validation (small sample)
+3. **Survey 100 parents** → More validation (larger sample)
+4. **Prototype test with 20 parents** → Strong validation (behavior observed)
+5. **Launch MVP, track engagement** → Very strong validation (real usage)
+6. **Measure retention after 3 months** → Proven (sustained behavior)
+
+## Problem Validation vs. Solution Validation
+
+### Problem Validation
+
+**Question**: "Is this a problem worth solving?"
+
+**Goal**: Confirm that:
+- The problem exists
+- It's painful enough that people want it solved
+- It's common enough to matter
+- Current solutions are inadequate
+
+**Methods**:
+- Customer interviews (discovery-focused)
+- Ethnographic observation
+- Surveys about pain points
+- Data analysis (support tickets, analytics)
+- Jobs-to-be-Done interviews
+
+**Red flags that stop here**:
+- No one cares about this problem
+- Existing solutions work fine
+- Problem only affects tiny niche
+- Pain point is mild annoyance, not real pain
+
+**Output**: Problem statement + evidence it's worth solving
+
+See `assets/problem-validation-canvas.md` for a ready-to-use framework.
+
+---
+
+### Solution Validation
+
+**Question**: "Does our solution solve the problem?"
+
+**Goal**: Confirm that:
+- Our solution addresses the problem
+- Users understand it
+- Users would use it
+- It's better than alternatives
+- Value proposition resonates
+
+**Methods**:
+- Prototype testing
+- Landing page tests
+- Concierge MVP (manual solution)
+- Wizard of Oz (fake backend)
+- Pre-sales or waitlist signups
+
+**Red flags**:
+- Users don't understand the solution
+- They prefer their current workaround
+- Would use it but not pay for it
+- Solves wrong part of the problem
+
+**Output**: Validated solution concept + evidence of demand
+
+See `assets/solution-validation-checklist.md` for validation steps.
+
+---
+
+## The Assumption-Validation Cycle
+
+### 1. Identify Assumptions
+
+Every product idea rests on assumptions. Make them explicit.
+
+**Types of assumptions**:
+
+**Desirability**: Will people want this?
+- "Users want to track their habits"
+- "They'll pay $10/month for this"
+- "They'll invite their friends"
+
+**Feasibility**: Can we build this?
+- "We can process data in under 1 second"
+- "We can integrate with their existing tools"
+- "Our team can build this in 3 months"
+
+**Viability**: Should we build this?
+- "Customer acquisition cost will be under $50"
+- "Retention will be above 40% after 30 days"
+- "We can reach 10k users in 12 months"
+
+**Best practice**: Write assumptions as testable hypotheses
+- Not: "Users need this feature"
+- Yes: "At least 60% of users will use this feature weekly"
+
+---
+
+### 2. Prioritize Assumptions to Test
+
+Not all assumptions are equally important. Prioritize by:
+
+**Risk**: How uncertain are we? (Higher risk = higher priority)
+**Impact**: What happens if we're wrong? (Higher impact = higher priority)
+
+**Prioritization matrix**:
+
+| Risk/Impact | High Impact | Low Impact |
+|------------|-------------|------------|
+| **High Risk** | **Test first** | Test second |
+| **Low Risk** | Test second | Maybe skip |
+
+**Riskiest assumptions** (test these first):
+- Leap-of-faith assumptions the entire business depends on
+- Things you've never done before
+- Assumptions about user behavior with no data
+- Technical feasibility of core functionality
+
+**Lower-risk assumptions** (test later or assume):
+- Based on prior experience
+- Easy to change if wrong
+- Industry best practices
+- Minor features
+
+---
+
+### 3. Design Validation Experiments
+
+For each assumption, design the cheapest test that could prove it wrong.
+
+**Experiment design principles**:
+
+**1. Falsifiable**: Can produce evidence that assumption is wrong
+**2. Specific**: Clear success/failure criteria defined upfront
+**3. Minimal**: Smallest effort needed to learn
+**4. Fast**: Results in days/weeks, not months
+**5. Ethical**: Don't mislead or harm users
+
+**The Experiment Canvas**: See `assets/validation-experiment-template.md`
+
+---
+
+### 4. Run the Experiment
+
+**Before starting**:
+- [ ] Define success criteria ("At least 40% will click")
+- [ ] Set sample size ("Test with 50 users")
+- [ ] Decide timeframe ("Run for 1 week")
+- [ ] Identify what success/failure would mean for product
+
+**During**:
+- Track metrics rigorously
+- Document unexpected learnings
+- Don't change experiment mid-flight
+
+**After**:
+- Analyze results honestly (avoid confirmation bias)
+- Document what you learned
+- Decide: Pivot, persevere, or iterate
+
+---
+
+## Validation Methods by Fidelity
+
+### Low-Fidelity (Hours to Days)
+
+**1. Customer Interviews**
+- **Cost**: Very low (just time)
+- **Validates**: Problem existence, pain level, current solutions
+- **Limitations**: What people say ≠ what they do
+- **Best for**: Early problem validation
+
+**2. Surveys**
+- **Cost**: Low
+- **Validates**: Problem prevalence, feature preferences
+- **Limitations**: Response bias, can't probe deeply
+- **Best for**: Quantifying what you learned qualitatively
+
+**3. Landing Page Test**
+- **Cost**: Low (1-2 days to build)
+- **Validates**: Interest in solution, value proposition clarity
+- **Measure**: Email signups, clicks to "Get Started"
+- **Best for**: Demand validation before building
+
+**4. Paper Prototypes**
+- **Cost**: Very low (sketch on paper/whiteboard)
+- **Validates**: Concept understanding, workflow feasibility
+- **Limitations**: Low realism
+- **Best for**: Very early solution concepts
+
+---
+
+### Medium-Fidelity (Days to Weeks)
+
+**5. Clickable Prototypes**
+- **Cost**: Medium (design tool, 2-5 days)
+- **Validates**: User flow, interaction patterns, comprehension
+- **Tools**: Figma, Sketch, Adobe XD
+- **Best for**: Usability validation pre-development
+
+**6. Concierge MVP**
+- **Cost**: Medium (your time delivering manually)
+- **Validates**: Value of outcome, willingness to use/pay
+- **Example**: Manually curate recommendations before building algorithm
+- **Best for**: Validating value before automation
+
+**7. Wizard of Oz MVP**
+- **Cost**: Medium (build facade, manual backend)
+- **Validates**: User behavior, feature usage, workflows
+- **Example**: "AI" feature that's actually humans behind the scenes
+- **Best for**: Testing expensive-to-build features
+
+---
+
+### High-Fidelity (Weeks to Months)
+
+**8. Functional Prototype**
+- **Cost**: High (weeks of development)
+- **Validates**: Technical feasibility, actual usage patterns
+- **Limitations**: Expensive if you're wrong
+- **Best for**: After other validation, final pre-launch check
+
+**9. Private Beta**
+- **Cost**: High (full build + support)
+- **Validates**: Real-world usage, retention, bugs
+- **Best for**: Pre-launch final validation with early adopters
+
+**10. Public MVP**
+- **Cost**: Very high (full product)
+- **Validates**: Product-market fit, business model viability
+- **Best for**: After all other validation passed
+
+---
+
+## Validation Metrics and Success Criteria
+
+### Quantitative Validation Metrics
+
+**Interest/Demand**:
+- Landing page conversion rate
+- Waitlist signup rate
+- Pre-order conversion
+- Prototype test completion rate
+
+**Usage**:
+- Feature adoption rate
+- Task completion rate
+- Time on task
+- Frequency of use
+
+**Engagement**:
+- DAU/MAU (daily/monthly active users)
+- Session length
+- Return rate
+- Feature usage depth
+
+**Value Realization**:
+- Aha moment rate (reached key action)
+- Time to value
+- Retention curves
+- Referral rate
+
+**Business**:
+- Conversion to paid
+- Lifetime value (LTV)
+- Customer acquisition cost (CAC)
+- Churn rate
+
+---
+
+### Qualitative Validation Signals
+
+**Strong positive signals**:
+- Users ask when they can buy it
+- Users describe it to others excitedly
+- Users find use cases you didn't imagine
+- Users return without prompting
+- Users complain when you take it away
+
+**Weak positive signals**:
+- "That's interesting"
+- "I might use that"
+- Polite feedback with no follow-up
+- Theoretical interest but no commitment
+
+**Negative signals**:
+- Confusion about value proposition
+- Preference for existing solution
+- Feature requests that miss the point
+- Can't articulate who would use it
+- Ghosting after initial interest
+
+---
+
+## Setting Success Criteria
+
+**Before running experiment**, define what success looks like.
+
+**Framework**: Set three thresholds
+
+1. **Strong success**: Clear green light, proceed confidently
+2. **Moderate success**: Promising but needs iteration
+3. **Failure**: Pivot or abandon
+
+**Example: Landing page test**
+- Strong success: > 30% email signup rate
+- Moderate success: 15-30% signup rate
+- Failure: < 15% signup rate
+
+**Example: Prototype test**
+- Strong success: 8/10 users complete task, would use weekly
+- Moderate success: 5-7/10 complete, would use monthly
+- Failure: < 5/10 complete or no usage intent
+
+**Important**: Decide criteria before seeing results to avoid bias.
+
+---
+
+## Teresa Torres Continuous Discovery Validation
+
+### Opportunity Solution Trees
+
+Map opportunities (user needs) to solutions to validate:
+
+```
+Outcome
+└── Opportunity 1
+    ├── Solution A
+    ├── Solution B
+    └── Solution C
+└── Opportunity 2
+    └── Solution D
+```
+
+**Validate each level**:
+1. Outcome: Is this the right goal?
+2. Opportunities: Are these real user needs?
+3. Solutions: Will this solution address the opportunity?
+
+### Assumption Testing
+
+For each solution, map assumptions:
+
+**Desirability assumptions**: Will users want this?
+**Usability assumptions**: Can users use this?
+**Feasibility assumptions**: Can we build this?
+**Viability assumptions**: Should we build this?
+
+Then test riskiest assumptions first with smallest possible experiments.
+
+### Weekly Touchpoints
+
+Continuous discovery = continuous validation:
+- Weekly customer interviews (problem + solution validation)
+- Weekly prototype tests with 2-3 users
+- Weekly assumption tests (small experiments)
+
+**Goal**: Continuous evidence flow, not one-time validation.
+
+See `references/lean-startup-validation.md` and `references/assumption-testing-methods.md` for detailed methodologies.
+
+---
+
+## Common Validation Anti-Patterns
+
+### 1. Fake Validation
+
+**What it looks like**:
+- Asking friends and family (they'll lie to be nice)
+- Leading questions ("Wouldn't you love...?")
+- Testing with employees
+- Cherry-picking positive feedback
+
+**Fix**: Talk to real users, ask open-ended questions, seek disconfirming evidence.
+
+---
+
+### 2. Analysis Paralysis
+
+**What it looks like**:
+- Endless research without decisions
+- Testing everything before building anything
+- Demanding statistical significance with 3 data points
+
+**Fix**: Accept uncertainty, set decision deadlines, bias toward action.
+
+---
+
+### 3. Confirmation Bias
+
+**What it looks like**:
+- Only hearing what confirms existing beliefs
+- Dismissing negative feedback as "they don't get it"
+- Stopping research when you hear what you wanted
+
+**Fix**: Actively seek disconfirming evidence, set falsifiable criteria upfront.
+
+---
+
+### 4. Vanity Validation
+
+**What it looks like**:
+- "I got 500 email signups!" (but 0 conversions)
+- "People loved the demo!" (but won't use it)
+- "We got great feedback!" (but all feature requests, no usage)
+
+**Fix**: Focus on behavior over opinions, retention over acquisition.
+
+---
+
+### 5. Building Instead of Validating
+
+**What it looks like**:
+- "Let's build it and see if anyone uses it"
+- "It'll only take 2 weeks" (takes 2 months)
+- Full build before any user contact
+
+**Fix**: Always do cheapest possible test first, build only after validation.
+
+---
+
+## Validation Workflow: End-to-End Example
+
+**Idea**: Mobile app for tracking baby sleep patterns
+
+### Phase 1: Problem Validation
+
+**Assumption**: "New parents struggle to understand their baby's sleep patterns"
+
+**Experiment 1: Customer Interviews**
+- Interview 10 new parents
+- Ask about sleep tracking current methods, pain points
+- Success criteria: 7/10 express frustration with current methods
+
+**Result**: 9/10 keep notes on paper, all frustrated by inconsistency
+
+**Conclusion**: Problem validated, proceed.
+
+---
+
+### Phase 2: Solution Validation (Concept)
+
+**Assumption**: "Parents would use an app that auto-tracks sleep from smart monitor"
+
+**Experiment 2: Landing Page**
+- Create page describing solution
+- Run ads targeting new parents
+- Success criteria: 20% email signup rate
+
+**Result**: 8% signup rate
+
+**Conclusion**: Interest is lower than hoped. Talk to non-signups to understand why.
+
+**Learning**: They don't have smart monitors, manual entry would be fine.
+
+---
+
+### Phase 3: Solution Validation (Refined)
+
+**Assumption**: "Parents will manually log sleep times if it's fast"
+
+**Experiment 3: Concierge MVP**
+- Ask 5 parents to text sleep times
+- Manually create charts/insights
+- Send back daily summaries
+- Success criteria: 4/5 use it for 2 weeks
+
+**Result**: All 5 use it for 2 weeks, ask to keep using
+
+**Conclusion**: Strong validation, build simple MVP.
+
+---
+
+### Phase 4: Build & Validate MVP
+
+**Build**: Simple app with manual time entry + basic charts
+
+**Experiment 4: Private Beta**
+- 30 parents use MVP for 4 weeks
+- Success criteria: 60% retention after 2 weeks, 40% after 4 weeks
+
+**Result**: 70% 2-week retention, 50% 4-week retention
+
+**Conclusion**: Exceeded criteria, prepare for launch.
+
+---
+
+This workflow: problem → concept → refined solution → MVP took 8 weeks and cost minimal development before validation.
+
+---
+
+## Validation Checklist by Stage
+
+### Idea Stage
+- [ ] Problem validated through customer interviews
+- [ ] Current solutions identified and evaluated
+- [ ] Target user segment defined and accessible
+- [ ] Pain level assessed (nice-to-have vs. must-have)
+
+### Concept Stage
+- [ ] Solution concept tested with users
+- [ ] Value proposition resonates
+- [ ] Demand signal measured (signups, interest)
+- [ ] Key assumptions identified and prioritized
+
+### Pre-Build Stage
+- [ ] Prototype tested with target users
+- [ ] Core workflow validated
+- [ ] Pricing validated (willingness to pay)
+- [ ] Technical feasibility confirmed
+
+### MVP Stage
+- [ ] Beta users recruited
+- [ ] Usage patterns observed
+- [ ] Retention measured
+- [ ] Unit economics validated
+
+---
+
+## Ready-to-Use Resources
+
+### In `assets/`:
+- **problem-validation-canvas.md**: Framework for validating problems before solutions
+- **solution-validation-checklist.md**: Step-by-step checklist for solution validation
+- **validation-experiment-template.md**: Design experiments to test assumptions
+
+### In `references/`:
+- **lean-startup-validation.md**: Build-Measure-Learn cycle, MVP types, pivot decisions
+- **assumption-testing-methods.md**: Comprehensive assumption testing techniques
+
+---
+
+## Further Reading
+
+**Books**:
+- "The Lean Startup" by Eric Ries
+- "The Mom Test" by Rob Fitzpatrick
+- "Continuous Discovery Habits" by Teresa Torres
+- "Testing Business Ideas" by Bland & Osterwalder
+
+**Key Concepts**:
+- Minimum Viable Product (MVP)
+- Build-Measure-Learn
+- Validated learning
+- Pivot or persevere
+- Assumption mapping
+- Evidence-based product development
+
+---
+
+**Remember**: Validation isn't about proving you're right. It's about learning what's true. Seek evidence that you're wrong - that's the fastest path to building something valuable.