gh-lyndonkl-claude/skills/decision-matrix/resources/methodology.md

# Decision Matrix: Advanced Methodology

## Workflow

Copy this checklist for complex decision scenarios:

```
Advanced Decision Matrix Progress:
- [ ] Step 1: Diagnose decision complexity
- [ ] Step 2: Apply advanced weighting techniques
- [ ] Step 3: Calibrate and normalize scores
- [ ] Step 4: Perform rigorous sensitivity analysis
- [ ] Step 5: Facilitate group convergence
```

**Step 1: Diagnose decision complexity** - Identify complexity factors (stakeholder disagreement, high uncertainty, strategic importance). See [1. Decision Complexity Assessment](#1-decision-complexity-assessment).

**Step 2: Apply advanced weighting techniques** - Use AHP or other rigorous methods for contentious decisions. See [2. Advanced Weighting Methods](#2-advanced-weighting-methods).

**Step 3: Calibrate and normalize scores** - Handle different scoring approaches and normalize across scorers. See [3. Score Calibration & Normalization](#3-score-calibration--normalization).

**Step 4: Perform rigorous sensitivity analysis** - Test decision robustness with Monte Carlo or scenario analysis. See [4. Advanced Sensitivity Analysis](#4-advanced-sensitivity-analysis).

**Step 5: Facilitate group convergence** - Use Delphi method or consensus-building techniques. See [5. Group Decision Facilitation](#5-group-decision-facilitation).

---

## 1. Decision Complexity Assessment

### Complexity Indicators

**Low Complexity** (use basic template):
- Clear stakeholder alignment on priorities
- Objective criteria with available data
- Low stakes (reversible decision)
- 3-5 alternatives

**Medium Complexity** (use enhanced techniques):
- Moderate stakeholder disagreement
- Mix of objective and subjective criteria
- Moderate stakes (partially reversible)
- 5-8 alternatives

**High Complexity** (use full methodology):
- Significant stakeholder disagreement on priorities
- Mostly subjective criteria or high uncertainty
- High stakes (irreversible or strategic decision)
- >8 alternatives or multi-phase decision
- Regulatory or compliance implications

### Complexity Scoring

| Factor | Low (1) | Medium (2) | High (3) |
|--------|---------|------------|----------|
| **Stakeholder alignment** | Aligned priorities | Some disagreement | Conflicting priorities |
| **Criteria objectivity** | Mostly data-driven | Mix of data & judgment | Mostly subjective |
| **Decision stakes** | Reversible, low cost | Partially reversible | Irreversible, strategic |
| **Uncertainty level** | Low uncertainty | Moderate uncertainty | High uncertainty |
| **Number of alternatives** | 3-4 options | 5-7 options | 8+ options |

**Complexity Score = Sum of factors**
- **5-7 points:** Use basic template
- **8-11 points:** Use enhanced techniques (sections 2-3)
- **12-15 points:** Use full methodology (all sections)

---

## 2. Advanced Weighting Methods

### Analytic Hierarchy Process (AHP)

**When to use:** High-stakes decisions with contentious priorities, need rigorous justification

**Process:**

1. **Create pairwise comparison matrix:** For each pair, rate 1-9 (1=equal, 3=slightly more important, 5=moderately, 7=strongly, 9=extremely)
2. **Calculate weights:** Normalize columns, average rows
3. **Check consistency:** CR < 0.10 acceptable (use online AHP calculator: bpmsg.com/ahp/ahp-calc.php)

**Example:** Comparing Cost, Performance, Risk, Ease pairwise yields weights: Performance 55%, Risk 20%, Cost 15%, Ease 10%

**Advantage:** Rigorous, forces logical consistency in pairwise judgments.

### Swing Weighting

**When to use:** Need to justify weights based on value difference, not just importance

**Process:**

1. **Baseline:** Imagine all criteria at worst level
2. **Swing:** For each criterion, ask "What value does moving from worst to best create?"
3. **Rank swings:** Which swing creates most value?
4. **Assign points:** Give highest swing 100 points, others relative to it
5. **Convert to weights:** Normalize points to percentages

**Example:**

| Criterion | Worst → Best Scenario | Value of Swing | Points | Weight |
|-----------|----------------------|----------------|--------|--------|
| Performance | 50ms → 5ms response | Huge value gain | 100 | 45% |
| Cost | $100K → $50K | Moderate value | 60 | 27% |
| Risk | High → Low risk | Significant value | 50 | 23% |
| Ease | Hard → Easy to use | Minor value | 10 | 5% |

**Total points:** 220 → **Weights:** 100/220=45%, 60/220=27%, 50/220=23%, 10/220=5%

**Advantage:** Focuses on marginal value, not abstract importance. Reveals if criteria with wide option variance should be weighted higher.

### Multi-Voting (Group Weighting)

**When to use:** Group of 5-15 stakeholders needs to converge on weights

**Process:**

1. **Round 1 - Individual allocation:** Each person assigns 100 points across criteria
2. **Reveal distribution:** Show average and variance for each criterion
3. **Discuss outliers:** Why did some assign 40% to Cost while others assigned 10%?
4. **Round 2 - Revised allocation:** Re-allocate with new information
5. **Converge:** Repeat until variance is acceptable or use average

**Example:**

| Criterion | Round 1 Avg | Round 1 Variance | Round 2 Avg | Round 2 Variance |
|-----------|-------------|------------------|-------------|------------------|
| Cost | 25% | High (±15%) | 30% | Low (±5%) |
| Performance | 40% | Medium (±10%) | 38% | Low (±4%) |
| Risk | 20% | Low (±5%) | 20% | Low (±3%) |
| Ease | 15% | High (±12%) | 12% | Low (±4%) |

**Convergence achieved** when variance <±5% for all criteria.

---

## 3. Score Calibration & Normalization

### Handling Different Scorer Tendencies

**Problem:** Some scorers are "hard graders" (6-7 range), others are "easy graders" (8-9 range). This skews results.

**Solution: Z-score normalization**

**Step 1: Calculate each scorer's mean and standard deviation**

Scorer A: Gave scores [8, 9, 7, 8] → Mean=8, SD=0.8
Scorer B: Gave scores [5, 6, 4, 6] → Mean=5.25, SD=0.8

**Step 2: Normalize each score**

Z-score = (Raw Score - Scorer Mean) / Scorer SD

**Step 3: Re-scale to 1-10**

Normalized Score = 5.5 + (Z-score × 1.5)

**Result:** Scorers are calibrated to same scale, eliminating grading bias.

### Dealing with Missing Data

**Scenario:** Some alternatives can't be scored on all criteria (e.g., vendor A won't share cost until later).

**Approach 1: Conditional matrix**

Score available criteria only, note which are missing. Once data arrives, re-run matrix.

**Approach 2: Pessimistic/Optimistic bounds**

Assign worst-case and best-case scores for missing data. Run matrix twice:
- Pessimistic scenario: Missing data gets low score (e.g., 3)
- Optimistic scenario: Missing data gets high score (e.g., 8)

If same option wins both scenarios → Decision is robust. If different winners → Missing data is decision-critical, must obtain before deciding.

### Non-Linear Scoring Curves

**Problem:** Not all criteria are linear. E.g., cost difference between $10K and $20K matters more than $110K vs $120K.

**Solution: Apply utility curves**

**Diminishing returns curve** (Cost, Time):
- Score = 10 × (1 - e^(-k × Cost Improvement))
- k = sensitivity parameter (higher k = faster diminishing returns)

**Threshold curve** (Must meet minimum):
- Score = 0 if below threshold
- Score = 1-10 linear above threshold

**Example:** Load time criterion with 2-second threshold:
- Option A: 1.5s → Score = 10 (below threshold = great)
- Option B: 3s → Score = 5 (above threshold, linear penalty)
- Option C: 5s → Score = 1 (way above threshold)

---

## 4. Advanced Sensitivity Analysis

### Monte Carlo Sensitivity

**When to use:** High uncertainty in scores, want to understand probability distribution of outcomes

**Process:**

1. **Define uncertainty ranges** for each score
   - Option A Cost score: 6 ± 2 (could be 4-8)
   - Option A Performance: 9 ± 0.5 (could be 8.5-9.5)

2. **Run simulations** (1000+ iterations):
   - Randomly sample scores within uncertainty ranges
   - Calculate weighted total for each option
   - Record winner

3. **Analyze results:**
   - Option A wins: 650/1000 = 65% probability
   - Option B wins: 300/1000 = 30% probability
   - Option C wins: 50/1000 = 5% probability

**Interpretation:**
- **>80% win rate:** High confidence in decision
- **50-80% win rate:** Moderate confidence, option is likely but not certain
- **<50% win rate:** Low confidence, gather more data or consider decision is close call

**Tools:** Excel (=RANDBETWEEN or =NORM.INV), Python (numpy.random), R (rnorm)

### Scenario Analysis

**When to use:** Future is uncertain, decisions need to be robust across scenarios

**Process:**

1. **Define scenarios** (typically 3-4):
   - Best case: Favorable market conditions
   - Base case: Expected conditions
   - Worst case: Unfavorable conditions
   - Black swan: Unlikely but high-impact event

2. **Adjust criterion weights or scores per scenario:**

| Scenario | Cost Weight | Performance Weight | Risk Weight |
|----------|-------------|--------------------|-------------|
| Best case | 20% | 50% | 30% |
| Base case | 30% | 40% | 30% |
| Worst case | 40% | 20% | 40% |

3. **Run matrix for each scenario**, identify winner

4. **Evaluate robustness:**
   - **Dominant option:** Wins in all scenarios → Robust choice
   - **Scenario-dependent:** Different winners → Need to assess scenario likelihood
   - **Mixed:** Wins in base + one other → Moderately robust

### Threshold Analysis

**Question:** At what weight does the decision flip?

**Process:**

1. **Vary one criterion weight** from 0% to 100% (keeping others proportional)
2. **Plot total scores** for all options vs. weight
3. **Identify crossover point** where lines intersect (decision flips)

**Example:**

When Performance weight < 25% → Option B wins (cost-optimized)
When Performance weight > 25% → Option A wins (performance-optimized)

**Insight:** Current weight is 40% for Performance. Decision is robust unless Performance drops below 25% importance.

**Practical use:** Communicate to stakeholders: "Even if we reduce Performance priority to 25% (vs current 40%), Option A still wins. Decision is robust."

---

## 5. Group Decision Facilitation

### Delphi Method (Asynchronous Consensus)

**When to use:** Experts geographically distributed, want to avoid groupthink, need convergence without meetings

**Process:**

**Round 1:**
- Each expert scores options independently (no discussion)
- Facilitator compiles scores, calculates median and range

**Round 2:**
- Share Round 1 results (anonymous)
- Experts see median scores and outliers
- Ask experts to re-score, especially if they were outliers (optional: provide reasoning)

**Round 3:**
- Share Round 2 results
- Experts make final adjustments
- Converge on consensus scores (median or mean)

**Convergence criteria:** Standard deviation of scores <1.5 points per criterion

**Example:**

| Option | Criterion | R1 Scores | R1 Median | R2 Scores | R2 Median | R3 Scores | R3 Median |
|--------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
| A | Cost | [5, 7, 9, 6] | 6.5 | [6, 7, 8, 6] | 6.5 | [6, 7, 7, 7] | **7** |

**Advantage:** Avoids dominance by loudest voice, reduces groupthink, allows reflection time.

### Nominal Group Technique (Structured Meeting)

**When to use:** In-person or virtual meeting, need structured discussion to surface disagreements

**Process:**

1. **Silent generation (10 min):** Each person scores options independently
2. **Round-robin sharing (20 min):** Each person shares one score and rationale (no debate yet)
3. **Discussion (30 min):** Debate differences, especially outliers
4. **Re-vote (5 min):** Independent re-scoring after hearing perspectives
5. **Aggregation:** Calculate final scores (mean or median)

**Facilitation tips:**
- Enforce "no interruptions" during round-robin
- Time-box discussion to avoid analysis paralysis
- Focus debate on criteria with widest score variance

### Handling Persistent Disagreement

**Scenario:** After multiple rounds, stakeholders still disagree on weights or scores.

**Options:**

**1. Separate matrices by stakeholder group:**

Run matrix for Engineering priorities, Sales priorities, Executive priorities separately. Present all three results. Highlight where recommendations align vs. differ.

**2. Escalate to decision-maker:**

Present divergence transparently: "Engineering weights Performance at 60%, Sales weights Cost at 50%. Under Engineering weights, Option A wins. Under Sales weights, Option B wins. Recommendation: [Decision-maker] must adjudicate priority trade-off."

**3. Multi-criteria satisficing:**

Instead of optimizing weighted sum, find option that meets minimum thresholds on all criteria. This avoids weighting debate.

**Example:** Option must score ≥7 on Performance AND ≤$50K cost AND ≥6 on Ease of Use. Find options that satisfy all constraints.

---

## 6. Matrix Variations & Extensions

### Weighted Pros/Cons Matrix
Hybrid: Add "Key Pros/Cons/Dealbreakers" columns to matrix for qualitative context alongside quantitative scores.

### Multi-Phase Decision Matrix
**Phase 1:** High-level filter (simple criteria) → shortlist top 3
**Phase 2:** Deep-dive (detailed criteria) → select winner
Avoids analysis paralysis by not deep-diving on all options upfront.

### Risk-Adjusted Matrix
For uncertain scores, use expected value: (Optimistic + 4×Most Likely + Pessimistic) / 6
Accounts for score uncertainty in final weighted total.

---

## 7. Common Failure Modes & Recovery

| Failure Mode | Symptoms | Recovery |
|--------------|----------|----------|
| **Post-Rationalization** | Oddly specific weights, generous scores for preferred option | Assign weights BEFORE scoring, use third-party facilitator |
| **Analysis Paralysis** | >10 criteria, endless tweaking, winner changes repeatedly | Set deadline, time-box criteria (5 max), use satisficing rule |
| **Garbage In, Garbage Out** | Scores are guesses, no data sources, false confidence | Flag uncertainties, gather real data, acknowledge limits |
| **Criterion Soup** | Overlapping criteria, scorer confusion | Consolidate redundant criteria, define each clearly |
| **Spreadsheet Error** | Calculation mistakes, weights don't sum to 100% | Use templates with formulas, peer review calculations |

---

## 8. When to Abandon the Matrix

Despite best efforts, sometimes a decision matrix is not the right tool:

**Abandon if:**

1. **Purely emotional decision:** Choosing baby name, selecting wedding venue (no "right" answer)
   - **Use instead:** Gut feel, user preference vote

2. **Single dominant criterion:** Only cost matters, everything else is noise
   - **Use instead:** Simple cost comparison table

3. **Decision already made:** Political realities mean decision is predetermined
   - **Use instead:** Document decision rationale (not fake analysis)

4. **Future is too uncertain:** Can't meaningfully score because context will change dramatically
   - **Use instead:** Scenario planning, real options analysis, reversible pilot

5. **Stakeholders distrust process:** Matrix seen as "math washing" to impose decision
   - **Use instead:** Deliberative dialog, voting, or delegated authority

**Recognize when structured analysis adds value vs. when it's theater.** Decision matrices work best when:
- Multiple alternatives genuinely exist
- Trade-offs are real and must be balanced
- Stakeholders benefit from transparency
- Data is available or can be gathered
- Decision is reversible if matrix misleads

If these don't hold, consider alternative decision frameworks.