Initial commit

This commit is contained in:
Zhongwei Li
2025-11-30 08:38:26 +08:00
commit 41d9f6b189
304 changed files with 98322 additions and 0 deletions

View File

@@ -0,0 +1,495 @@
---
name: scout-mindset-bias-check
description: Use to detect and remove cognitive biases from reasoning. Invoke when prediction feels emotional, stuck at 50/50, or when you want to validate forecasting process. Use when user mentions scout mindset, soldier mindset, bias check, reversal test, scope sensitivity, or cognitive distortions.
---
# Scout Mindset & Bias Check
## Table of Contents
- [What is Scout Mindset?](#what-is-scout-mindset)
- [When to Use This Skill](#when-to-use-this-skill)
- [Interactive Menu](#interactive-menu)
- [Quick Reference](#quick-reference)
- [Resource Files](#resource-files)
---
## What is Scout Mindset?
**Scout Mindset** (Julia Galef) is the motivation to see things as they are, not as you wish them to be. Contrast with **Soldier Mindset**, which defends a position regardless of evidence.
**Core Principle:** Your goal is to map the territory accurately, not win an argument.
**Why It Matters:**
- Forecasting requires intellectual honesty
- Biases systemically distort probabilities
- Emotional attachment clouds judgment
- Motivated reasoning leads to overconfidence
---
## When to Use This Skill
Use this skill when:
- **Prediction feels emotional** - You want a certain outcome
- **Stuck at 50/50** - Indecisive, can't commit to probability
- **Defending a position** - Arguing for your forecast, not questioning it
- **After inside view analysis** - Used specific details, need bias check
- **Disagreement with others** - Different people, different probabilities
- **Before finalizing** - Last sanity check
Do NOT skip this when stakes are high, you have strong priors, or forecast affects you personally.
---
## Interactive Menu
**What would you like to do?**
### Core Workflows
**1. [Run the Reversal Test](#1-run-the-reversal-test)** - Check if you'd accept opposite evidence
- Detect motivated reasoning
- Validate evidence standards
- Expose special pleading
**2. [Check Scope Sensitivity](#2-check-scope-sensitivity)** - Ensure probabilities scale with inputs
- Linear scaling test
- Reference point calibration
- Magnitude assessment
**3. [Test Status Quo Bias](#3-test-status-quo-bias)** - Challenge "no change" assumptions
- Entropy principle
- Change vs stability energy
- Default state inversion
**4. [Audit Confidence Intervals](#4-audit-confidence-intervals)** - Validate CI width
- Surprise test
- Historical calibration
- Overconfidence check
**5. [Run Full Bias Audit](#5-run-full-bias-audit)** - Comprehensive bias scan
- All major cognitive biases
- Systematic checklist
- Prioritized remediation
**6. [Learn the Framework](#6-learn-the-framework)** - Deep dive into methodology
- Read [Scout vs Soldier Mindset](resources/scout-vs-soldier.md)
- Read [Cognitive Bias Catalog](resources/cognitive-bias-catalog.md)
- Read [Debiasing Techniques](resources/debiasing-techniques.md)
**7. Exit** - Return to main forecasting workflow
---
## 1. Run the Reversal Test
**Check if you'd accept evidence pointing the opposite direction.**
```
Reversal Test Progress:
- [ ] Step 1: State your current conclusion
- [ ] Step 2: Identify supporting evidence
- [ ] Step 3: Reverse the evidence
- [ ] Step 4: Ask "Would I still accept it?"
- [ ] Step 5: Adjust for double standards
```
### Step 1: State your current conclusion
**What are you predicting?**
- Prediction: [Event]
- Probability: [X]%
- Direction: [High/Low confidence]
### Step 2: Identify supporting evidence
**List the evidence that supports your conclusion.**
**Example:** Candidate A will win (75%)
1. Polls show A ahead by 5%
2. A has more campaign funding
3. Expert pundits favor A
4. A has better debate ratings
### Step 3: Reverse the evidence
**Imagine the same evidence pointed the OTHER way.**
**Reversed:** What if polls showed B ahead, B had more funding, experts favored B, and B had better ratings?
### Step 4: Ask "Would I still accept it?"
**The Critical Question:**
> If this reversed evidence existed, would I accept it as valid and change my prediction?
**Three possible answers:**
**A) YES - I would accept reversed evidence**
✓ No bias detected, continue with current reasoning
**B) NO - I would dismiss reversed evidence**
**Warning:** Motivated reasoning - you're accepting evidence when it supports you, dismissing equivalent evidence when it doesn't (special pleading)
**C) UNSURE - I'd need to think about it**
**Warning:** Asymmetric evidence standards suggest rationalizing, not reasoning
### Step 5: Adjust for double standards
**If you answered B or C:**
**Ask:** Why do I dismiss this evidence in one direction but accept it in the other? Is there an objective reason, or am I motivated by preference?
**Common rationalizations:**
- "This source is biased" (only when it disagrees)
- "Sample size too small" (only for unfavorable polls)
- "Outlier data" (only for data you dislike)
- "Context matters" (invoked selectively)
**The Fix:**
- **Option 1:** Reject the evidence entirely (if you wouldn't trust it reversed, don't trust it now)
- **Option 2:** Accept it in both directions (trust evidence regardless of direction)
- **Option 3:** Weight it appropriately (maybe it's weak evidence both ways)
**Probability adjustment:** If you detected double standards, move probability 10-15% toward 50%
**Next:** Return to [menu](#interactive-menu)
---
## 2. Check Scope Sensitivity
**Ensure your probabilities scale appropriately with magnitude.**
```
Scope Sensitivity Progress:
- [ ] Step 1: Identify the variable scale
- [ ] Step 2: Test linear scaling
- [ ] Step 3: Check reference point calibration
- [ ] Step 4: Validate magnitude assessment
- [ ] Step 5: Adjust for scope insensitivity
```
### Step 1: Identify the variable scale
**What dimension has magnitude?**
- Number of people (100 vs 10,000 vs 1,000,000)
- Dollar amounts ($1K vs $100K vs $10M)
- Time duration (1 month vs 1 year vs 10 years)
### Step 2: Test linear scaling
**The Linearity Test:** Double the input, check if impact doubles.
**Example: Startup funding**
- If raised $1M: ___%
- If raised $10M: ___%
- If raised $100M: ___%
**Scope sensitivity check:** Did probabilities scale reasonably? If they barely changed → Scope insensitive
### Step 3: Check reference point calibration
**The Anchoring Test:** Did you start with a number (base rate, someone else's forecast, round number) and insufficiently adjust?
**The fix:**
- Generate probability from scratch without looking at others
- Then compare and reconcile differences
- Don't just "split the difference" - reason about why estimates differ
### Step 4: Validate magnitude assessment
**The "1 vs 10 vs 100" Test:** For your forecast, vary the scale by 10×.
**Example: Project timeline**
- 1 month: P(success) = ___%
- 10 months: P(success) = ___%
- 100 months: P(success) = ___%
**Expected:** Probability should change significantly. If all three estimates are within 10 percentage points → Scope insensitivity
### Step 5: Adjust for scope insensitivity
**The problem:** Your emotional system responds to the category, not the magnitude.
**The fix:**
**Method 1: Logarithmic scaling** - Use log scale for intuition
**Method 2: Reference class by scale** - Don't use "startups" as reference class. Use "Startups that raised $1M" (10% success) vs "Startups that raised $100M" (60% success)
**Method 3: Explicit calibration** - Use a formula: P(success) = base_rate + k × log(amount)
**Next:** Return to [menu](#interactive-menu)
---
## 3. Test Status Quo Bias
**Challenge the assumption that "no change" is the default.**
```
Status Quo Bias Progress:
- [ ] Step 1: Identify status quo prediction
- [ ] Step 2: Calculate energy to maintain status quo
- [ ] Step 3: Invert the default
- [ ] Step 4: Apply entropy principle
- [ ] Step 5: Adjust probabilities
```
### Step 1: Identify status quo prediction
**Are you predicting "no change"?** Examples: "This trend will continue," "Market share will stay the same," "Policy won't change"
Status quo predictions often get inflated probabilities because change feels risky.
### Step 2: Calculate energy to maintain status quo
**The Entropy Principle:** In the absence of active energy input, systems decay toward disorder.
**Question:** "What effort is required to keep things the same?"
**Examples:**
- **Market share:** To maintain requires matching competitor innovation → Energy required: High → Status quo is HARD
- **Policy:** To maintain requires no proposals for change → Energy required: Low → Status quo is easier
### Step 3: Invert the default
**Mental Exercise:**
- **Normal framing:** "Will X change?" (Default = no)
- **Inverted framing:** "Will X stay the same?" (Default = no)
**Bias check:** If P(change) + P(same) ≠ 100%, you have status quo bias.
### Step 4: Apply entropy principle
**Second Law of Thermodynamics (applied to forecasting):**
**Ask:**
1. Is this system open or closed?
2. Is energy being input to maintain/improve?
3. Is that energy sufficient?
### Step 5: Adjust probabilities
**If you detected status quo bias:**
**For "no change" predictions that require high energy:**
- Reduce P(status quo) by 10-20%
- Increase P(change) correspondingly
**For predictions where inertia truly helps:** No adjustment needed
**The heuristic:** If maintaining status quo requires active effort, decay is more likely than you think.
**Next:** Return to [menu](#interactive-menu)
---
## 4. Audit Confidence Intervals
**Validate that your CI width reflects true uncertainty.**
```
Confidence Interval Audit Progress:
- [ ] Step 1: State current CI
- [ ] Step 2: Run surprise test
- [ ] Step 3: Check historical calibration
- [ ] Step 4: Compare to reference class variance
- [ ] Step 5: Adjust CI width
```
### Step 1: State current CI
**Current confidence interval:**
- Point estimate: ___%
- Lower bound: ___%
- Upper bound: ___%
- Width: ___ percentage points
- Confidence level: ___ (usually 80% or 90%)
### Step 2: Run surprise test
**The Surprise Test:** "Would I be **genuinely shocked** if the true value fell outside my confidence interval?"
**Calibration:**
- 80% CI → Should be shocked 20% of the time
- 90% CI → Should be shocked 10% of the time
**Test:** Imagine the outcome lands just below your lower bound or just above your upper bound.
**Three possible answers:**
- **A) "Yes, I'd be very surprised"** - ✓ CI appropriately calibrated
- **B) "No, not that surprised"** - ⚠ CI too narrow (overconfident) → Widen interval
- **C) "I'd be amazed if it landed in the range"** - ⚠ CI too wide → Narrow interval
### Step 3: Check historical calibration
**Look at your past forecasts:**
1. Collect last 20-50 forecasts with CIs
2. Count how many actual outcomes fell outside your CIs
3. Compare to theoretical expectation
| CI Level | Expected Outside | Your Actual |
|----------|------------------|-------------|
| 80% | 20% | ___% |
| 90% | 10% | ___% |
**Diagnosis:** Actual > Expected → CIs too narrow (overconfident) - Most common
### Step 4: Compare to reference class variance
**If you have reference class data:**
1. Calculate standard deviation of reference class outcomes
2. Your CI should roughly match that variance
**Example:** Reference class SD = 12%, your 80% CI ≈ Point estimate ± 15%
If your CI is narrower than reference class variance, you're claiming to know more than average. Justify why, or widen CI.
### Step 5: Adjust CI width
**Adjustment rules:**
- **If overconfident:** Multiply current width by 1.5× to 2×
- **If underconfident:** Reduce width by 0.5× to 0.75×
**Next:** Return to [menu](#interactive-menu)
---
## 5. Run Full Bias Audit
**Comprehensive scan of major cognitive biases.**
```
Full Bias Audit Progress:
- [ ] Step 1: Confirmation bias check
- [ ] Step 2: Availability bias check
- [ ] Step 3: Anchoring bias check
- [ ] Step 4: Affect heuristic check
- [ ] Step 5: Overconfidence check
- [ ] Step 6: Attribution error check
- [ ] Step 7: Prioritize and remediate
```
See [Cognitive Bias Catalog](resources/cognitive-bias-catalog.md) for detailed descriptions.
**Quick audit questions:**
### 1. Confirmation Bias
- [ ] Did I seek out disconfirming evidence?
- [ ] Did I give equal weight to evidence against my position?
- [ ] Did I actively try to prove myself wrong?
**If NO to any → Confirmation bias detected**
### 2. Availability Bias
- [ ] Did I rely on recent/memorable examples?
- [ ] Did I use systematic data vs "what comes to mind"?
- [ ] Did I check if my examples are representative?
**If NO to any → Availability bias detected**
### 3. Anchoring Bias
- [ ] Did I generate my estimate independently first?
- [ ] Did I avoid being influenced by others' numbers?
- [ ] Did I adjust sufficiently from initial anchor?
**If NO to any → Anchoring bias detected**
### 4. Affect Heuristic
- [ ] Do I have an emotional preference for the outcome?
- [ ] Did I separate "what I want" from "what will happen"?
- [ ] Would I make the same forecast if incentives were reversed?
**If NO to any → Affect heuristic detected**
### 5. Overconfidence
- [ ] Did I run a premortem?
- [ ] Are my CIs wide enough (surprise test)?
- [ ] Did I identify ways I could be wrong?
**If NO to any → Overconfidence detected**
### 6. Fundamental Attribution Error
- [ ] Did I attribute success to skill vs luck appropriately?
- [ ] Did I consider situational factors, not just personal traits?
- [ ] Did I avoid "great man" narratives?
**If NO to any → Attribution error detected**
### Step 7: Prioritize and remediate
**For each detected bias:**
1. **Severity:** High / Medium / Low
2. **Direction:** Pushing probability up or down?
3. **Magnitude:** Estimated percentage point impact
**Remediation example:**
| Bias | Severity | Direction | Adjustment |
|------|----------|-----------|------------|
| Confirmation | High | Up | -15% |
| Availability | Medium | Up | -10% |
| Affect heuristic | High | Up | -20% |
**Net adjustment:** -45% → Move probability down by 45 points (e.g., 80% → 35%)
**Next:** Return to [menu](#interactive-menu)
---
## 6. Learn the Framework
**Deep dive into the methodology.**
### Resource Files
📄 **[Scout vs Soldier Mindset](resources/scout-vs-soldier.md)**
- Julia Galef's framework
- Motivated reasoning
- Intellectual honesty
- Identity and beliefs
📄 **[Cognitive Bias Catalog](resources/cognitive-bias-catalog.md)**
- 20+ major biases
- How they affect forecasting
- Detection methods
- Remediation strategies
📄 **[Debiasing Techniques](resources/debiasing-techniques.md)**
- Systematic debiasing process
- Pre-commitment strategies
- External accountability
- Algorithmic aids
**Next:** Return to [menu](#interactive-menu)
---
## Quick Reference
### The Scout Commandments
1. **Truth over comfort** - Accuracy beats wishful thinking
2. **Seek disconfirmation** - Try to prove yourself wrong
3. **Hold beliefs lightly** - Probabilistic, not binary
4. **Update incrementally** - Change mind with evidence
5. **Separate wanting from expecting** - Desire ≠ Forecast
6. **Check your work** - Run bias audits routinely
7. **Stay calibrated** - Track accuracy over time
> Scout mindset is the drive to see things as they are, not as you wish them to be.
---
## Resource Files
📁 **resources/**
- [scout-vs-soldier.md](resources/scout-vs-soldier.md) - Mindset framework
- [cognitive-bias-catalog.md](resources/cognitive-bias-catalog.md) - Comprehensive bias reference
- [debiasing-techniques.md](resources/debiasing-techniques.md) - Remediation strategies
---
**Ready to start? Choose a number from the [menu](#interactive-menu) above.**

View File

@@ -0,0 +1,374 @@
# Cognitive Bias Catalog
## Quick Reference Table
| Bias | Category | Impact | Detection | Remediation |
|------|----------|--------|-----------|-------------|
| Confirmation | Confirmation | Seek supporting evidence only | Search for disconfirming evidence? | Red team your forecast |
| Desirability | Confirmation | Want outcome → believe it's likely | Do I want this outcome? | Outsource to neutral party |
| Availability | Availability | Recent/vivid events dominate | What recent news influenced me? | Look up actual statistics |
| Recency | Availability | Overweight recent data | Considering full history? | Expand time window |
| Anchoring | Anchoring | First number sticks | Too close to initial number? | Generate estimate first |
| Affect | Affect | Feelings override data | How do I feel about this? | Acknowledge, then set aside |
| Loss Aversion | Affect | Overweight downside | Weighting losses more? | Evaluate symmetrically |
| Overconfidence | Overconfidence | Intervals too narrow | Track calibration | Widen intervals to 20-80% |
| Dunning-Kruger | Overconfidence | Novices overestimate | How experienced am I? | Seek expert feedback |
| Optimism | Overconfidence | "Won't happen to me" | What's the base rate? | Apply base rate to self |
| Pessimism | Overconfidence | Overweight negatives | Only considering downsides? | List positive scenarios |
| Attribution Error | Attribution | Blame person, not situation | What situational factors? | Consider constraints first |
| Self-Serving | Attribution | Success=skill, failure=luck | Consistent attribution? | Same standard for both |
| Framing | Framing | Presentation changes answer | How is this framed? | Rephrase multiple ways |
| Narrative Fallacy | Framing | Simple stories mislead | Story too clean? | Prefer stats over stories |
| Sunk Cost | Temporal | Can't abandon past investment | Only future costs/benefits? | Decide as if starting fresh |
| Hindsight | Temporal | "Knew it all along" | Written record of prediction? | Record forecasts beforehand |
| Planning Fallacy | Temporal | Underestimate time/cost | Reference class timeline? | Add 2-3x buffer |
| Outcome Bias | Temporal | Judge by result not process | Evaluating process or outcome? | Judge by info available then |
| Clustering Illusion | Pattern | See patterns in randomness | Statistically significant? | Test significance |
| Gambler's Fallacy | Pattern | Expect short-term balancing | Are events independent? | Use actual probability |
| Base Rate Neglect | Bayesian | Ignore prior probabilities | Did I start with base rate? | Always start with base rate |
| Conjunction Fallacy | Bayesian | Specific > general | Is A&B > A alone? | P(A&B) ≤ P(A) always |
| Halo Effect | Social | One trait colors everything | Generalizing from one trait? | Assess dimensions separately |
| Authority Bias | Social | Overweight expert opinions | Expert's track record? | Evaluate evidence not credentials |
| Peak-End | Memory | Remember peaks/endings only | Remembering whole sequence? | Review full historical record |
---
## Confirmation Cluster
### Confirmation Bias
**Definition:** Search for, interpret, and recall information that confirms pre-existing beliefs.
**Affects forecasting:** Only look for supporting evidence, discount contradictions, selective memory.
**Detect:** Did I search for disconfirming evidence? Can I steelman the opposite view?
**Remediate:** Red team your forecast, list disconfirming evidence first, ask "How could I be wrong?"
### Desirability Bias
**Definition:** Believing outcomes you want are more likely than they are.
**Affects forecasting:** Bullish on own startup, wishful thinking masquerading as analysis.
**Detect:** Do I want this outcome? Am I emotionally invested?
**Remediate:** Outsource to neutral party, imagine opposite outcome, forecast before declaring preference.
---
## Availability Cluster
### Availability Heuristic
**Definition:** Judging probability by how easily examples come to mind.
**Affects forecasting:** Overestimate vivid risks (terrorism), underestimate mundane (heart disease), media coverage distorts frequency perception.
**Detect:** What recent news am I thinking of? Is this vivid/emotional/recent?
**Remediate:** Look up actual statistics, use reference class not memorable examples.
### Recency Bias
**Definition:** Overweighting recent events relative to historical patterns.
**Affects forecasting:** Extrapolate recent trends linearly, forget cycles and mean reversion.
**Detect:** How much history am I considering? Is forecast just recent trend?
**Remediate:** Expand time window (decades not months), check for cyclicality.
---
## Anchoring Cluster
### Anchoring Bias
**Definition:** Over-relying on first piece of information encountered.
**Affects forecasting:** First number becomes estimate, can't adjust sufficiently from anchor.
**Detect:** What was first number I heard? Am I too close to it?
**Remediate:** Generate own estimate first, use multiple independent sources.
### Priming
**Definition:** Prior stimulus influences subsequent response.
**Affects forecasting:** Reading disaster primes pessimism, context shapes judgment unconsciously.
**Detect:** What did I just read/see/hear? Is mood affecting forecast?
**Remediate:** Clear mind before forecasting, wait between exposure and estimation.
---
## Affect Cluster
### Affect Heuristic
**Definition:** Letting feelings about something determine beliefs about it.
**Affects forecasting:** Like it → think it's safe, dislike it → think it's dangerous.
**Detect:** How do I feel about this? Would I forecast differently if neutral?
**Remediate:** Acknowledge emotion then set aside, focus on base rates and evidence.
### Loss Aversion
**Definition:** Losses hurt more than equivalent gains feel good (2:1 ratio).
**Affects forecasting:** Overweight downside scenarios, status quo bias, asymmetric risk evaluation.
**Detect:** Am I weighting losses more? Would I accept bet if gains/losses swapped?
**Remediate:** Evaluate gains and losses symmetrically, use expected value calculation.
---
## Overconfidence Cluster
### Overconfidence Bias
**Definition:** Confidence exceeds actual accuracy.
**Affects forecasting:** 90% intervals capture truth 50% of time, narrow ranges, extreme probabilities.
**Detect:** Track calibration, are intervals too narrow? Can I be surprised?
**Remediate:** Widen confidence intervals, track calibration, use 20-80% as default.
### Dunning-Kruger Effect
**Definition:** Unskilled overestimate competence; experts underestimate.
**Affects forecasting:** Novices predict with false precision, don't know what they don't know.
**Detect:** How experienced am I in this domain? Do experts agree?
**Remediate:** If novice widen intervals, seek expert feedback, learn domain deeply first.
### Optimism Bias
**Definition:** Believing you're less likely than others to experience negatives.
**Affects forecasting:** "My startup is different" (90% fail), "This time is different" (rarely is).
**Detect:** What's base rate for people like me? Am I assuming I'm special?
**Remediate:** Use reference class for yourself, apply base rates, assume average then adjust slightly.
### Pessimism Bias
**Definition:** Overweighting negative outcomes, underweighting positive.
**Affects forecasting:** Disaster predictions rarely materialize, underestimate human adaptability.
**Detect:** Only considering downsides? What positive scenarios missing?
**Remediate:** Explicitly list positive scenarios, consider adaptive responses.
---
## Attribution Cluster
### Fundamental Attribution Error
**Definition:** Overattribute behavior to personality, underattribute to situation.
**Affects forecasting:** "CEO is brilliant" ignores market conditions, predict based on person not circumstances.
**Detect:** What situational factors am I ignoring? How much is luck vs. skill?
**Remediate:** Consider situational constraints first, estimate luck vs. skill proportion.
### Self-Serving Bias
**Definition:** Attribute success to skill, failure to bad luck.
**Affects forecasting:** Can't learn from mistakes (was luck!), overconfident after wins (was skill!).
**Detect:** Would I explain someone else's outcome this way? Do I attribute consistently?
**Remediate:** Apply same standard to wins and losses, assume 50% luck/50% skill, focus on process.
---
## Framing Cluster
### Framing Effect
**Definition:** Same information, different presentation, different decision.
**Affects forecasting:** "90% survival" vs "10% death" changes estimate, format matters.
**Detect:** How is question framed? Do I get same answer both ways?
**Remediate:** Rephrase multiple ways, convert to neutral format, use frequency (100 out of 1000).
### Narrative Fallacy
**Definition:** Constructing simple stories to explain complex reality.
**Affects forecasting:** Post-hoc explanations feel compelling, smooth narratives overpower messy data.
**Detect:** Is story too clean? Can I fit multiple narratives to same data?
**Remediate:** Prefer statistics over stories, generate alternative narratives, use base rates.
---
## Temporal Biases
### Sunk Cost Fallacy
**Definition:** Continuing endeavor because of past investment, not future value.
**Affects forecasting:** "Invested $1M, can't stop now", hold losing positions too long.
**Detect:** If I started today, would I choose this? Considering only future costs/benefits?
**Remediate:** Consider only forward-looking value, treat sunk costs as irrelevant.
### Hindsight Bias
**Definition:** After outcome known, "I knew it all along."
**Affects forecasting:** Can't recall prior uncertainty, overestimate predictability, can't learn from surprises.
**Detect:** What did I actually predict beforehand? Written record exists?
**Remediate:** Write forecasts before outcome, record confidence levels, review predictions regularly.
### Planning Fallacy
**Definition:** Underestimate time, costs, risks; overestimate benefits.
**Affects forecasting:** Projects take 2-3x longer than planned, inside view ignores reference class.
**Detect:** How long did similar projects take? Using inside view only?
**Remediate:** Use reference class forecasting, add 2-3x buffer, consider outside view first.
### Outcome Bias
**Definition:** Judging decision quality by result, not by information available at time.
**Affects forecasting:** Good outcome ≠ good decision, can't separate luck from skill.
**Detect:** What did I know when I decided? Evaluating process or outcome?
**Remediate:** Judge decisions by process not results, evaluate with info available then.
---
## Pattern Recognition Biases
### Clustering Illusion
**Definition:** Seeing patterns in random data.
**Affects forecasting:** "Winning streak" in random sequence, stock "trends" that are noise, "hot hand" fallacy.
**Detect:** Is this statistically significant? Could this be random chance?
**Remediate:** Test statistical significance, use appropriate sample size, consider null hypothesis.
### Gambler's Fallacy
**Definition:** Believing random events "balance out" in short run.
**Affects forecasting:** "Due for a win" after losses, expecting mean reversion too quickly.
**Detect:** Are these events independent? Does past affect future probability?
**Remediate:** Recognize independent events, don't expect short-term balancing.
---
## Bayesian Reasoning Failures
### Base Rate Neglect
**Definition:** Ignoring prior probabilities, focusing only on new evidence.
**Affects forecasting:** "Test is 90% accurate" ignores base rate, vivid case study overrides statistics.
**Detect:** What's the base rate? Did I start with prior probability?
**Remediate:** Always start with base rate, update incrementally with evidence.
### Conjunction Fallacy
**Definition:** Believing specific scenario is more probable than general one.
**Affects forecasting:** "Librarian who likes poetry" > "Librarian", detailed scenarios feel more likely.
**Detect:** Is A&B more likely than A alone? Confusing plausibility with probability?
**Remediate:** Remember P(A&B) ≤ P(A), strip away narrative details.
---
## Social Biases
### Halo Effect
**Definition:** One positive trait colors perception of everything else.
**Affects forecasting:** Successful CEO → good at everything, one win → forecaster must be skilled.
**Detect:** Am I generalizing from one trait? Are dimensions actually correlated?
**Remediate:** Assess dimensions separately, don't assume correlation, judge each forecast independently.
### Authority Bias
**Definition:** Overweight opinions of authorities, underweight evidence.
**Affects forecasting:** "Expert said so" → must be true, defer to credentials over data.
**Detect:** What's expert's track record? Does evidence support claim?
**Remediate:** Evaluate expert track record, consider evidence not just credentials.
---
## Memory Biases
### Peak-End Rule
**Definition:** Judging experience by peak and end, ignoring duration and average.
**Affects forecasting:** Remember market peak, ignore average returns, distorted recall of sequences.
**Detect:** Am I remembering whole sequence? What was average not just peak/end?
**Remediate:** Review full historical record, calculate averages not memorable moments.
### Rosy Retrospection
**Definition:** Remembering past as better than it was.
**Affects forecasting:** "Things were better in old days", underestimate historical problems.
**Detect:** What do contemporary records show? Am I romanticizing the past?
**Remediate:** Consult historical data not memory, read contemporary accounts.
---
## Application to Forecasting
### Pre-Forecast Checklist
1. What's the base rate? (Base rate neglect)
2. Am I anchored on a number? (Anchoring)
3. Do I want this outcome? (Desirability bias)
4. What recent events am I recalling? (Availability)
5. Am I overconfident? (Overconfidence)
### During Forecast
1. Did I search for disconfirming evidence? (Confirmation)
2. Am I using inside or outside view? (Planning fallacy)
3. Is this pattern real or random? (Clustering illusion)
4. Am I framing this question neutrally? (Framing)
5. What would change my mind? (Motivated reasoning)
### Post-Forecast Review
1. Record what I predicted before (Hindsight bias)
2. Judge decision by process, not outcome (Outcome bias)
3. Attribute success/failure consistently (Self-serving bias)
4. Update calibration tracking (Overconfidence)
5. What did I learn? (Growth mindset)
---
## Bias Remediation Framework
**Five principles:**
1. **Awareness:** Know which biases affect you most
2. **Process:** Use checklists and frameworks
3. **Calibration:** Track accuracy over time
4. **Humility:** Assume you're biased, not immune
5. **Updating:** Learn from mistakes, adjust process
**Key insight:** You can't eliminate biases, but you can design systems that compensate for them.
---
**Return to:** [Main Skill](../SKILL.md#interactive-menu)

View File

@@ -0,0 +1,421 @@
# Debiasing Techniques
## A Practical Guide to Removing Bias from Forecasts
---
## The Systematic Debiasing Process
**The Four-Stage Framework:**
1. **Recognition** - Identify which biases are present, assess severity and direction
2. **Intervention** - Apply structured methods (not willpower), make bias mathematically impossible
3. **Validation** - Check if intervention worked, compare pre/post probabilities
4. **Institutionalization** - Build into routine process, create checklists, track effectiveness
**Key Principle:** You cannot "try harder" to avoid bias. Biases are unconscious. You need **systematic interventions**.
---
## Pre-Commitment Strategies
**Definition:** Locking in decision rules BEFORE seeing evidence, when you're still objective.
**Why it works:** Removes motivated reasoning by making updates automatic and mechanical.
### Technique 1: Pre-Registered Update Rules
**Before looking at evidence, write down:**
1. Current belief: "I believe X with Y% confidence"
2. Update rule: "If I observe Z, I will update to W%"
3. Decision criteria: "I will accept evidence Z as valid if it meets criteria Q"
**Example - Election Forecast:**
- Current: "Candidate A has 60% chance of winning"
- Update rule: "If next 3 polls show Candidate B ahead by >5%, I will update to 45%"
- Criteria: "Polls must be rated B+ or higher by 538, sample size >800, conducted in last 7 days"
**Prevents:** Cherry-picking polls, moving goalposts, asymmetric evidence standards
### Technique 2: Prediction Intervals with Triggers
**Method:** Set probability ranges that trigger re-evaluation.
**Example - Startup Valuation:**
```
If valuation announced is:
- <$50M → Update P(success) from 40% → 25%
- $50M-$100M → Keep at 40%
- $100M-$200M → Update to 55%
- >$200M → Update to 70%
```
Lock this in before you know the actual valuation.
**Prevents:** Post-hoc rationalization, scope insensitivity, anchoring
### Technique 3: Conditional Forecasting
**Method:** Make forecasts for different scenarios in advance.
**Example - Product Launch:**
- "If launch delayed >2 months: P(success) = 30%"
- "If launch on time: P(success) = 50%"
- "If launch early: P(success) = 45%"
When scenario occurs, use pre-committed probability.
**Prevents:** Status quo bias, under-updating when conditions change
---
## External Accountability
**Why it works:** Public predictions create reputational stake. Incentive to be accurate > incentive to be "right."
### Technique 4: Forecasting Tournaments
**Platforms:** Good Judgment Open, Metaculus, Manifold Markets, PredictIt
**How to use:**
1. Make 50-100 forecasts over 6 months
2. Track your Brier score (lower = better)
3. Review what you got wrong
4. Adjust your process
**Fixes:** Overconfidence, confirmation bias, motivated reasoning
### Technique 5: Public Prediction Logging
**Method:** Post forecasts publicly (Twitter, blog, Slack, email) before outcomes known.
**Format:**
```
Forecast: [Event]
Probability: X%
Date: [Today]
Resolution date: [When we'll know]
Reasoning: [2-3 sentences]
```
**Prevents:** Hindsight bias, selective memory, probability creep
### Technique 6: Forecasting Partners
**Method:** Find a "prediction buddy" who reviews your forecasts.
**Their job:** Ask "Why this probability and not 10% higher/lower?", point out motivated reasoning, suggest alternative reference classes, track systematic biases
**Prevents:** Blind spots, soldier mindset, lazy reasoning
---
## Algorithmic Aids
**Research finding:** Simple formulas outperform expert judgment. Formulas are consistent; humans are noisy.
### Technique 7: Base Rate + Adjustment Formula
```
Final Probability = Base Rate + (Adjustment × Confidence)
```
**Example - Startup Success:**
- Base rate: 10% (seed startups reach $10M revenue)
- Specific evidence: Great team (+20%), weak market fit (-10%), strong competition (-5%) = Net +5%
- Confidence in evidence: 0.5
- Calculation: 10% + (5% × 0.5) = 12.5%
**Prevents:** Ignoring base rates, overweighting anecdotes, inconsistent weighting
### Technique 8: Bayesian Update Calculator
**Formula:** Posterior Odds = Prior Odds × Likelihood Ratio
**Example - Medical Test:**
- Prior: 1% have disease X, Test: 90% true positive, 10% false positive, You test positive
- Prior odds: 1:99, Likelihood ratio: 0.9/0.1 = 9, Posterior odds: 9:99 = 1:11
- Posterior probability: 1/12 = 8.3%
**Lesson:** Even with positive test, only 8.3% chance (low base rate dominates).
**Tool:** Use online Bayes calculator or spreadsheet template.
### Technique 9: Ensemble Forecasting
**Method:** Use multiple methods, average results.
1. Reference class forecasting → X₁%
2. Inside view analysis → X₂%
3. Extrapolation from trends → X₃%
4. Expert consensus → X₄%
5. Weighted average: 0.4×(Ref class) + 0.3×(Inside) + 0.2×(Trends) + 0.1×(Expert)
**Prevents:** Over-reliance on single method, blind spots, methodology bias
---
## Consider-the-Opposite Technique
**Core Question:** "What would have to be true for the opposite outcome to occur?"
### Technique 10: Steelman the Opposite View
**Method:**
1. State your forecast: "70% probability Event X"
2. Build STRONGEST case for opposite (don't strawman, steelman)
3. Articulate so well that someone holding that view would agree
4. Re-evaluate probability based on strength
**Example - AGI Timeline:**
- Your view: "70% chance AGI by 2030"
- Steelman: "Every previous AI timeline wrong by decades, current systems lack common sense, scaling may hit limits, regulatory slowdowns, no clear path from LLMs to reasoning, hardware constraints, reference class: major tech breakthroughs take 20-40 years"
- Re-evaluation: "Hmm, that's strong. Updating to 45%."
### Technique 11: Ideological Turing Test
**Method:** Write 200-word argument for opposite of your forecast. Show to someone who holds that view. Ask "Can you tell I don't believe this?"
**If they can tell:** You don't understand the opposing view
**If they can't tell:** You've properly steelmanned it
**Prevents:** Strawmanning, tribalism, missing legitimate counter-arguments
---
## Red Teaming and Devil's Advocate
### Technique 12: Structured Red Team Review
**Roles (60-min session):**
- **Pessimist:** "What's worst case?" (10 min)
- **Optimist:** "Are we underestimating success?" (10 min)
- **Historian:** Find historical analogies that contradict forecast (10 min)
- **Statistician:** Check the math, CI width (10 min)
- **Devil's Advocate:** Argue opposite conclusion (10 min)
- **Moderator (You):** Listen without defending, take notes, update (15 min synthesis)
**No rebuttals allowed - just listen.**
### Technique 13: Premortem + Pre-parade
**Premortem:** "It's 1 year from now, prediction was WRONG (too low). Why?"
**Pre-parade:** "It's 1 year from now, prediction was WRONG (too high). Why?"
**Method:** Assume wrong in each direction, generate 5-10 plausible reasons. If BOTH lists are plausible → confidence too high, widen range.
---
## Calibration Training
**Calibration:** When you say "70%", it happens 70% of the time.
### Technique 14: Trivia-Based Calibration
**Method:** Answer 50 trivia questions with confidence levels. Group by confidence bucket. Calculate actual accuracy in each.
**Example results:**
| Confidence | # Questions | Actual Accuracy |
|------------|-------------|-----------------|
| 60-70% | 12 | 50% (overconfident) |
| 70-80% | 15 | 60% (overconfident) |
| 80-90% | 8 | 75% (overconfident) |
**Fix:** Lower confidence levels by 15-20 points. Repeat monthly until calibrated.
**Resources:** Calibrate.app, PredictionBook.com, Good Judgment Open
### Technique 15: Confidence Interval Training
**Exercise:** Answer questions with 80% confidence intervals (population of Australia, year Eiffel Tower completed, Earth-Moon distance, etc.)
**Your 80% CIs should capture true answer 80% of time.**
**Most people:** Capture only 40-50% (too narrow = overconfident)
**Training:** Do 20 questions/week, track hit rate, widen intervals until you hit 80%
---
## Keeping a Forecasting Journal
**Problem:** Memory unreliable - we remember hits, forget misses, unconsciously revise forecasts.
**Solution:** Written record
### Technique 16: Structured Forecast Log
**Format:**
```
=== FORECAST #[Number] ===
Date: [YYYY-MM-DD]
Question: [Precise, falsifiable]
Resolution Date: [When we'll know]
Base Rate: [Reference class frequency]
My Probability: [X%]
Confidence Interval: [Lower - Upper]
REASONING:
- Reference class: [Which, why]
- Evidence for: [Bullets]
- Evidence against: [Bullets]
- Main uncertainty: [What could change]
- Biases checked: [Techniques used]
OUTCOME (fill later):
Actual: [Yes/No or value]
Brier Score: [Calculated]
What I learned: [Post-mortem]
```
### Technique 17: Monthly Calibration Review
**Process (last day of month):**
1. Review all resolved forecasts
2. Calculate: Brier score, calibration plot, trend
3. Identify patterns: Which forecast types wrong? Which biases recurring?
4. Adjust process: Add techniques, adjust confidence levels
5. Set goals: "Reduce Brier by 0.05", "Achieve 75-85% calibration on 80% forecasts"
---
## Practicing on Low-Stakes Predictions
**Problem:** High-stakes forecasts bad for learning (emotional, rare, outcome bias).
**Solution:** Practice on low-stakes, fast-resolving questions.
### Technique 18: Daily Micro-Forecasts
**Method:** Make 1-5 small predictions daily.
**Examples:** "Will it rain tomorrow?" (70%), "Email response <24h?" (80%), "Meeting on time?" (40%)
**Benefits:** Fast feedback (hours/days), low stakes, high volume (100+/month), rapid iteration
**Track in spreadsheet, calculate rolling Brier score weekly.**
### Technique 19: Sports Forecasting Practice
**Why sports:** Clear resolution, abundant data, frequent events, low stakes, good reference classes
**Method (weekly session):**
1. Pick 10 upcoming games
2. Research: team records, head-to-head, injuries, home/away splits
3. Make probability forecasts
4. Compare to Vegas odds (well-calibrated baseline)
5. Track accuracy
**Goal:** Get within 5% of Vegas odds consistently
**Skills practiced:** Reference class, Bayesian updating, regression to mean, base rate anchoring
---
## Team Forecasting Protocols
**Team problems:** Groupthink, herding (anchor on first speaker), authority bias, social desirability
### Technique 20: Independent Then Combine
**Protocol:**
1. **Independent (15 min):** Each makes forecast individually, no discussion, submit to moderator
2. **Reveal (5 min):** Show all anonymously, display range, calculate median
3. **Discussion (20 min):** Outliers speak first, others respond, no splitting difference
4. **Re-forecast (10 min):** Update independently, can stay at original
5. **Aggregate:** Median of final forecasts = team forecast (or weighted by track record, or extremize median)
**Prevents:** Anchoring on first speaker, groupthink, authority bias
### Technique 21: Delphi Method
**Protocol:** Multi-round expert elicitation
- **Round 1:** Forecast independently, provide reasoning, submit anonymously
- **Round 2:** See Round 1 summary (anonymized), read reasoning, revise if convinced
- **Round 3:** See Round 2 summary, make final forecast
- **Final:** Median of Round 3
**Prevents:** Loud voice dominance, social pressure, first-mover anchoring
**Use when:** High-stakes forecasts, diverse expert team
### Technique 22: Red Team / Blue Team Split
**Setup:**
- **Blue Team:** Argues forecast should be HIGH
- **Red Team:** Argues forecast should be LOW
- **Gray Team:** Judges and synthesizes
**Process (75 min):**
1. Preparation (20 min): Each team finds evidence for their side
2. Presentations (30 min): Blue presents (10), Red presents (10), Gray questions (10)
3. Deliberation (15 min): Gray weighs evidence, makes forecast
4. Debrief (10 min): All reconvene, discuss learning
**Prevents:** Confirmation bias, groupthink, missing arguments
---
## Quick Reference: Technique Selection Guide
**Overconfidence:** → Calibration training (#14, #15), Premortem (#13)
**Confirmation Bias:** → Consider-opposite (#10), Red teaming (#12), Steelman (#10)
**Anchoring:** → Independent-then-combine (#20), Pre-commitment (#1), Ensemble (#9)
**Base Rate Neglect:** → Base rate formula (#7), Reference class + adjustment (#7)
**Availability Bias:** → Statistical lookup, Forecasting journal (#16)
**Motivated Reasoning:** → Pre-commitment (#1, #2), Public predictions (#5), Tournaments (#4)
**Scope Insensitivity:** → Algorithmic scaling (#7), Reference class by magnitude
**Status Quo Bias:** → Pre-parade (#13), Consider-opposite (#10)
**Groupthink:** → Independent-then-combine (#20), Delphi (#21), Red/Blue teams (#22)
---
## Integration into Forecasting Workflow
**Before forecast:** Pre-commitment (#1, #2, #3), Independent (if team) (#20)
**During forecast:** Algorithmic aids (#7, #8, #9), Consider-opposite (#10, #11)
**After initial:** Red teaming (#12, #13), Calibration check (#14, #15)
**Before finalizing:** Journal entry (#16), Public logging (#5)
**After resolution:** Journal review (#17), Calibration analysis (#17)
**Ongoing practice:** Micro-forecasts (#18), Sports (#19), Tournaments (#4)
---
## The Minimum Viable Debiasing Process
**If you only do THREE things:**
**1. Pre-commit to update rules** - Write "If I see X, I'll update to Y" before evidence (prevents motivated reasoning)
**2. Keep a forecasting journal** - Log all forecasts with reasoning, review monthly (prevents hindsight bias)
**3. Practice on low-stakes predictions** - Make 5-10 micro-forecasts/week (reduces overconfidence)
---
## Summary Table
| Technique | Bias Addressed | Difficulty | Time |
|-----------|----------------|------------|------|
| Pre-registered updates (#1) | Motivated reasoning | Easy | 5 min |
| Prediction intervals (#2) | Scope insensitivity | Easy | 5 min |
| Conditional forecasting (#3) | Status quo bias | Medium | 10 min |
| Tournaments (#4) | Overconfidence | Easy | Ongoing |
| Public logging (#5) | Hindsight bias | Easy | 2 min |
| Forecasting partners (#6) | Blind spots | Medium | Ongoing |
| Base rate formula (#7) | Base rate neglect | Easy | 3 min |
| Bayesian calculator (#8) | Update errors | Medium | 5 min |
| Ensemble methods (#9) | Method bias | Medium | 15 min |
| Steelman opposite (#10) | Confirmation bias | Medium | 10 min |
| Turing test (#11) | Tribalism | Hard | 20 min |
| Red team review (#12) | Groupthink | Hard | 60 min |
| Premortem/Pre-parade (#13) | Overconfidence | Medium | 15 min |
| Trivia calibration (#14) | Overconfidence | Easy | 20 min |
| CI training (#15) | Overconfidence | Easy | 15 min |
| Forecast journal (#16) | Multiple | Easy | 5 min |
| Monthly review (#17) | Calibration drift | Medium | 30 min |
| Micro-forecasts (#18) | Overconfidence | Easy | 5 min/day |
| Sports practice (#19) | Multiple | Medium | 30 min/week |
| Independent-then-combine (#20) | Groupthink | Medium | 50 min |
| Delphi method (#21) | Authority bias | Hard | 90 min |
| Red/Blue teams (#22) | Confirmation bias | Hard | 75 min |
---
**Return to:** [Main Skill](../SKILL.md#interactive-menu)

View File

@@ -0,0 +1,392 @@
# Scout vs Soldier Mindset
## Julia Galef's Framework
### The Two Mindsets
**Soldier Mindset:**
- Goal: Defend your position
- Metaphor: Protecting territory
- Reasoning: Motivated by desired conclusion
- Evidence: Cherry-pick what supports you
- Beliefs: Fortifications to defend
- Identity: Tied to being right
**Scout Mindset:**
- Goal: Map the territory accurately
- Metaphor: Exploring terrain
- Reasoning: Motivated by accuracy
- Evidence: Seek all relevant data
- Beliefs: Working hypotheses
- Identity: Tied to good process
---
## Why Soldier Mindset Exists
### Evolutionary Benefits
**In ancestral environment:**
- Defending tribe increased survival
- Loyalty signaled trustworthiness
- Confidence attracted mates/followers
- Changing mind showed weakness
**Result:** Humans evolved to defend positions, not seek truth.
---
### Social Benefits (Today)
**Soldier mindset helps with:**
- Tribal signaling (showing loyalty)
- Self-esteem (feeling right)
- Persuasion (confidence convinces)
- Belonging (fitting in)
**The trap:** These benefits come at the cost of accuracy.
---
## Why Scout Mindset Wins (For Forecasting)
### Accuracy Beats Persuasion
**In forecasting:**
- No one cares if you "sound confident"
- The territory doesn't care about your loyalty
- Reality punishes inaccuracy
**Scouts get promoted. Soldiers die in wrong positions.**
---
### Long-Term Reputation
**Scout mindset builds:**
- Credibility (you admit mistakes)
- Trust (not ideologically motivated)
- Respect (intellectually honest)
**Soldier mindset destroys:**
- Credibility (predict confidently, wrong often)
- Trust (clearly biased)
- Respect (never update beliefs)
---
## Recognizing Soldier Mindset in Yourself
### Telltale Signs
**1. Directional Motivated Reasoning**
- You want a specific conclusion
- Evidence is ammunition, not information
- You feel defensive when challenged
**2. Asymmetric Evidence Standards**
- Friendly evidence: Low bar
- Unfriendly evidence: High bar
- "I need extraordinary proof for claims I dislike"
**3. Identity Protective Cognition**
- Changing mind feels like betrayal
- Beliefs are tribal markers
- Being wrong = being a bad person
**4. Affect Heuristic**
- If I like it → It must be true
- If I dislike it → It must be false
- Emotions drive conclusions
---
## Cultivating Scout Mindset
### 1. Separate Identity from Beliefs
**Soldier:** "I am a [belief-holder]"
- "I am a Democrat"
- "I am an atheist"
- "I am a Bitcoin believer"
**Scout:** "I currently believe [X] with [Y]% confidence"
- "I lean left on policy with 70% confidence"
- "I assign 90% probability God doesn't exist"
- "I think Bitcoin has 40% chance of long-term success"
**Why this helps:**
- Beliefs become probabilities, not identities
- Updating doesn't threaten who you are
- Confidence is explicit, not assumed
---
### 2. Reframe "Being Wrong"
**Soldier framing:**
- Being wrong = Failure
- Changing mind = Weakness
- Admitting error = Losing
**Scout framing:**
- Being wrong = Learning
- Changing mind = Updating
- Admitting error = Intellectual honesty
**Practice:**
- Celebrate discovering you were wrong (you learned something!)
- Track your updates publicly (builds reputation for honesty)
- Say "I changed my mind because..." not "I was wrong, sorry"
---
### 3. Make Accuracy Your Goal
**Soldier goal:** Win the argument
**Scout goal:** Get the right answer
**Practical shift:**
- Don't ask "How can I defend this?"
- Ask "What would I need to see to change my mind?"
---
### 4. Value Process Over Outcome
**Outcome-focused (Soldier):**
- Judge decisions by results
- Outcome bias (good outcome = good decision)
- Rewarded for being lucky
**Process-focused (Scout):**
- Judge decisions by reasoning
- Good process despite bad outcome = Success
- Rewarded for good epistemics
---
## Motivated Reasoning
### What It Is
**Definition:** Unconsciously selecting evidence and arguments that support a desired conclusion.
**Key word:** Unconscious. You genuinely believe you're being objective.
---
### How It Works
**Normal reasoning:**
1. Gather evidence
2. Weigh evidence
3. Form conclusion
**Motivated reasoning:**
1. Have desired conclusion
2. Search for supporting evidence
3. Stop when you have "enough"
4. Ignore or discount contradictory evidence
5. Conclude what you wanted from the start
**Feels like:** Objective analysis
**Actually is:** Rationalization
---
### Detecting Motivated Reasoning
**Ask yourself:**
**1. Do I want this conclusion to be true?**
- If yes → High risk of motivated reasoning
**2. Would I be bothered if evidence contradicted this?**
- If yes → Motivated reasoning likely
**3. Am I trying to prove X or trying to find the truth?**
- If "prove X" → Soldier mindset active
**4. If I was wrong, how would I know?**
- If "I wouldn't be wrong" → Motivated reasoning
---
## Intellectual Honesty
### What It Looks Like
**Honest forecaster:**
- States assumptions clearly
- Admits uncertainties
- Shows work (not just conclusions)
- Updates when wrong
- Says "I don't know" when appropriate
- Gives credit to alternative views
**Dishonest (or self-deceiving) forecaster:**
- Hides assumptions
- Expresses false certainty
- Only shares conclusions
- Never updates
- Always has an answer
- Strawmans opposing views
---
### The "Outsourcing Test"
**Question:** "If I hired someone else to make this forecast, and I paid them for accuracy, would they reach the same conclusion?"
**If NO:** You're probably letting motivated reasoning distort your judgment.
---
## Identity and Beliefs
### The Trap
**When beliefs become identity:**
- Challenging belief = Challenging identity
- Changing belief = Losing self
- Defense becomes automatic
**Example:**
- "I am a Democrat" → Can't update on any right-wing policy
- "I am an entrepreneur" → Can't admit my startup will fail
- "I am a skeptic" → Can't update toward belief in X
---
### The Fix: Hold Beliefs Lightly
**Strong opinions, loosely held**
**Distinguish:**
- **Values:** Core identity, rarely change (e.g., "I value honesty")
- **Beliefs:** Maps of the world, update frequently (e.g., "I think X is true")
**Forecasting-compatible identity:**
- "I am someone who seeks truth"
- "I am calibrated"
- "I update my beliefs with evidence"
**NOT:**
- "I am bullish on crypto"
- "I am a pessimist about AI"
- "I am a contrarian"
---
## Practical Exercises
### 1. The "And I Could Be Wrong" Suffix
**Practice:**
Every time you state a belief, add: "...and I could be wrong."
**Example:**
- "I think this startup will succeed... and I could be wrong."
**Why it helps:**
- Reminds you beliefs are probabilistic
- Reduces emotional attachment
- Signals intellectual humility
---
### 2. Steelman the Opposition
**Don't strawman** (weak version of opposing view)
**Do steelman** (strongest version of opposing view)
**Method:**
1. State opposing view
2. Make it as strong as possible
3. Articulate why someone smart would believe it
4. Then and only then evaluate it
**Example:**
**Strawman:** "Skeptics of Bitcoin just don't understand technology."
**Steelman:** "Bitcoin skeptics note that high volatility makes it poor as currency, energy costs are unsustainable, regulatory risk is high, and most use cases can be solved with traditional databases. These are serious, well-founded concerns."
---
### 3. Ideological Turing Test
**Test:** Can you explain the opposing view so well that someone from that side can't tell you're not one of them?
**If YES:** You understand the position
**If NO:** You're strawmanning (Soldier mindset)
---
### 4. Pre-Commitment to Update
**Before looking at evidence:**
**State:**
- "I currently believe X with Y% confidence"
- "If I see evidence Z, I will update to W%"
**Why this helps:**
- Locks in update rule before emotions engage
- Prevents post-hoc rationalization
- Makes updating mechanical, not emotional
---
## Integration with Forecasting
### Scout Mindset Improves Every Step
**1. Reference Class Selection**
- Scout: Choose class objectively
- Soldier: Choose class that supports conclusion
**2. Evidence Gathering**
- Scout: Seek disconfirming evidence
- Soldier: Seek confirming evidence
**3. Probability Estimation**
- Scout: Calibrate based on accuracy
- Soldier: Express confidence to persuade
**4. Updating**
- Scout: Update incrementally with evidence
- Soldier: Stick to position regardless
**5. Confidence Intervals**
- Scout: Wide CIs reflecting uncertainty
- Soldier: Narrow CIs to sound confident
---
## Summary
**Scout Mindset:**
- Goal: Accuracy
- Process: Seek truth
- Beliefs: Probabilities
- Identity: Good process
- Updates: Frequently
- Confidence: Calibrated
**Soldier Mindset:**
- Goal: Win argument
- Process: Defend position
- Beliefs: Certainties
- Identity: Being right
- Updates: Rarely
- Confidence: Overconfident
**For forecasting:** Be a scout. Soldiers get killed by reality.
---
**Return to:** [Main Skill](../SKILL.md#interactive-menu)