Initial commit

skills/ethics-safety-impact/resources/methodology.md

@@ -0,0 +1,416 @@
+# Ethics, Safety & Impact Assessment Methodology
+
+Advanced techniques for fairness metrics, privacy analysis, safety assessment, bias detection, and participatory design.
+
+## Workflow
+
+```
+Ethics & Safety Assessment Progress:
+- [ ] Step 1: Map stakeholders and identify vulnerable groups
+- [ ] Step 2: Analyze potential harms and benefits
+- [ ] Step 3: Assess fairness and differential impacts
+- [ ] Step 4: Evaluate severity and likelihood
+- [ ] Step 5: Design mitigations and safeguards
+- [ ] Step 6: Define monitoring and escalation protocols
+```
+
+**Step 1: Map stakeholders and identify vulnerable groups**
+
+Apply stakeholder analysis and vulnerability assessment frameworks.
+
+**Step 2: Analyze potential harms and benefits**
+
+Use harm taxonomies and benefit frameworks to systematically catalog impacts.
+
+**Step 3: Assess fairness and differential impacts**
+
+Apply [1. Fairness Metrics](#1-fairness-metrics) to measure group disparities.
+
+**Step 4: Evaluate severity and likelihood**
+
+Use [2. Safety Assessment Methods](#2-safety-assessment-methods) for safety-critical systems.
+
+**Step 5: Design mitigations and safeguards**
+
+Apply [3. Mitigation Strategies](#3-mitigation-strategies) and [4. Privacy-Preserving Techniques](#4-privacy-preserving-techniques).
+
+**Step 6: Define monitoring and escalation protocols**
+
+Implement [5. Bias Detection in Deployment](#5-bias-detection-in-deployment) and participatory oversight.
+
+---
+
+## 1. Fairness Metrics
+
+Mathematical definitions of fairness for algorithmic systems.
+
+### Group Fairness Metrics
+
+**Demographic Parity** (Statistical Parity, Independence)
+- **Definition**: Positive outcome rate equal across groups
+- **Formula**: P(Ŷ=1 | A=a) = P(Ŷ=1 | A=b) for all groups a, b
+- **When to use**: When equal representation in positive outcomes is goal (admissions, hiring pipelines)
+- **Limitations**: Ignores base rates, may require different treatment to achieve equal outcomes
+- **Example**: 40% approval rate for all racial groups
+
+**Equalized Odds** (Error Rate Balance)
+- **Definition**: False positive and false negative rates equal across groups
+- **Formula**: P(Ŷ=1 | Y=y, A=a) = P(Ŷ=1 | Y=y, A=b) for all y, a, b
+- **When to use**: When fairness means equal error rates (lending, criminal justice, medical diagnosis)
+- **Strengths**: Accounts for true outcomes, balances burden of errors
+- **Example**: 5% false positive rate and 10% false negative rate for all groups
+
+**Equal Opportunity** (True Positive Rate Parity)
+- **Definition**: True positive rate equal across groups (among qualified individuals)
+- **Formula**: P(Ŷ=1 | Y=1, A=a) = P(Ŷ=1 | Y=1, A=b)
+- **When to use**: When accessing benefit/opportunity is key concern (scholarships, job offers)
+- **Strengths**: Ensures qualified members of all groups have equal chance
+- **Example**: 80% of qualified applicants from each group receive offers
+
+**Calibration** (Test Fairness)
+- **Definition**: Predicted probabilities match observed frequencies for all groups
+- **Formula**: P(Y=1 | Ŷ=p, A=a) = P(Y=1 | Ŷ=p, A=b) = p
+- **When to use**: When probability scores used for decision-making (risk scores, credit scores)
+- **Strengths**: Predictions are well-calibrated across groups
+- **Example**: Among all applicants scored 70%, 70% actually repay loan in each group
+
+**Disparate Impact** (80% Rule, Four-Fifths Rule)
+- **Definition**: Selection rate for protected group ≥ 80% of selection rate for reference group
+- **Formula**: P(Ŷ=1 | A=protected) / P(Ŷ=1 | A=reference) ≥ 0.8
+- **When to use**: Legal compliance (EEOC guidelines for hiring, lending)
+- **Regulatory threshold**: <0.8 triggers investigation, <0.5 strong evidence of discrimination
+- **Example**: If 50% of white applicants hired, ≥40% of Black applicants should be hired
+
+### Individual Fairness Metrics
+
+**Similar Individuals Treated Similarly** (Lipschitz Fairness)
+- **Definition**: Similar inputs receive similar outputs, regardless of protected attributes
+- **Formula**: d(f(x), f(x')) ≤ L · d(x, x') where d is distance metric, L is Lipschitz constant
+- **When to use**: When individual treatment should depend on relevant factors only
+- **Challenge**: Defining "similarity" in a fair way (what features are relevant?)
+- **Example**: Two loan applicants with same income, credit history get similar rates regardless of race
+
+**Counterfactual Fairness**
+- **Definition**: Outcome would be same if protected attribute were different (causal fairness)
+- **Formula**: P(Ŷ | A=a, X=x) = P(Ŷ | A=a', X=x) for all a, a'
+- **When to use**: When causal reasoning appropriate, can model interventions
+- **Strengths**: Captures intuition "would outcome change if only race/gender differed?"
+- **Example**: Applicant's loan decision same whether they're coded male or female
+
+### Fairness Tradeoffs
+
+**Impossibility results**: Cannot satisfy all fairness definitions simultaneously (except in trivial cases)
+
+Key tradeoffs:
+- **Demographic parity vs. calibration**: If base rates differ, cannot have both (Chouldechova 2017)
+- **Equalized odds vs. calibration**: Generally incompatible unless perfect accuracy (Kleinberg et al. 2017)
+- **Individual vs. group fairness**: Treating similar individuals similarly may still produce group disparities
+
+**Choosing metrics**: Context-dependent
+- **High-stakes binary decisions** (hire/fire, admit/reject): Equalized odds or equal opportunity
+- **Scored rankings** (credit scores, risk assessments): Calibration
+- **Access to benefits** (scholarships, programs): Demographic parity or equal opportunity
+- **Legal compliance**: Disparate impact (80% rule)
+
+### Fairness Auditing Process
+
+1. **Identify protected groups**: Race, gender, age, disability, religion, national origin, etc.
+2. **Collect disaggregated data**: Outcome metrics by group (requires demographic data collection with consent)
+3. **Compute fairness metrics**: Calculate demographic parity, equalized odds, disparate impact across groups
+4. **Test statistical significance**: Are differences statistically significant or due to chance? (Chi-square, t-tests)
+5. **Investigate causes**: If unfair, why? Biased training data? Proxy features? Measurement error?
+6. **Iterate on mitigation**: Debiasing techniques, fairness constraints, data augmentation, feature engineering
+
+---
+
+## 2. Safety Assessment Methods
+
+Systematic techniques for identifying and mitigating safety risks.
+
+### Failure Mode and Effects Analysis (FMEA)
+
+**Purpose**: Identify ways system can fail and prioritize mitigation efforts
+
+**Process**:
+1. **Decompose system**: Break into components, functions
+2. **Identify failure modes**: For each component, how can it fail? (hardware failure, software bug, human error, environmental condition)
+3. **Analyze effects**: What happens if this fails? Local effect? System effect? End effect?
+4. **Score severity** (1-10): 1 = negligible, 10 = catastrophic (injury, death)
+5. **Score likelihood** (1-10): 1 = rare, 10 = very likely
+6. **Score detectability** (1-10): 1 = easily detected, 10 = undetectable before harm
+7. **Compute Risk Priority Number (RPN)**: RPN = Severity × Likelihood × Detectability
+8. **Prioritize**: High RPN = high priority for mitigation
+9. **Design mitigations**: Eliminate failure mode, reduce likelihood, improve detection, add safeguards
+10. **Re-compute RPN**: Has mitigation adequately reduced risk?
+
+**Example - Medical AI diagnosis**:
+- **Failure mode**: AI misclassifies cancer as benign
+- **Effect**: Patient not treated, cancer progresses, death
+- **Severity**: 10 (death)
+- **Likelihood**: 3 (5% false negative rate)
+- **Detectability**: 8 (hard to catch without second opinion)
+- **RPN**: 10×3×8 = 240 (high, requires mitigation)
+- **Mitigation**: Human review of all negative diagnoses, require 2nd AI model for confirmation, patient follow-up at 3 months
+- **New RPN**: 10×1×3 = 30 (acceptable)
+
+### Fault Tree Analysis (FTA)
+
+**Purpose**: Identify root causes that lead to hazard (top-down causal reasoning)
+
+**Process**:
+1. **Define top event** (hazard): e.g., "Patient receives wrong medication"
+2. **Work backward**: What immediate causes could lead to top event?
+   - Use logic gates: AND (all required), OR (any sufficient)
+3. **Decompose recursively**: For each cause, what are its causes?
+4. **Reach basic events**: Hardware failure, software bug, human error, environmental
+5. **Compute probability**: If basic event probabilities known, compute probability of top event
+6. **Find minimal cut sets**: Smallest combinations of basic events that cause top event
+7. **Prioritize mitigations**: Address minimal cut sets (small changes with big safety impact)
+
+**Example - Wrong medication**:
+- Top event: Patient receives wrong medication (OR gate)
+  - Path 1: Prescription error (AND gate)
+    - Doctor prescribes wrong drug (human error)
+    - Pharmacist doesn't catch (human error)
+  - Path 2: Dispensing error (AND gate)
+    - Correct prescription but wrong drug selected (human error)
+    - Barcode scanner fails (equipment failure)
+  - Path 3: Administration error
+    - Nurse administers wrong drug (human error)
+- Minimal cut sets: {Doctor error AND Pharmacist error}, {Dispensing error AND Scanner failure}, {Nurse error}
+- Mitigation: Double-check systems (reduce AND probability), barcode scanning (detect errors), nurse training (reduce error rate)
+
+### Hazard and Operability Study (HAZOP)
+
+**Purpose**: Systematic brainstorming to find deviations from intended operation
+
+**Process**:
+1. **Divide system into nodes**: Functional components
+2. **For each node, apply guide words**: NO, MORE, LESS, AS WELL AS, PART OF, REVERSE, OTHER THAN
+3. **Identify deviations**: Guide word + parameter (e.g., "MORE pressure", "NO flow")
+4. **Analyze causes**: What could cause this deviation?
+5. **Analyze consequences**: What harm results?
+6. **Propose safeguards**: Detection, prevention, mitigation
+
+**Example - Content moderation system**:
+- Node: Content moderation AI
+- Deviation: "MORE false positives" (over-moderation)
+  - Causes: Model too aggressive, training data skewed, threshold too low
+  - Consequences: Silencing legitimate speech, especially marginalized voices
+  - Safeguards: Appeals process, human review sample, error rate dashboard by demographic
+- Deviation: "NO moderation" (under-moderation)
+  - Causes: Model failure, overwhelming volume, adversarial evasion
+  - Consequences: Harmful content remains, harassment, misinformation spreads
+  - Safeguards: Redundant systems, rate limiting, user reporting, human backup
+
+### Worst-Case Scenario Analysis
+
+**Purpose**: Stress-test system against extreme but plausible threats
+
+**Process**:
+1. **Brainstorm worst cases**: Adversarial attacks, cascading failures, edge cases, Murphy's Law
+2. **Assess plausibility**: Could this actually happen? Historical precedents?
+3. **Estimate impact**: If it happened, how bad?
+4. **Identify single points of failure**: What one thing, if it fails, causes catastrophe?
+5. **Design resilience**: Redundancy, fail-safes, graceful degradation, circuit breakers
+6. **Test**: Chaos engineering, red teaming, adversarial testing
+
+**Examples**:
+- **AI model**: Adversary crafts inputs that fool model (adversarial examples) → Test robustness, ensemble models
+- **Data breach**: All user data leaked → Encrypt data, minimize collection, differential privacy
+- **Bias amplification**: Feedback loop causes AI to become more biased over time → Monitor drift, periodic retraining, fairness constraints
+- **Denial of service**: System overwhelmed by load → Rate limiting, auto-scaling, graceful degradation
+
+---
+
+## 3. Mitigation Strategies
+
+Taxonomy of interventions to reduce harm.
+
+### Prevention (Eliminate Harm)
+
+**Design out the risk**:
+- Don't collect sensitive data you don't need (data minimization)
+- Don't build risky features (dark patterns, addictive mechanics, manipulation)
+- Use less risky alternatives (aggregate statistics vs. individual data, contextual recommendations vs. behavioral targeting)
+
+**Examples**:
+- Instead of collecting browsing history, use contextual ads (keywords on current page)
+- Instead of infinite scroll (addiction), paginate with clear endpoints
+- Instead of storing plaintext passwords, use salted hashes (can't be leaked)
+
+### Reduction (Decrease Likelihood or Severity)
+
+**Technical mitigations**:
+- Rate limiting (prevent abuse)
+- Friction (slow down impulsive harmful actions - time delays, confirmations, warnings)
+- Debiasing algorithms (pre-processing data, in-processing fairness constraints, post-processing calibration)
+- Differential privacy (add noise to protect individuals while preserving aggregate statistics)
+
+**Process mitigations**:
+- Staged rollouts (limited exposure to catch problems early)
+- A/B testing (measure impact before full deployment)
+- Diverse teams (more perspectives catch more problems)
+- External audits (independent review)
+
+**Examples**:
+- Limit posts per hour to prevent spam
+- Require confirmation before deleting account or posting sensitive content
+- Apply fairness constraints during model training to reduce disparate impact
+- Release to 1% of users, monitor for issues, then scale
+
+### Detection (Monitor and Alert)
+
+**Dashboards**: Real-time metrics on harm indicators (error rates by group, complaints, safety incidents)
+
+**Anomaly detection**: Alert when metrics deviate from baseline (spike in false positives, drop in engagement from specific group)
+
+**User reporting**: Easy channels for reporting harms, responsive investigation
+
+**Audit logs**: Track decisions for later investigation (who accessed what data, which users affected by algorithm)
+
+**Examples**:
+- Bias dashboard showing approval rates by race, gender, age updated daily
+- Alert if moderation false positive rate >2× baseline for any language
+- "Report this" button on all content with category options
+- Log all loan denials with reason codes for audit
+
+### Response (Address Harm When Found)
+
+**Appeals**: Process for contesting decisions (human review, overturn if wrong)
+
+**Redress**: Compensate those harmed (refunds, apologies, corrective action)
+
+**Incident response**: Playbook for handling harms (who to notify, how to investigate, when to escalate, communication plan)
+
+**Iterative improvement**: Learn from incidents to prevent recurrence
+
+**Examples**:
+- Allow users to appeal content moderation decisions, review within 48 hours
+- Offer compensation to users affected by outage or data breach
+- If bias detected, pause system, investigate, retrain model, re-audit before re-launch
+- Publish transparency report on harms, mitigations, outcomes
+
+### Safeguards (Redundancy and Fail-Safes)
+
+**Human oversight**: Human in the loop (review all decisions) or human on the loop (review samples, alert on anomalies)
+
+**Redundancy**: Multiple independent systems, consensus required
+
+**Fail-safes**: If system fails, default to safe state (e.g., medical device fails → alarm, not silent failure)
+
+**Circuit breakers**: Kill switches to shut down harmful features quickly
+
+**Examples**:
+- High-stakes decisions (loan denial, medical diagnosis, criminal sentencing) require human review
+- Two independent AI models must agree before autonomous action
+- If fraud detection fails, default to human review rather than approving all transactions
+- CEO can halt product launch if ethics concerns raised, even at last minute
+
+---
+
+## 4. Privacy-Preserving Techniques
+
+Methods to protect individual privacy while enabling data use.
+
+### Data Minimization
+
+- **Collect only necessary data**: Purpose limitation (collect only for stated purpose), don't collect "just in case"
+- **Aggregate where possible**: Avoid individual-level data when population-level sufficient
+- **Short retention**: Delete data when no longer needed, enforce retention limits
+
+### De-identification
+
+**Anonymization**: Remove direct identifiers (name, SSN, email)
+- **Limitation**: Re-identification attacks possible (linkage to other datasets, inference from quasi-identifiers)
+- **Example**: Netflix dataset de-identified, but researchers re-identified users by linking to IMDB reviews
+
+**K-anonymity**: Each record indistinguishable from k-1 others (generalize quasi-identifiers like zip, age, gender)
+- **Limitation**: Attribute disclosure (if all k records have same sensitive attribute), composition attacks
+- **Example**: {Age=32, Zip=12345} → {Age=30-35, Zip=123**}
+
+**Differential Privacy**: Add calibrated noise such that individual's presence/absence doesn't significantly change query results
+- **Definition**: P(M(D) = O) / P(M(D') = O) ≤ e^ε where D, D' differ by one person, ε is privacy budget
+- **Strengths**: Provable privacy guarantee, composes well, resistant to post-processing
+- **Limitations**: Accuracy-privacy tradeoff (more privacy → more noise → less accuracy), privacy budget exhausted over many queries
+- **Example**: Census releases aggregate statistics with differential privacy (Apple, Google use for local learning)
+
+### Access Controls
+
+- **Least privilege**: Users access only data needed for their role
+- **Audit logs**: Track who accessed what data when, detect anomalies
+- **Encryption**: At rest (storage), in transit (network), in use (processing)
+- **Multi-party computation**: Compute on encrypted data without decrypting
+
+### Consent and Control
+
+- **Granular consent**: Opt-in for each purpose, not blanket consent
+- **Transparency**: Explain what data collected, how used, who it's shared with (in plain language)
+- **User controls**: Export data (GDPR right to portability), delete data (right to erasure), opt-out of processing
+- **Meaningful choice**: Consent not coerced (service available without consent for non-essential features)
+
+---
+
+## 5. Bias Detection in Deployment
+
+Ongoing monitoring to detect and respond to bias post-launch.
+
+### Bias Dashboards
+
+**Disaggregated metrics**: Track outcomes by protected groups (race, gender, age, disability)
+- Approval/rejection rates
+- False positive/negative rates
+- Recommendation quality (precision, recall, ranking)
+- User engagement (click-through, conversion, retention)
+
+**Visualizations**:
+- Bar charts showing metric by group, flag >20% disparities
+- Time series to detect drift (bias increasing over time?)
+- Heatmaps for intersectional analysis (race × gender)
+
+**Alerting**: Automated alerts when disparity crosses threshold
+
+### Drift Detection
+
+**Distribution shift**: Has data distribution changed since training? (Covariate shift, concept drift)
+- Monitor input distributions, flag anomalies
+- Retrain periodically on recent data
+
+**Performance degradation**: Is model accuracy declining? For which groups?
+- A/B test new model vs. old continuously
+- Track metrics by group, ensure improvements don't harm any group
+
+**Feedback loops**: Is model changing environment in ways that amplify bias?
+- Example: Predictive policing → more arrests in flagged areas → more training data from those areas → more policing (vicious cycle)
+- Monitor for amplification: Are disparities increasing over time?
+
+### Participatory Auditing
+
+**Stakeholder involvement**: Include affected groups in oversight, not just internal teams
+- Community advisory boards
+- Public comment periods
+- Transparency reports reviewed by civil society
+
+**Contests**: Bug bounties for finding bias (reward researchers/users who identify fairness issues)
+
+**External audits**: Independent third-party assessment (not self-regulation)
+
+---
+
+## 6. Common Pitfalls
+
+**Fairness theater**: Performative statements without material changes. Impact assessments must change decisions, not just document them.
+
+**Sampling bias in testing**: Testing only on employees (young, educated, English-speaking) misses how diverse users experience harm. Test with actual affected populations.
+
+**Assuming "colorblind" = fair**: Not collecting race data doesn't eliminate bias, it makes bias invisible and impossible to audit. Collect demographic data (with consent and safeguards) to measure fairness.
+
+**Optimization without constraints**: Maximizing engagement/revenue unconstrained leads to amplifying outrage, addiction, polarization. Set ethical boundaries as constraints, not just aspirations.
+
+**Privacy vs. fairness tradeoff**: Can't audit bias without demographic data. Balance: Collect minimal data necessary for fairness auditing, strong access controls, differential privacy.
+
+**One-time assessment**: Ethics is not a launch checkbox. Continuous monitoring required as systems evolve, data drifts, harms emerge over time.
+
+**Technochauvinism**: Believing technical fixes alone solve social problems. Bias mitigation algorithms can help but can't replace addressing root causes (historical discrimination, structural inequality).
+
+**Moving fast and apologizing later**: For safety/ethics, prevention > apology. Harms to vulnerable groups are not acceptable experiments. Staged rollouts, kill switches, continuous monitoring required.