gh-bandofai-puerto-plugins-design/skills/ux-research/SKILL.md

# UX Research

**User research methodologies and insights synthesis**

# Research Methods Skill

**Comprehensive guide to UX research methodologies and when to use each method**

This skill codifies best practices from hundreds of successful UX research projects across discovery, validation, and optimization phases.

---

## Core Principles

1. **Match Method to Question**: Choose research methods based on what you need to learn, not what's trendy
2. **Triangulate**: Use multiple methods to validate findings and reduce bias
3. **Include Users Early and Often**: Research throughout the product lifecycle, not just at the beginning
4. **Balance Qual and Quant**: Qualitative explores "why," quantitative measures "how much"
5. **Ethical Research Always**: Respect participant time, privacy, and dignity

---

## Research Method Selection Framework

### The 2x2 Framework: Attitudinal vs. Behavioral, Qualitative vs. Quantitative

```
                    QUALITATIVE                 QUANTITATIVE
                    (Why/How)                   (How many/much)
              ┌─────────────────────┬──────────────────────────┐
              │                     │                          │
ATTITUDINAL   │  • Interviews       │  • Surveys               │
(What people  │  • Focus Groups     │  • Card Sorting (quant)  │
say)          │  • Diary Studies    │  • A/B Tests (attitudes) │
              │                     │  • Analytics (stated)    │
              ├─────────────────────┼──────────────────────────┤
              │                     │                          │
BEHAVIORAL    │  • Usability Tests  │  • Analytics             │
(What people  │  • Field Studies    │  • A/B Tests             │
do)           │  • Contextual Inq.  │  • Heatmaps/Click tracks │
              │  • Task Analysis    │  • Performance Metrics   │
              │                     │                          │
              └─────────────────────┴──────────────────────────┘
```

### When to Use Each Quadrant

**Attitudinal + Qualitative**: Understanding motivations, perceptions, mental models
- "Why do users prefer X over Y?"
- "What do users think about this concept?"

**Attitudinal + Quantitative**: Measuring opinions at scale
- "How satisfied are users with feature X?"
- "How many users would pay for premium tier?"

**Behavioral + Qualitative**: Understanding how users interact and why
- "How do users currently accomplish task X?"
- "What workarounds do users create?"

**Behavioral + Quantitative**: Measuring what users do at scale
- "What percentage of users complete onboarding?"
- "Which features are used most frequently?"

---

## Research Methods by Purpose

### 1. GENERATIVE RESEARCH (Discovery & Exploration)

**Goal**: Understand user needs, contexts, and problems before defining solutions

#### User Interviews
**Best for**: Deep understanding of motivations, attitudes, experiences

**When to use**:
- Beginning of project to understand problem space
- Exploring new market or user segment
- Understanding "why" behind behaviors

**Pros**:
- Rich, deep insights
- Flexibility to explore unexpected topics
- Can uncover unarticulated needs

**Cons**:
- Time-consuming (typically 45-90 min per interview)
- Small sample size (5-12 participants)
- Susceptible to social desirability bias

**Sample size**: 5-12 participants per user segment

**Typical questions**:
- "Tell me about the last time you..."
- "Walk me through how you..."
- "What's most important to you when..."

---

#### Contextual Inquiry / Field Studies
**Best for**: Observing users in their natural environment

**When to use**:
- Understanding workflow in complex environments
- Observing actual behavior (not self-reported)
- Discovering workarounds and adaptations

**Pros**:
- Observe real behavior in context
- See environmental factors affecting use
- Discover unarticulated needs

**Cons**:
- Very time-intensive
- Logistically complex (travel, access)
- Can be intrusive to participants

**Sample size**: 4-8 site visits

**Process**:
1. Shadow user in their environment
2. Observe tasks and workflows
3. Ask questions about what you observe
4. Document context (photos, notes)

---

#### Diary Studies
**Best for**: Understanding behaviors and experiences over time

**When to use**:
- Behaviors that occur sporadically
- Understanding context of use over time
- Capturing in-the-moment experiences

**Pros**:
- Captures real-time experiences (less recall bias)
- Shows patterns over time
- Less researcher time than shadowing

**Cons**:
- Participant burden (compliance issues)
- Self-reported data
- Can take weeks to complete

**Sample size**: 10-20 participants over 1-4 weeks

**Tools**: Mobile apps (dscout, Indeemo), photo journals, SMS surveys

---

#### Surveys (Exploratory)
**Best for**: Identifying patterns and priorities across large populations

**When to use**:
- Need to understand frequency of behaviors
- Want to segment users
- Validate interview findings at scale

**Pros**:
- Large sample sizes
- Statistically significant results
- Cost-effective per response

**Cons**:
- Can't explore "why" deeply
- Response bias
- No opportunity to clarify answers

**Sample size**: Minimum 100 for basic analysis, 300+ for segmentation

**Question types**:
- Multiple choice for quantification
- Likert scales for attitudes
- Open-ended for unexpected insights (limited)

---

### 2. EVALUATIVE RESEARCH (Testing & Validation)

**Goal**: Test designs, concepts, or products to identify issues and validate solutions

#### Usability Testing (Moderated)
**Best for**: Identifying usability issues and understanding user behavior with product

**When to use**:
- Testing prototypes or live products
- Evaluating specific flows or features
- Before major product launches

**Pros**:
- Identifies specific usability issues
- Can probe to understand "why"
- Flexible to explore unexpected findings

**Cons**:
- Artificial environment (not real context)
- Moderator bias possible
- Time-consuming

**Sample size**: 5 users per user segment (finds ~85% of issues per Nielsen)

**Process**:
1. Give user realistic task scenarios
2. Ask them to think aloud
3. Observe struggles and successes
4. Measure time, errors, completion

**Metrics**:
- Task completion rate
- Time on task
- Error rate
- Satisfaction ratings (post-task)

---

#### Usability Testing (Unmoderated Remote)
**Best for**: Quick feedback on specific tasks at scale

**When to use**:
- Need results quickly
- Testing simple, linear flows
- Want larger sample size

**Pros**:
- Fast turnaround (hours, not weeks)
- Larger sample sizes
- Lower cost per participant

**Cons**:
- Can't ask follow-up questions
- Limited to simple tasks
- No way to clarify misunderstandings

**Sample size**: 15-30 participants

**Tools**: UserTesting, Maze, UserZoom

---

#### A/B Testing
**Best for**: Comparing two design alternatives with real users

**When to use**:
- Have two viable design options
- Need to measure impact on key metrics
- Have sufficient traffic for significance

**Pros**:
- Measures real behavior (not stated preferences)
- Statistically significant results
- Removes opinion-based decisions

**Cons**:
- Requires traffic volume
- Can only test one variable at a time
- Doesn't explain "why" variant won

**Sample size**: Depends on baseline conversion and desired lift (use calculator)

**Metrics**: Conversion rate, click-through rate, time on page, revenue

---

#### Concept Testing
**Best for**: Validating ideas before building them

**When to use**:
- Early stage, before detailed design
- Choosing between multiple concepts
- Validating value proposition

**Pros**:
- Fail fast and cheap
- Can test multiple concepts
- Provides direction for design

**Cons**:
- Reactions to concept ≠ actual usage
- Risk of overvaluing stated intent
- Hard to imagine new paradigms

**Sample size**: 8-15 for qualitative, 100+ for quantitative

**Methods**: Show concept (sketches, storyboards, mockups), gather reactions

---

#### Card Sorting
**Best for**: Testing or creating information architecture

**When to use**:
- Designing navigation structure
- Organizing content or features
- Understanding mental models of categorization

**Types**:
- **Open**: Users create and name categories
- **Closed**: Users sort into predefined categories
- **Hybrid**: Mix of both

**Pros**:
- User-centered IA
- Quantifiable results
- Easy to conduct remotely

**Cons**:
- Decontextualized from real use
- Only tests grouping, not findability
- Assumes users know all items

**Sample size**: 15-30 for open, 30+ for closed

**Tools**: Optimal Workshop, Maze, UserZoom

---

#### Tree Testing
**Best for**: Validating navigation structure

**When to use**:
- After creating IA (validates card sorting)
- Testing if users can find content
- Before visual design

**Pros**:
- Tests findability directly
- Quantifiable results
- Fast and cost-effective

**Cons**:
- No visual design context
- Text-only (misses visual cues)
- Limited to tree structures

**Sample size**: 50-100 participants

**Metrics**: Success rate, directness, time

---

### 3. ANALYTICS & BEHAVIORAL DATA

#### Web/Product Analytics
**Best for**: Understanding actual user behavior at scale

**When to use**:
- Identifying where users drop off
- Measuring feature adoption
- Tracking key metrics over time
- Generating hypotheses for research

**Pros**:
- Real behavior, not self-reported
- Large sample sizes
- Continuous monitoring

**Cons**:
- Doesn't explain "why"
- Privacy concerns
- Can be misinterpreted without context

**Key Metrics**:
- Acquisition: Where users come from
- Activation: Do they complete onboarding?
- Engagement: How often do they return?
- Retention: Do they stick around?
- Revenue: Do they pay?

**Tools**: Google Analytics, Mixpanel, Amplitude, Heap

---

#### Heatmaps & Session Recordings
**Best for**: Visualizing where users click, scroll, and focus attention

**When to use**:
- Optimizing page layouts
- Understanding confusion points
- Identifying ignored content

**Pros**:
- Visual and intuitive
- Shows aggregate patterns
- Can identify rage clicks (frustration)

**Cons**:
- Descriptive, not explanatory
- Can be over-interpreted
- Privacy concerns

**Tools**: Hotjar, Crazy Egg, FullStory

---

### 4. SPECIALIZED METHODS

#### Accessibility Testing
**Best for**: Ensuring product works for users with disabilities

**When to use**:
- Required for compliance (WCAG, ADA)
- Serving diverse user base
- Ethical product development

**Methods**:
- Automated testing (axe, WAVE, Lighthouse)
- Manual testing with assistive tech
- User testing with people with disabilities

**Sample size**: 3-5 users per disability type

---

#### Competitive Analysis
**Best for**: Understanding market landscape and best practices

**When to use**:
- Entering new market
- Benchmarking against competitors
- Identifying feature gaps

**Process**:
1. Identify competitors
2. Analyze key features, flows, pricing
3. Conduct comparative usability testing
4. Document strengths and weaknesses

---

#### Heuristic Evaluation
**Best for**: Expert review against usability principles

**When to use**:
- Quick assessment without users
- Budget/time constraints
- Before user testing to fix obvious issues

**Heuristics** (Nielsen's 10):
1. Visibility of system status
2. Match between system and real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency of use
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors
10. Help and documentation

**Evaluators**: 3-5 UX experts

---

## Research Method Selection Guide

### By Project Stage

**Discovery / Problem Definition**:
- User interviews
- Field studies / Contextual inquiry
- Diary studies
- Surveys (exploratory)
- Competitive analysis

**Ideation / Concept Validation**:
- Concept testing
- Participatory design workshops
- Card sorting (for IA)
- Surveys (concept preference)

**Design / Prototype Testing**:
- Usability testing (moderated)
- Tree testing
- First-click testing
- Heuristic evaluation

**Pre-Launch Validation**:
- Usability testing (unmoderated)
- Accessibility testing
- Beta testing
- A/B testing (if redesign)

**Post-Launch Optimization**:
- Analytics
- A/B testing
- Surveys (satisfaction)
- Session recordings
- Ongoing usability testing

---

### By Research Question Type

**"What do users need?"**
→ Interviews, field studies, diary studies

**"Will users understand this concept?"**
→ Concept testing, first-click tests

**"Can users complete this task?"**
→ Usability testing, task analysis

**"How many users experience this?"**
→ Surveys, analytics

**"Which design performs better?"**
→ A/B testing, preference testing

**"How satisfied are users?"**
→ Surveys (NPS, CSAT, SUS), interviews

**"How do we organize content?"**
→ Card sorting, tree testing

**"Is this accessible?"**
→ Accessibility testing, assistive tech testing

---

## Sample Size Guidelines

### Qualitative Research

**Interviews / Usability Tests**: 5-8 per user segment
- Diminishing returns after 5 (per Nielsen)
- If multiple distinct segments, 5 per segment
- For very simple products: 3-5 total
- For complex products: 8-12 total

**Field Studies**: 4-8 site visits
- Very time-intensive
- Smaller sample acceptable

**Diary Studies**: 10-20 participants
- Higher dropout rate
- Need larger initial sample

---

### Quantitative Research

**Surveys**:
- Exploratory: 100+ for basic insights
- Segmentation analysis: 300+
- Statistical modeling: 1000+

**A/B Tests**:
- Depends on baseline conversion rate and desired lift
- Typically need thousands of users per variant
- Use online calculator for specific estimates

**Card Sorting**:
- Open: 15-30
- Closed: 30-50

**Tree Testing**: 50-100

---

## Mixed Methods Approach

**Best practice**: Combine qualitative and quantitative for robust insights

### Example: Redesigning Checkout Flow

1. **Discover** (Qual): Interview 8 users about current checkout experience
   - Identify pain points and needs

2. **Validate** (Quant): Survey 300 users on identified pain points
   - Quantify how many experience each issue

3. **Test** (Qual): Usability test new design with 6 users
   - Identify usability issues

4. **Measure** (Quant): A/B test new design with live traffic
   - Measure impact on conversion

5. **Understand** (Qual): Interview users who abandoned new checkout
   - Understand why some still fail

This creates a complete picture: qualitative explains, quantitative measures.

---

## Common Research Mistakes to Avoid

### 1. Confirmation Bias
❌ **Mistake**: Only asking questions that support your hypothesis
✅ **Fix**: Include questions that could disprove your assumptions

### 2. Leading Questions
❌ **Mistake**: "Don't you think this design is confusing?"
✅ **Fix**: "What are your thoughts on this design?"

### 3. Hypothetical Questions
❌ **Mistake**: "Would you use this feature?"
✅ **Fix**: "Tell me about the last time you needed to [accomplish task]"

### 4. Treating Insights as Data
❌ **Mistake**: "5 users said X, so everyone must think X"
✅ **Fix**: "5/5 users experienced X (small sample, directional insight)"

### 5. Testing Too Late
❌ **Mistake**: Only testing after full build
✅ **Fix**: Test early with low-fidelity prototypes

### 6. Ignoring Negative Findings
❌ **Mistake**: Cherry-picking data that supports your design
✅ **Fix**: Report all findings, especially contradictory ones

### 7. Over-Researching
❌ **Mistake**: Endless research without action
✅ **Fix**: Define decision criteria upfront, research only to inform decision

### 8. Research Without Clear Objectives
❌ **Mistake**: "Let's do some user interviews and see what we find"
✅ **Fix**: "We need to understand [specific question] to decide [specific decision]"

---

## Ethical Research Guidelines

### Informed Consent
- [ ] Explain purpose of research clearly
- [ ] Explain how data will be used
- [ ] Explain recording and privacy practices
- [ ] Allow participant to opt out at any time
- [ ] Obtain explicit consent before proceeding

### Privacy and Confidentiality
- [ ] Anonymize participant data
- [ ] Secure storage of recordings and notes
- [ ] Delete personal information after analysis
- [ ] Don't share recordings outside research team
- [ ] Follow GDPR/privacy regulations

### Respectful Treatment
- [ ] Compensate participants fairly for time
- [ ] Don't make participants feel judged
- [ ] Accommodate accessibility needs
- [ ] Allow breaks and adjust pace
- [ ] Thank participants sincerely

### Vulnerable Populations
- [ ] Extra care with children, elderly, disabled
- [ ] May require guardian consent
- [ ] Trauma-informed approach for sensitive topics
- [ ] Have resources ready if distress occurs

---

## Research Planning Checklist

**Before Research**:
- [ ] Clear research questions defined
- [ ] Method(s) selected and justified
- [ ] Participant criteria specified
- [ ] Recruitment plan in place
- [ ] Materials prepared (guides, prototypes)
- [ ] Consent forms ready
- [ ] Recording setup tested
- [ ] Pilot test conducted

**During Research**:
- [ ] Follow protocol consistently
- [ ] Take detailed notes
- [ ] Record sessions (with consent)
- [ ] Note unexpected findings
- [ ] Debrief after each session
- [ ] Adjust if critical issues found

**After Research**:
- [ ] Analyze data systematically
- [ ] Identify patterns and themes
- [ ] Create actionable insights
- [ ] Prioritize recommendations
- [ ] Share findings with stakeholders
- [ ] Track implementation of recommendations

---

## Quick Reference: Method Selection Matrix

| If you need to... | Use this method |
|-------------------|----------------|
| Understand user needs before designing | Interviews, Field Studies |
| Test if users can complete a task | Usability Testing |
| Choose between two designs | A/B Testing, Preference Testing |
| Organize content/features | Card Sorting |
| Validate navigation structure | Tree Testing |
| Measure satisfaction | Surveys (SUS, NPS, CSAT) |
| Understand "why" behind analytics | Interviews, Session Recordings |
| Validate a new concept | Concept Testing |
| Track behavior over time | Diary Studies, Analytics |
| Quick feedback on prototype | Unmoderated Usability Testing |
| Understand expert perspective | Heuristic Evaluation |
| Ensure accessibility | Accessibility Testing |

---

**Version**: 1.0
**Last Updated**: January 2025
**Success Rate**: Based on industry best practices and research literature