Initial commit

skills/e2e-testing-strategies/SKILL.md

@@ -0,0 +1,290 @@
+---
+name: e2e-testing-strategies
+description: Use when designing E2E test architecture, choosing between Cypress/Playwright/Selenium, prioritizing which flows to test, fixing flaky E2E tests, or debugging slow E2E test suites - provides production-tested patterns and anti-patterns
+---
+
+# E2E Testing Strategies
+
+## Overview
+
+**Core principle:** E2E tests are expensive. Use them sparingly for critical multi-system flows. Everything else belongs lower in the test pyramid.
+
+**Test pyramid target:** 5-10% E2E, 20-25% integration, 65-75% unit
+
+**Scope:** This skill focuses on web application E2E testing (browser-based). For mobile app testing (iOS/Android), decision tree points to Appium, but patterns/anti-patterns here are web-specific. Mobile testing requires different strategies for device capabilities, native selectors, and app lifecycle.
+
+## Framework Selection Decision Tree
+
+Choose framework based on constraints:
+
+| Your Constraint | Choose | Why |
+|----------------|--------|-----|
+| Need cross-browser (Chrome/Firefox/Safari) | **Playwright** | Native multi-browser, auto-wait, trace viewer |
+| Team unfamiliar with testing | **Cypress** | Simpler API, better DX, larger community |
+| Enterprise/W3C standard requirement | **WebdriverIO** | Full W3C WebDriver protocol |
+| Headless Chrome only, fine-grained control | **Puppeteer** | Lower-level, faster for Chrome-only |
+| Testing Electron apps | **Spectron** or **Playwright** | Native Electron support |
+| Mobile apps (iOS/Android) | **Appium** | Mobile-specific protocol (Note: rest of this skill is web-focused) |
+
+**For most web apps:** Playwright (modern, reliable) or Cypress (simpler DX)
+
+## Flow Prioritization Matrix
+
+When you have 50 flows but can only test 10 E2E:
+
+| Score | Criteria | Weight |
+|-------|----------|--------|
+| +3 | Revenue impact (checkout, payment, subscription) | High |
+| +3 | Multi-system integration (API + DB + email + payment) | High |
+| +2 | Historical production failures (has broken before) | Medium |
+| +2 | Complex state management (auth, sessions, caching) | Medium |
+| +1 | User entry point (login, signup, search) | Medium |
+| +1 | Regulatory/compliance requirement | Medium |
+| -2 | Can be tested at integration level | Penalty |
+| -3 | Mostly UI interaction, no backend | Penalty |
+
+**Score flows 0-10, test top 10.** Everything else → integration/unit tests.
+
+**Example:**
+- "User checkout flow" = +3 revenue +3 multi-system +2 historical +2 state = **10** → E2E
+- "User changes email preference" = +1 entry -2 integration level = **-1** → Integration test
+
+## Anti-Patterns Catalog
+
+### ❌ Pyramid Inversion
+**Symptom:** 200 E2E tests, 50 integration tests, 100 unit tests
+
+**Why bad:** E2E tests are slow (30min CI), brittle (UI changes break tests), hard to debug
+
+**Fix:** Invert back - move 150 E2E tests down to integration/unit
+
+---
+
+### ❌ Testing Through the UI
+**Symptom:** E2E test creates 10 users through signup form to test one admin feature
+
+**Why bad:** Slow, couples unrelated features
+
+**Fix:** Seed data via API/database, test only the admin feature flow
+
+---
+
+### ❌ Arbitrary Timeouts
+**Symptom:** `wait(5000)` sprinkled throughout tests
+
+**Why bad:** Flaky - sometimes too short, sometimes wastes time
+
+**Fix:** Explicit waits for conditions
+```javascript
+// ❌ Bad
+await page.click('button');
+await page.waitForTimeout(5000);
+
+// ✅ Good
+await page.click('button');
+await page.waitForSelector('.success-message');
+```
+
+---
+
+### ❌ God Page Objects
+**Symptom:** Single `PageObject` class with 50 methods for entire app
+
+**Why bad:** Tight coupling, hard to maintain, unclear responsibilities
+
+**Fix:** One page object per logical page/component
+```javascript
+// ❌ Bad: God object
+class AppPage {
+  async login() {}
+  async createPost() {}
+  async deleteUser() {}
+  async exportReport() {}
+  // ... 50 more methods
+}
+
+// ✅ Good: Focused page objects
+class AuthPage {
+  async login() {}
+  async logout() {}
+}
+
+class PostsPage {
+  async create() {}
+  async delete() {}
+}
+```
+
+---
+
+###❌ Brittle Selectors
+**Symptom:** `page.click('.btn-primary.mt-4.px-3')`
+
+**Why bad:** Breaks when CSS changes
+
+**Fix:** Use `data-testid` attributes
+```javascript
+// ❌ Bad
+await page.click('.submit-button.btn.btn-primary');
+
+// ✅ Good
+await page.click('[data-testid="submit"]');
+```
+
+---
+
+### ❌ Test Interdependence
+**Symptom:** Test 5 fails if Test 3 doesn't run first
+
+**Why bad:** Can't run tests in parallel, hard to debug
+
+**Fix:** Each test sets up own state
+```javascript
+// ❌ Bad
+test('create user', async () => {
+  // creates user "test@example.com"
+});
+
+test('login user', async () => {
+  // assumes user from previous test exists
+});
+
+// ✅ Good
+test('login user', async ({ page }) => {
+  await createUserViaAPI('test@example.com'); // independent setup
+  await page.goto('/login');
+  // test login flow
+});
+```
+
+## Flakiness Patterns Catalog
+
+Common flake sources and fixes:
+
+| Pattern | Symptom | Fix |
+|---------|---------|-----|
+| **Network Race** | "Element not found" intermittently | `await page.waitForLoadState('networkidle')` |
+| **Animation Race** | "Element not clickable" | `await page.waitForSelector('.element', { state: 'visible' })` or disable animations |
+| **Async State** | "Expected 'success' but got ''" | Wait for specific state, not timeout |
+| **Test Data Pollution** | Test passes alone, fails in suite | Isolate data per test (unique IDs, cleanup) |
+| **Browser Caching** | Different results first vs second run | Clear cache/cookies between tests |
+| **Date/Time Sensitivity** | Test fails at midnight, passes during day | Mock system time in tests |
+| **External Service** | Third-party API occasionally down | Mock external dependencies |
+
+**Rule:** If test fails <5% of time, it's flaky. Fix it before adding more tests.
+
+## Page Object Anti-Patterns
+
+### ❌ Business Logic in Page Objects
+```javascript
+// ❌ Bad
+class CheckoutPage {
+  async calculateTotal(items) {
+    return items.reduce((sum, item) => sum + item.price, 0); // business logic
+  }
+}
+
+// ✅ Good
+class CheckoutPage {
+  async getTotal() {
+    return await page.textContent('[data-testid="total"]'); // UI interaction only
+  }
+}
+```
+
+### ❌ Assertions in Page Objects
+```javascript
+// ❌ Bad
+class LoginPage {
+  async login(email, password) {
+    await this.page.fill('[data-testid="email"]', email);
+    await this.page.fill('[data-testid="password"]', password);
+    await this.page.click('[data-testid="submit"]');
+    expect(this.page.url()).toContain('/dashboard'); // assertion
+  }
+}
+
+// ✅ Good
+class LoginPage {
+  async login(email, password) {
+    await this.page.fill('[data-testid="email"]', email);
+    await this.page.fill('[data-testid="password"]', password);
+    await this.page.click('[data-testid="submit"]');
+  }
+
+  async isOnDashboard() {
+    return this.page.url().includes('/dashboard');
+  }
+}
+
+// Test file handles assertions
+test('login', async () => {
+  await loginPage.login('user@test.com', 'password');
+  expect(await loginPage.isOnDashboard()).toBe(true);
+});
+```
+
+## Quick Reference
+
+### When to Use E2E vs Integration vs Unit
+
+| Scenario | Test Level | Reasoning |
+|----------|-----------|-----------|
+| Form validation logic | Unit | Pure function, no UI needed |
+| API error handling | Integration | Test API contract, no browser |
+| Multi-step checkout | E2E | Crosses systems, critical revenue |
+| Button hover states | Visual regression | Not functional behavior |
+| Login → dashboard redirect | E2E | Auth critical, multi-system |
+| Database query performance | Integration | No UI, just DB |
+| User can filter search results | E2E (1 test) + Integration (variations) | 1 E2E for happy path, rest integration |
+
+### Test Data Strategies
+
+| Approach | When to Use | Pros | Cons |
+|----------|-------------|------|------|
+| **API Seeding** | Most tests | Fast, consistent | Requires API access |
+| **Database Seeding** | Integration tests | Complete control | Slow, requires DB access |
+| **UI Creation** | Testing creation flow itself | Tests real user path | Slow, couples tests |
+| **Mocking** | External services | Fast, reliable | Misses real integration issues |
+| **Fixtures** | Consistent test data | Reusable, version-controlled | Stale if schema changes |
+
+## Common Mistakes
+
+### ❌ Running Full Suite on Every Commit
+**Symptom:** 30-minute CI blocking every PR
+
+**Fix:** Smoke tests (5-10 critical flows) on PR, full suite on merge/nightly
+
+---
+
+### ❌ Not Capturing Failure Artifacts
+**Symptom:** "Test failed in CI but I can't reproduce"
+
+**Fix:** Save video + trace on failure
+```javascript
+// playwright.config.js
+use: {
+  video: 'retain-on-failure',
+  trace: 'retain-on-failure',
+}
+```
+
+---
+
+### ❌ Testing Implementation Details
+**Symptom:** Tests assert internal component state
+
+**Fix:** Test user-visible behavior only
+
+---
+
+### ❌ One Assert Per Test
+**Symptom:** 50 E2E tests all navigate to same page, test one thing
+
+**Fix:** Group related assertions in one flow test (but keep focused)
+
+## Bottom Line
+
+**E2E tests verify critical multi-system flows work for real users.**
+
+If you can test it faster/more reliably at a lower level, do that instead.