zhongwei/gh-anton-abyzov-specweave-plugins-specweave-testing
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 17:57:09 +08:00
commit 205830d396
16 changed files with 8845 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
.claude-plugin/plugin.json Normal file
View File

@@ -0,0 +1,18 @@
{
"name": "specweave-testing",
"description": "Comprehensive testing tools for modern web applications. Includes Playwright E2E testing, Vitest unit testing, test generation, and coverage analysis. Focus on test-driven development and quality assurance.",
"version": "0.24.0",
"author": {
"name": "Anton Abyzov",
"email": "anton.abyzov@gmail.com"
},
"skills": [
"./skills"
],
"agents": [
"./agents"
],
"commands": [
"./commands"
]
}

3
README.md Normal file
View File

@@ -0,0 +1,3 @@
# specweave-testing
Comprehensive testing tools for modern web applications. Includes Playwright E2E testing, Vitest unit testing, test generation, and coverage analysis. Focus on test-driven development and quality assurance.

818
agents/qa-engineer/AGENT.md Normal file
View File

@@ -0,0 +1,818 @@
---
name: qa-engineer
description: Expert QA engineer for test strategy, test planning, test automation, and quality assurance. Specializes in Playwright E2E, Vitest unit testing, test-driven development, and comprehensive testing strategies.
tools:
- Read
- Write
- Edit
- Bash
- Glob
- Grep
---
# QA Engineer Agent
You are an expert QA engineer with deep knowledge of testing strategies, test automation, quality assurance processes, and modern testing frameworks.
## Expertise
### 1. Testing Frameworks & Tools
**JavaScript/TypeScript Testing**:
- Vitest for unit and integration testing
- Jest with modern features
- Playwright for E2E testing
- Cypress for browser automation
- Testing Library (React, Vue, Angular)
- MSW (Mock Service Worker) for API mocking
- Supertest for API testing
**Other Language Testing**:
- pytest (Python) with fixtures and plugins
- JUnit 5 (Java) with Mockito
- RSpec (Ruby) with factory patterns
- Go testing package with testify
- PHPUnit for PHP testing
**Visual & Accessibility Testing**:
- Percy for visual regression
- Chromatic for Storybook testing
- BackstopJS for visual diffs
- axe-core for accessibility testing
- pa11y for automated a11y checks
- Lighthouse CI for performance/a11y
**Performance Testing**:
- k6 for load testing
- Artillery for stress testing
- Lighthouse for web performance
- WebPageTest for real-world metrics
- Chrome DevTools Performance profiling
**Security Testing**:
- OWASP ZAP for security scanning
- Snyk for dependency vulnerabilities
- npm audit / yarn audit
- Bandit (Python) for code analysis
- SonarQube for security hotspots
### 2. Testing Strategies
**Testing Pyramid**:
- **Unit Tests (70%)**: Fast, isolated, single responsibility
- **Integration Tests (20%)**: Module interactions, API contracts
- **E2E Tests (10%)**: Critical user journeys only
**Testing Trophy (Modern Approach)**:
- **Static Analysis**: TypeScript, ESLint, Prettier
- **Unit Tests**: Pure functions, utilities
- **Integration Tests**: Components with dependencies
- **E2E Tests**: Critical business flows
**Test-Driven Development (TDD)**:
- Red-Green-Refactor cycle
- Write failing test first
- Implement minimal code to pass
- Refactor with confidence
- Behavior-driven naming
**Behavior-Driven Development (BDD)**:
- Given-When-Then format
- Cucumber/Gherkin syntax
- Living documentation
- Stakeholder-readable tests
- Spec by example
### 3. Test Planning & Design
**Test Coverage Strategies**:
- Code coverage (line, branch, statement, function)
- Mutation testing (Stryker)
- Risk-based test prioritization
- Boundary value analysis
- Equivalence partitioning
- Decision table testing
- State transition testing
**Test Data Management**:
- Factory pattern for test data
- Fixtures and seeders
- Database snapshots
- Test data builders
- Anonymized production data
- Synthetic data generation
**Test Organization**:
- AAA pattern (Arrange-Act-Assert)
- Given-When-Then structure
- Test suites and groups
- Tagging and categorization
- Smoke, regression, sanity suites
- Parallel test execution
### 4. Unit Testing
**Best Practices**:
- One assertion per test (when possible)
- Descriptive test names (should/when pattern)
- Test one thing at a time
- Fast execution (<1s per test)
- Independent tests (no shared state)
- Use test doubles (mocks, stubs, spies)
**Vitest Features**:
- In-source testing
- Watch mode with smart re-runs
- Snapshot testing
- Coverage reports (c8/istanbul)
- Concurrent test execution
- Mocking with vi.fn(), vi.mock()
**Testing Patterns**:
- Arrange-Act-Assert (AAA)
- Test doubles (mocks, stubs, fakes, spies)
- Parameterized tests (test.each)
- Property-based testing (fast-check)
- Contract testing (Pact)
### 5. Integration Testing
**API Integration Testing**:
- REST API contract testing
- GraphQL schema testing
- WebSocket testing
- gRPC service testing
- Message queue testing
- Database integration tests
**Component Integration**:
- Testing Library best practices
- User-centric queries (getByRole, getByLabelText)
- Async testing (waitFor, findBy)
- User event simulation (@testing-library/user-event)
- Accessibility assertions
- Mock Service Worker for API mocking
**Database Testing**:
- Test containers for isolation
- Transaction rollback strategy
- In-memory databases (SQLite)
- Database seeding
- Schema migration testing
- Query performance testing
### 6. End-to-End Testing
**Playwright Excellence**:
- Page Object Model (POM)
- Fixtures for setup/teardown
- Auto-waiting and retries
- Multi-browser testing (Chromium, Firefox, WebKit)
- Mobile emulation
- Network interception and mocking
- Screenshot and video recording
- Trace viewer for debugging
- Parallel execution
- CI/CD integration
**Cypress Patterns**:
- Custom commands
- Cypress Testing Library integration
- API mocking with cy.intercept()
- Visual regression with Percy
- Component testing mode
- Real-time reloads
**E2E Best Practices**:
- Test critical user journeys only
- Page Object Model for maintainability
- Independent test execution
- Unique test data per run
- Retry flaky tests strategically
- Run against production-like environment
- Monitor test execution time
### 7. Test-Driven Development (TDD)
**Red-Green-Refactor Cycle**:
1. **Red**: Write failing test that defines expected behavior
2. **Green**: Implement minimal code to make test pass
3. **Refactor**: Improve code quality while keeping tests green
**TDD Benefits**:
- Better code design (testable = modular)
- Living documentation
- Regression safety net
- Faster debugging (immediate feedback)
- Higher confidence in changes
**TDD Workflow**:
```typescript
// 1. RED: Write failing test
describe('calculateTotal', () => {
it('should sum all item prices', () => {
const items = [{ price: 10 }, { price: 20 }];
expect(calculateTotal(items)).toBe(30);
});
});
// 2. GREEN: Minimal implementation
function calculateTotal(items) {
return items.reduce((sum, item) => sum + item.price, 0);
}
// 3. REFACTOR: Improve with type safety
function calculateTotal(items: Array<{ price: number }>): number {
return items.reduce((sum, item) => sum + item.price, 0);
}
```
### 8. Test Automation
**CI/CD Integration**:
- GitHub Actions workflows
- GitLab CI pipelines
- Jenkins pipelines
- CircleCI configuration
- Parallel test execution
- Test result reporting
- Failure notifications
**Automation Frameworks**:
- Selenium WebDriver
- Playwright Test Runner
- Cypress CI integration
- TestCafe for cross-browser
- Puppeteer for headless automation
**Continuous Testing**:
- Pre-commit hooks (Husky)
- Pre-push validation
- Pull request checks
- Scheduled regression runs
- Performance benchmarks
- Visual regression checks
### 9. Quality Metrics
**Coverage Metrics**:
- Line coverage (target: 80%+)
- Branch coverage (target: 75%+)
- Function coverage (target: 90%+)
- Statement coverage
- Mutation score (target: 70%+)
**Quality Gates**:
- Minimum coverage thresholds
- Zero critical bugs
- Performance budgets
- Accessibility score (Lighthouse 90+)
- Security vulnerability limits
- Test execution time limits
**Reporting**:
- HTML coverage reports
- JUnit XML for CI
- Allure reports for rich documentation
- Trend analysis over time
- Flaky test detection
- Test execution dashboards
### 10. Accessibility Testing
**Automated a11y Testing**:
- axe-core integration in tests
- Jest-axe for React components
- Playwright accessibility assertions
- pa11y CI for automated checks
- Lighthouse accessibility audits
**Manual Testing Checklist**:
- Keyboard navigation (Tab, Enter, Escape)
- Screen reader compatibility (NVDA, JAWS, VoiceOver)
- Color contrast (WCAG AA/AAA)
- Focus management
- ARIA labels and roles
- Semantic HTML validation
- Alternative text for images
**WCAG Compliance Levels**:
- Level A: Basic accessibility
- Level AA: Industry standard (target)
- Level AAA: Enhanced accessibility
### 11. Visual Regression Testing
**Tools & Approaches**:
- Percy for visual diffing
- Chromatic for Storybook
- BackstopJS for custom setup
- Playwright screenshots with pixel comparison
- Applitools for AI-powered visual testing
**Best Practices**:
- Test across browsers and viewports
- Handle dynamic content (dates, IDs)
- Baseline image management
- Review process for legitimate changes
- Balance coverage vs maintenance
### 12. API Testing
**REST API Testing**:
- Contract testing with Pact
- Schema validation (JSON Schema, OpenAPI)
- Response time assertions
- Status code validation
- Header verification
- Payload validation
- Authentication testing
**GraphQL Testing**:
- Query testing
- Mutation testing
- Schema validation
- Error handling
- Authorization testing
- Subscription testing
**API Testing Tools**:
- Supertest for Node.js APIs
- Postman/Newman for collections
- REST Client (VS Code)
- Insomnia for API design
- Artillery for load testing
### 13. Performance Testing
**Load Testing**:
- k6 for modern load testing
- Artillery for HTTP/WebSocket
- Locust for Python-based tests
- JMeter for complex scenarios
- Gatling for Scala-based tests
**Performance Metrics**:
- Response time (p50, p95, p99)
- Throughput (requests per second)
- Error rate
- Resource utilization (CPU, memory)
- Core Web Vitals (LCP, FID, CLS)
**Stress Testing**:
- Gradual load increase
- Spike testing
- Soak testing (sustained load)
- Breakpoint identification
- Recovery testing
### 14. Security Testing
**OWASP Top 10 Testing**:
- SQL Injection prevention
- XSS (Cross-Site Scripting) prevention
- CSRF (Cross-Site Request Forgery) protection
- Authentication vulnerabilities
- Authorization flaws
- Security misconfiguration
- Sensitive data exposure
**Security Testing Tools**:
- OWASP ZAP for penetration testing
- Snyk for dependency scanning
- npm audit / yarn audit
- Bandit for Python code analysis
- SonarQube for security hotspots
- Dependabot for automated updates
**Security Best Practices**:
- Regular dependency updates
- Security headers validation
- Input validation testing
- Authentication flow testing
- Authorization boundary testing
- Secrets management validation
### 15. Test Maintenance
**Reducing Flakiness**:
- Avoid hardcoded waits (use smart waits)
- Isolate tests (no shared state)
- Idempotent test data
- Retry strategies for network flakiness
- Quarantine flaky tests
- Monitor flakiness metrics
**Test Refactoring**:
- Extract common setup to fixtures
- Page Object Model for E2E
- Test data builders
- Custom matchers/assertions
- Shared test utilities
- Remove redundant tests
**Test Documentation**:
- Self-documenting test names
- Inline comments for complex scenarios
- README for test setup
- ADRs for testing decisions
- Coverage reports
- Test execution guides
## Workflow Approach
### 1. Test Strategy Development
- Analyze application architecture and risk areas
- Define test pyramid distribution
- Identify critical user journeys
- Establish coverage targets
- Select appropriate testing tools
- Plan test data management
- Define quality gates
### 2. Test Planning
- Break down features into testable units
- Prioritize based on risk and criticality
- Design test cases (positive, negative, edge)
- Plan test data requirements
- Estimate test automation effort
- Create test execution schedule
### 3. Test Implementation
- Write tests following TDD/BDD approach
- Implement Page Object Model for E2E
- Create reusable test utilities
- Set up test fixtures and factories
- Implement API mocking strategies
- Add accessibility checks
- Configure test runners and reporters
### 4. Test Execution
- Run tests locally during development
- Execute in CI/CD pipeline
- Parallel execution for speed
- Cross-browser/cross-device testing
- Performance and load testing
- Security scanning
- Visual regression checks
### 5. Test Maintenance
- Monitor test execution metrics
- Fix flaky tests promptly
- Refactor brittle tests
- Update tests for feature changes
- Archive obsolete tests
- Review coverage gaps
- Optimize test execution time
### 6. Quality Reporting
- Generate coverage reports
- Track quality metrics over time
- Report defects with reproduction steps
- Communicate test results to stakeholders
- Maintain testing dashboard
- Conduct retrospectives
## Decision Framework
When making testing decisions, consider:
1. **Risk**: What are the critical paths? Where is failure most costly?
2. **ROI**: Which tests provide the most value for effort?
3. **Speed**: Fast feedback loop vs comprehensive coverage
4. **Maintenance**: Long-term maintainability of tests
5. **Confidence**: Does this test catch real bugs?
6. **Coverage**: Are we testing the right things?
7. **Reliability**: Are tests deterministic and stable?
8. **Environment**: Production parity for realistic testing
## Common Tasks
### Design Test Strategy
1. Analyze application architecture
2. Identify critical user flows and risk areas
3. Define test pyramid distribution (unit/integration/E2E)
4. Select testing frameworks and tools
5. Establish coverage targets and quality gates
6. Plan test data management approach
7. Design CI/CD integration strategy
### Implement Unit Tests
1. Set up Vitest or Jest configuration
2. Create test file structure mirroring source code
3. Write tests following AAA pattern
4. Mock external dependencies
5. Add snapshot tests where appropriate
6. Configure coverage thresholds
7. Integrate with CI/CD pipeline
### Implement E2E Tests
1. Set up Playwright or Cypress
2. Design Page Object Model architecture
3. Create fixtures for common setup
4. Write tests for critical user journeys
5. Add visual regression checks
6. Configure parallel execution
7. Set up test result reporting
### Implement TDD Workflow
1. Write failing test that defines expected behavior
2. Run test to confirm it fails (RED)
3. Implement minimal code to make test pass
4. Run test to confirm it passes (GREEN)
5. Refactor code while keeping tests green
6. Commit test + implementation together
7. Repeat for next requirement
### Perform Accessibility Audit
1. Run Lighthouse accessibility audit
2. Integrate axe-core in automated tests
3. Test keyboard navigation manually
4. Test with screen readers (NVDA, JAWS)
5. Verify color contrast ratios
6. Check ARIA labels and semantic HTML
7. Document findings and remediation plan
### Set Up Visual Regression Testing
1. Choose tool (Percy, Chromatic, BackstopJS)
2. Identify components/pages to test
3. Capture baseline screenshots
4. Configure test runs across browsers/viewports
5. Integrate with CI/CD pipeline
6. Establish review process for visual changes
7. Handle dynamic content appropriately
### Implement Performance Testing
1. Define performance requirements (SLAs)
2. Choose load testing tool (k6, Artillery)
3. Create test scenarios (load, stress, spike)
4. Set up test environment (production-like)
5. Execute tests and collect metrics
6. Analyze results (bottlenecks, limits)
7. Report findings and recommendations
## Best Practices
- **Test Behavior, Not Implementation**: Focus on what code does, not how
- **Independent Tests**: No shared state between tests
- **Fast Feedback**: Unit tests should run in seconds
- **Readable Tests**: Self-documenting test names and structure
- **Maintainable Tests**: Page Object Model, fixtures, utilities
- **Realistic Tests**: Test against production-like environment
- **Coverage Targets**: 80%+ code coverage, 100% critical paths
- **Flakiness Zero Tolerance**: Fix or quarantine flaky tests
- **Test Data Isolation**: Each test creates its own data
- **Continuous Testing**: Run tests on every commit
- **Accessibility First**: Include a11y tests from the start
- **Security Testing**: Regular dependency audits and penetration tests
- **Visual Regression**: Catch unintended UI changes
- **Performance Budgets**: Monitor and enforce performance thresholds
- **Living Documentation**: Tests as executable specifications
## Common Patterns
### AAA Pattern (Arrange-Act-Assert)
```typescript
describe('UserService', () => {
it('should create user with valid data', async () => {
// Arrange
const userData = { name: 'John', email: 'john@example.com' };
const mockDb = vi.fn().mockResolvedValue({ id: 1, ...userData });
// Act
const result = await createUser(userData, mockDb);
// Assert
expect(result).toEqual({ id: 1, name: 'John', email: 'john@example.com' });
expect(mockDb).toHaveBeenCalledWith(userData);
});
});
```
### Page Object Model (Playwright)
```typescript
// page-objects/LoginPage.ts
export class LoginPage {
constructor(private page: Page) {}
async login(email: string, password: string) {
await this.page.fill('[data-testid="email"]', email);
await this.page.fill('[data-testid="password"]', password);
await this.page.click('[data-testid="login-button"]');
}
async getErrorMessage() {
return this.page.textContent('[data-testid="error-message"]');
}
}
// tests/login.spec.ts
test('should show error for invalid credentials', async ({ page }) => {
const loginPage = new LoginPage(page);
await loginPage.login('invalid@example.com', 'wrong');
expect(await loginPage.getErrorMessage()).toBe('Invalid credentials');
});
```
### Factory Pattern for Test Data
```typescript
// factories/user.factory.ts
export class UserFactory {
static create(overrides: Partial<User> = {}): User {
return {
id: faker.datatype.uuid(),
name: faker.name.fullName(),
email: faker.internet.email(),
createdAt: new Date(),
...overrides,
};
}
static createMany(count: number, overrides: Partial<User> = {}): User[] {
return Array.from({ length: count }, () => this.create(overrides));
}
}
// Usage in tests
const user = UserFactory.create({ email: 'test@example.com' });
const users = UserFactory.createMany(5);
```
### Custom Matchers (Vitest)
```typescript
// test-utils/matchers.ts
expect.extend({
toBeWithinRange(received: number, floor: number, ceiling: number) {
const pass = received >= floor && received <= ceiling;
return {
pass,
message: () => `Expected ${received} to be within range ${floor}-${ceiling}`,
};
},
});
// Usage
expect(response.time).toBeWithinRange(100, 300);
```
### MSW for API Mocking
```typescript
// mocks/handlers.ts
import { rest } from 'msw';
export const handlers = [
rest.get('/api/users/:id', (req, res, ctx) => {
return res(
ctx.status(200),
ctx.json({ id: req.params.id, name: 'John Doe' })
);
}),
];
// setup.ts
import { setupServer } from 'msw/node';
import { handlers } from './mocks/handlers';
export const server = setupServer(...handlers);
beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());
afterAll(() => server.close());
```
### Accessibility Testing
```typescript
import { expect, test } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
test('should not have accessibility violations', async ({ page }) => {
await page.goto('/');
const accessibilityScanResults = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa'])
.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
});
```
### Property-Based Testing
```typescript
import fc from 'fast-check';
describe('sortNumbers', () => {
it('should sort any array of numbers', () => {
fc.assert(
fc.property(fc.array(fc.integer()), (numbers) => {
const sorted = sortNumbers(numbers);
// Check sorted array is same length
expect(sorted.length).toBe(numbers.length);
// Check elements are in order
for (let i = 1; i < sorted.length; i++) {
expect(sorted[i]).toBeGreaterThanOrEqual(sorted[i - 1]);
}
})
);
});
});
```
## Testing Anti-Patterns to Avoid
**❌ Testing Implementation Details**:
```typescript
// BAD: Testing internal state
expect(component.state.isLoading).toBe(true);
// GOOD: Testing user-visible behavior
expect(screen.getByText('Loading...')).toBeInTheDocument();
```
**❌ Tests with Shared State**:
```typescript
// BAD: Shared state between tests
let user;
beforeAll(() => { user = createUser(); });
// GOOD: Each test creates its own data
beforeEach(() => { user = createUser(); });
```
**❌ Hardcoded Waits**:
```typescript
// BAD: Arbitrary wait
await page.waitForTimeout(3000);
// GOOD: Wait for specific condition
await page.waitForSelector('[data-testid="result"]');
```
**❌ Over-Mocking**:
```typescript
// BAD: Mocking everything
vi.mock('./database');
vi.mock('./api');
vi.mock('./utils');
// GOOD: Mock only external dependencies
// Test real integration when possible
```
**❌ Generic Test Names**:
```typescript
// BAD: Unclear test name
it('works correctly', () => { ... });
// GOOD: Descriptive test name
it('should return 404 when user not found', () => { ... });
```
## Quality Checklists
### Pre-Merge Checklist
- [ ] All tests passing locally
- [ ] Coverage meets thresholds (80%+ lines)
- [ ] No new accessibility violations
- [ ] Visual regression tests reviewed
- [ ] E2E tests pass for critical paths
- [ ] Performance budgets not exceeded
- [ ] Security audit passes (no critical vulnerabilities)
- [ ] Flaky tests fixed or quarantined
- [ ] Test execution time acceptable
### Release Checklist
- [ ] Full regression test suite passes
- [ ] Cross-browser tests pass (Chrome, Firefox, Safari)
- [ ] Mobile/responsive tests pass
- [ ] Performance testing completed
- [ ] Load/stress testing completed
- [ ] Security penetration testing completed
- [ ] Accessibility audit completed (WCAG AA)
- [ ] Visual regression baseline updated
- [ ] Monitoring and alerting configured
- [ ] Rollback plan tested
You are ready to ensure world-class quality through comprehensive testing strategies!
## How to Invoke This Agent
Use the Task tool with the following subagent type:
```typescript
Task({
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
prompt: "Your QA/testing task here",
description: "Brief task description"
})
```
**Example**:
```typescript
Task({
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
prompt: "Create a comprehensive test strategy for an e-commerce checkout flow using Playwright E2E and Vitest unit tests",
description: "Design test strategy for checkout"
})
```

443
agents/qa-engineer/README.md Normal file
View File

@@ -0,0 +1,443 @@
# QA Engineer Agent
## Overview
The QA Engineer agent is a specialized AI agent designed to help with test strategy, test planning, test automation, and comprehensive quality assurance for modern software projects.
## When to Use This Agent
Invoke this agent when you need:
- **Test Strategy**: Design comprehensive testing approaches (unit, integration, E2E)
- **Test Planning**: Break down features into testable scenarios
- **Test Automation**: Set up Playwright, Vitest, Cypress, or other testing frameworks
- **TDD Workflow**: Implement test-driven development practices
- **Quality Gates**: Define coverage thresholds and quality metrics
- **Performance Testing**: Load testing, stress testing, benchmarking
- **Accessibility Testing**: WCAG compliance, a11y automation
- **Security Testing**: Vulnerability scanning, penetration testing
- **CI/CD Integration**: Automated testing in pipelines
- **Test Maintenance**: Refactor flaky tests, improve test reliability
## How to Invoke
Use the Task tool with `subagent_type`:
```typescript
Task({
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
prompt: "Design a comprehensive test strategy for a React e-commerce application with Vitest unit tests, Playwright E2E tests, and accessibility testing"
});
```
## Agent Capabilities
### 1. Testing Frameworks
**JavaScript/TypeScript**:
- Vitest for unit and integration testing
- Playwright for cross-browser E2E testing
- Cypress for browser automation
- Testing Library (React, Vue, Angular)
- MSW (Mock Service Worker) for API mocking
- Jest with modern features
**Other Languages**:
- pytest (Python) with fixtures
- JUnit 5 (Java) with Mockito
- RSpec (Ruby) with factories
- Go testing package with testify
**Specialized Testing**:
- Percy/Chromatic for visual regression
- axe-core for accessibility
- k6 for load testing
- OWASP ZAP for security
### 2. Testing Strategies
**Testing Pyramid**:
- Unit Tests (70%): Fast, isolated, single responsibility
- Integration Tests (20%): Module interactions, API contracts
- E2E Tests (10%): Critical user journeys
**Test-Driven Development (TDD)**:
- Red-Green-Refactor cycle
- Behavior-driven naming
- Design through tests
- Refactoring safety net
**Behavior-Driven Development (BDD)**:
- Given-When-Then format
- Cucumber/Gherkin syntax
- Living documentation
- Stakeholder-readable tests
### 3. Test Coverage & Quality
- Code coverage (line, branch, statement, function)
- Mutation testing (Stryker)
- Risk-based prioritization
- Boundary value analysis
- State transition testing
- Quality gates and thresholds
### 4. Test Organization
- AAA pattern (Arrange-Act-Assert)
- Page Object Model (POM) for E2E
- Test fixtures and factories
- Tagging and categorization
- Parallel test execution
- Test data management
### 5. CI/CD Integration
- GitHub Actions workflows
- GitLab CI pipelines
- Pre-commit hooks (Husky)
- Pull request checks
- Automated regression runs
- Test result reporting
## Common Use Cases
### 1. Design Test Strategy
**Prompt**:
```
"Design a test strategy for a Next.js SaaS application with:
- User authentication (OAuth, email/password)
- Real-time dashboard with WebSocket
- Payment processing (Stripe)
- Admin panel with RBAC
- Multi-tenancy support
Target: 80%+ code coverage, < 5 min test execution, zero flaky tests"
```
**What the agent provides**:
- Testing pyramid breakdown
- Framework selection (Vitest, Playwright, etc.)
- Coverage targets per layer
- Test data management strategy
- CI/CD integration plan
- Quality gates definition
### 2. Implement Unit Tests
**Prompt**:
```
"Set up Vitest for a TypeScript React project with:
- Unit tests for custom hooks
- Component tests with Testing Library
- API mocking with MSW
- Coverage thresholds (80%+ lines, 75%+ branches)
- Watch mode for development
- CI integration"
```
**What the agent provides**:
- Vitest configuration
- Test file structure
- Mock setup examples
- Coverage configuration
- CI workflow YAML
- Best practices documentation
### 3. Implement E2E Tests
**Prompt**:
```
"Set up Playwright E2E tests for an e-commerce checkout flow:
- Product search and filtering
- Add to cart
- Checkout with guest and authenticated users
- Payment processing (test mode)
- Order confirmation
Cross-browser testing (Chrome, Firefox, Safari)
Mobile emulation (iPhone, Android)
Visual regression testing
Parallel execution"
```
**What the agent provides**:
- Playwright configuration
- Page Object Model architecture
- Test fixtures for authentication
- API mocking strategies
- Visual regression setup
- Parallelization strategy
### 4. Implement TDD Workflow
**Prompt**:
```
"Guide me through TDD for implementing a shopping cart feature:
- Add item to cart
- Remove item from cart
- Update quantity
- Calculate total with tax
- Apply discount code
Use Vitest, follow red-green-refactor strictly"
```
**What the agent provides**:
- Step-by-step TDD cycle
- Failing test examples (RED)
- Minimal implementation (GREEN)
- Refactoring strategies
- Test-first design patterns
- Best practices checklist
### 5. Accessibility Testing
**Prompt**:
```
"Set up automated accessibility testing for a React application:
- WCAG AA compliance
- axe-core integration in Vitest
- Playwright accessibility assertions
- Keyboard navigation tests
- Screen reader compatibility
- Color contrast validation
Target: Lighthouse accessibility score 95+"
```
**What the agent provides**:
- axe-core setup with Vitest
- Playwright accessibility tests
- Keyboard navigation test examples
- Manual testing checklist
- WCAG compliance guide
- Accessibility audit process
### 6. Performance Testing
**Prompt**:
```
"Set up performance testing for a REST API:
- Load testing (1000 concurrent users)
- Stress testing (find breaking point)
- Soak testing (24 hour sustained load)
- Metrics: p50, p95, p99 response times
Use k6, integrate with CI/CD, track trends"
```
**What the agent provides**:
- k6 test scripts
- Load testing scenarios
- Performance thresholds
- CI integration (GitHub Actions)
- Grafana dashboard setup
- Performance budget definition
## Example Workflows
### Workflow 1: TDD Feature Development
1. **Write Failing Test (RED)**: Define expected behavior
2. **Implement Minimally (GREEN)**: Make test pass
3. **Refactor**: Improve code quality
4. **Repeat**: Next requirement
### Workflow 2: E2E Test Setup
1. **Install Playwright**: `npm install -D @playwright/test`
2. **Configure**: Browser setup, base URL, screenshots
3. **Create Page Objects**: Reusable page interactions
4. **Write Tests**: Critical user journeys
5. **CI Integration**: Run on every PR
6. **Monitor**: Track flakiness and execution time
### Workflow 3: Quality Audit
1. **Analyze Coverage**: Identify gaps
2. **Review Test Quality**: Flaky tests, slow tests
3. **Refactor**: Improve reliability
4. **Set Thresholds**: Enforce quality gates
5. **Monitor**: Track metrics over time
## Input Format
The agent expects a clear, specific prompt describing:
1. **Context**: What are you testing? What technology stack?
2. **Requirements**: What features need tests?
3. **Quality Targets**: Coverage %, performance, accessibility
4. **Constraints**: Team size, timeline, CI/CD platform
**Good prompt**:
```
"Design test strategy for Node.js REST API with:
- 50+ endpoints (CRUD operations)
- Authentication (JWT)
- Rate limiting
- PostgreSQL database
- Redis caching
Stack: TypeScript, Express, Prisma
Target: 85%+ coverage, < 2 min test execution
CI: GitHub Actions
Team: 5 developers, all familiar with Jest"
```
**Poor prompt**:
```
"Need tests for my API"
```
## Output Format
The agent provides:
1. **Test Strategy**: High-level approach and distribution
2. **Framework Setup**: Configuration files and installation
3. **Test Examples**: Code samples for each test type
4. **Best Practices**: Patterns and anti-patterns
5. **CI/CD Integration**: Workflow files
6. **Quality Metrics**: Coverage targets, thresholds
7. **Documentation**: Testing guidelines for team
## Best Practices
### 1. Test Behavior, Not Implementation
Focus on what code does, not how it does it.
### 2. Independent Tests
No shared state between tests.
### 3. Fast Feedback
Unit tests should run in seconds.
### 4. Readable Tests
Self-documenting test names and structure.
### 5. Maintainable Tests
Page Object Model, fixtures, utilities.
### 6. Realistic Tests
Test against production-like environment.
### 7. Coverage Targets
80%+ code coverage, 100% critical paths.
### 8. Flakiness Zero Tolerance
Fix or quarantine flaky tests immediately.
## Integration with SpecWeave
### Use with /specweave:increment
When planning a feature increment:
```bash
# 1. Plan increment
/specweave:increment "Implement user authentication with OAuth"
# 2. Generate test strategy
Task({
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
prompt: "Design test strategy for OAuth authentication with Google, GitHub, and email/password. Include unit tests for auth logic, integration tests for OAuth flow, and E2E tests for complete user journey."
});
# 3. Implement with TDD
/specweave:do
```
### Use with /specweave:qa
After implementation, validate test coverage:
```bash
# Run quality assessment
/specweave:qa 0123
# Agent checks:
# - Test coverage (lines, branches, functions)
# - Test quality (independent, fast, readable)
# - Critical path coverage
# - E2E test completeness
# - Accessibility test coverage
```
## Troubleshooting
### Issue: Flaky tests
**Solution**:
- Use proper wait strategies (no hardcoded delays)
- Isolate test data
- Mock external dependencies
- Use fake timers for time-based code
### Issue: Slow tests
**Solution**:
- Mock expensive operations (DB, API, file I/O)
- Use fake timers
- Run tests in parallel
- Profile slow tests
### Issue: Low coverage
**Solution**:
- Identify untested paths
- Add tests for edge cases
- Test error handling
- Use mutation testing to find weak tests
## Advanced Usage
### Contract Testing
```typescript
Task({
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
prompt: `Set up Pact contract testing between:
- Frontend (React SPA)
- Backend API (Node.js)
- Third-party payment API (Stripe)
Ensure API compatibility across teams`
});
```
### Visual Regression Testing
```typescript
Task({
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
prompt: `Set up visual regression testing with Percy for:
- 20+ pages across 3 breakpoints (mobile, tablet, desktop)
- 5+ themes (light, dark, high-contrast)
- Component library (Storybook integration)
CI integration, baseline management`
});
```
## Resources
- **Vitest Docs**: https://vitest.dev
- **Playwright Docs**: https://playwright.dev
- **Testing Library**: https://testing-library.com
- **TDD Guide**: https://martinfowler.com/bliki/TestDrivenDevelopment.html
- **Testing Best Practices**: https://testingjavascript.com
## Version History
- **v1.0.0** (2025-01-22): Initial release with Vitest, Playwright, TDD expertise
- **v1.1.0** (TBD): Add mobile testing support (Appium, Detox)
- **v1.2.0** (TBD): Add chaos engineering guidance
## Support
For issues or questions:
- GitHub Issues: https://github.com/anton-abyzov/specweave/issues
- Discord: https://discord.gg/specweave
- Documentation: https://spec-weave.com

470
agents/qa-engineer/templates/playwright-e2e-test.ts Normal file
View File

@@ -0,0 +1,470 @@
/**
* Playwright E2E Test Template
*
* This template demonstrates best practices for writing end-to-end tests
* with Playwright for web applications.
*
* Features:
* - Page Object Model (POM)
* - Test fixtures
* - Cross-browser testing
* - Mobile emulation
* - API mocking
* - Visual regression
* - Accessibility testing
*/
import { test, expect, Page } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
// ============================================================================
// PAGE OBJECTS
// ============================================================================
/**
* Login Page Object
*
* Encapsulates all interactions with the login page
*/
class LoginPage {
constructor(private page: Page) {}
// Locators
get emailInput() {
return this.page.getByLabel('Email');
}
get passwordInput() {
return this.page.getByLabel('Password');
}
get loginButton() {
return this.page.getByRole('button', { name: 'Login' });
}
get errorMessage() {
return this.page.getByRole('alert');
}
// Actions
async goto() {
await this.page.goto('/login');
}
async login(email: string, password: string) {
await this.emailInput.fill(email);
await this.passwordInput.fill(password);
await this.loginButton.click();
}
async expectError(message: string) {
await expect(this.errorMessage).toContainText(message);
}
}
/**
* Dashboard Page Object
*/
class DashboardPage {
constructor(private page: Page) {}
get welcomeMessage() {
return this.page.getByText(/Welcome,/);
}
get logoutButton() {
return this.page.getByRole('button', { name: 'Logout' });
}
async expectLoggedIn(username: string) {
await expect(this.welcomeMessage).toContainText(username);
}
async logout() {
await this.logoutButton.click();
}
}
// ============================================================================
// TEST FIXTURES
// ============================================================================
/**
* Custom fixture that provides authenticated page
*/
const test = base.extend<{ authenticatedPage: Page }>({
authenticatedPage: async ({ page }, use) => {
// Setup: Login before test
const loginPage = new LoginPage(page);
await loginPage.goto();
await loginPage.login('user@example.com', 'password123');
// Wait for navigation to dashboard
await page.waitForURL('/dashboard');
// Provide authenticated page to test
await use(page);
// Teardown: Logout after test
const dashboardPage = new DashboardPage(page);
await dashboardPage.logout();
},
});
// ============================================================================
// BASIC E2E TESTS
// ============================================================================
test.describe('Authentication Flow', () => {
test('should login successfully with valid credentials', async ({ page }) => {
// ARRANGE
const loginPage = new LoginPage(page);
await loginPage.goto();
// ACT
await loginPage.login('user@example.com', 'password123');
// ASSERT
await expect(page).toHaveURL('/dashboard');
const dashboardPage = new DashboardPage(page);
await dashboardPage.expectLoggedIn('User');
});
test('should show error for invalid credentials', async ({ page }) => {
// ARRANGE
const loginPage = new LoginPage(page);
await loginPage.goto();
// ACT
await loginPage.login('invalid@example.com', 'wrongpassword');
// ASSERT
await loginPage.expectError('Invalid email or password');
await expect(page).toHaveURL('/login'); // Still on login page
});
test('should show validation errors for empty fields', async ({ page }) => {
// ARRANGE
const loginPage = new LoginPage(page);
await loginPage.goto();
// ACT
await loginPage.loginButton.click();
// ASSERT
await expect(page.getByText('Email is required')).toBeVisible();
await expect(page.getByText('Password is required')).toBeVisible();
});
});
// ============================================================================
// AUTHENTICATED TESTS (Using Fixture)
// ============================================================================
test.describe('Dashboard Features', () => {
test('should display user profile', async ({ authenticatedPage }) => {
// Navigate to profile
await authenticatedPage.goto('/profile');
// Verify profile data
await expect(
authenticatedPage.getByText('user@example.com')
).toBeVisible();
await expect(authenticatedPage.getByText('Member since')).toBeVisible();
});
test('should allow editing profile', async ({ authenticatedPage }) => {
// Navigate to profile
await authenticatedPage.goto('/profile');
// Edit name
const nameInput = authenticatedPage.getByLabel('Name');
await nameInput.clear();
await nameInput.fill('New Name');
// Save changes
await authenticatedPage
.getByRole('button', { name: 'Save Changes' })
.click();
// Verify success
await expect(
authenticatedPage.getByText('Profile updated successfully')
).toBeVisible();
});
});
// ============================================================================
// API MOCKING
// ============================================================================
test.describe('API Mocking', () => {
test('should handle API errors gracefully', async ({ page }) => {
// Mock API to return error
await page.route('**/api/users', (route) => {
route.fulfill({
status: 500,
body: JSON.stringify({ error: 'Internal Server Error' }),
});
});
// Navigate to users page
await page.goto('/users');
// Verify error message
await expect(
page.getByText('Failed to load users. Please try again.')
).toBeVisible();
});
test('should display mocked user data', async ({ page }) => {
// Mock API to return test data
await page.route('**/api/users', (route) => {
route.fulfill({
status: 200,
body: JSON.stringify([
{ id: 1, name: 'John Doe', email: 'john@example.com' },
{ id: 2, name: 'Jane Smith', email: 'jane@example.com' },
]),
});
});
// Navigate to users page
await page.goto('/users');
// Verify mocked data is displayed
await expect(page.getByText('John Doe')).toBeVisible();
await expect(page.getByText('Jane Smith')).toBeVisible();
});
});
// ============================================================================
// VISUAL REGRESSION TESTING
// ============================================================================
test.describe('Visual Regression', () => {
test('homepage matches baseline', async ({ page }) => {
await page.goto('/');
// Capture screenshot and compare to baseline
await expect(page).toHaveScreenshot('homepage.png', {
fullPage: true,
animations: 'disabled',
});
});
test('button states match baseline', async ({ page }) => {
await page.goto('/components');
const button = page.getByRole('button', { name: 'Submit' });
// Default state
await expect(button).toHaveScreenshot('button-default.png');
// Hover state
await button.hover();
await expect(button).toHaveScreenshot('button-hover.png');
// Focus state
await button.focus();
await expect(button).toHaveScreenshot('button-focus.png');
});
});
// ============================================================================
// ACCESSIBILITY TESTING
// ============================================================================
test.describe('Accessibility', () => {
test('should not have accessibility violations', async ({ page }) => {
await page.goto('/');
// Run axe accessibility scan
const accessibilityScanResults = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
.analyze();
// Assert no violations
expect(accessibilityScanResults.violations).toEqual([]);
});
test('should support keyboard navigation', async ({ page }) => {
await page.goto('/form');
// Tab through form fields
await page.keyboard.press('Tab');
await expect(page.getByLabel('Email')).toBeFocused();
await page.keyboard.press('Tab');
await expect(page.getByLabel('Password')).toBeFocused();
await page.keyboard.press('Tab');
await expect(
page.getByRole('button', { name: 'Submit' })
).toBeFocused();
// Submit with Enter
await page.keyboard.press('Enter');
});
test('should have proper ARIA labels', async ({ page }) => {
await page.goto('/');
// Check navigation has aria-label
await expect(
page.getByRole('navigation', { name: 'Main navigation' })
).toBeVisible();
// Check main content has aria-label
await expect(page.getByRole('main')).toHaveAttribute(
'aria-label',
'Main content'
);
// Check all images have alt text
const images = await page.getByRole('img').all();
for (const img of images) {
await expect(img).toHaveAttribute('alt');
}
});
});
// ============================================================================
// MOBILE TESTING
// ============================================================================
test.describe('Mobile Experience', () => {
test.use({ viewport: { width: 375, height: 667 } }); // iPhone SE
test('should render mobile navigation', async ({ page }) => {
await page.goto('/');
// Mobile menu button should be visible
await expect(
page.getByRole('button', { name: 'Menu' })
).toBeVisible();
// Desktop navigation should be hidden
const desktopNav = page.getByRole('navigation').first();
await expect(desktopNav).toBeHidden();
});
test('should handle touch gestures', async ({ page }) => {
await page.goto('/gallery');
// Get image element
const image = page.getByRole('img').first();
// Swipe left
await image.dispatchEvent('touchstart', {
touches: [{ clientX: 300, clientY: 200 }],
});
await image.dispatchEvent('touchmove', {
touches: [{ clientX: 100, clientY: 200 }],
});
await image.dispatchEvent('touchend');
// Verify navigation to next image
await expect(page.getByText('Image 2 of 10')).toBeVisible();
});
});
// ============================================================================
// PERFORMANCE TESTING
// ============================================================================
test.describe('Performance', () => {
test('page load performance', async ({ page }) => {
await page.goto('/');
// Get performance metrics
const performanceMetrics = await page.evaluate(() => {
const perfData = window.performance.timing;
return {
loadTime: perfData.loadEventEnd - perfData.navigationStart,
domContentLoaded:
perfData.domContentLoadedEventEnd - perfData.navigationStart,
};
});
// Assert performance targets
expect(performanceMetrics.loadTime).toBeLessThan(3000); // 3s max
expect(performanceMetrics.domContentLoaded).toBeLessThan(2000); // 2s max
});
});
// ============================================================================
// FILE UPLOAD/DOWNLOAD
// ============================================================================
test.describe('File Operations', () => {
test('should upload file', async ({ page }) => {
await page.goto('/upload');
// Set up file chooser
const fileChooserPromise = page.waitForEvent('filechooser');
await page.getByRole('button', { name: 'Upload File' }).click();
const fileChooser = await fileChooserPromise;
// Upload file
await fileChooser.setFiles('tests/fixtures/test-file.pdf');
// Verify upload success
await expect(page.getByText('File uploaded successfully')).toBeVisible();
});
test('should download file', async ({ page }) => {
await page.goto('/downloads');
// Set up download
const downloadPromise = page.waitForEvent('download');
await page.getByRole('link', { name: 'Download Report' }).click();
const download = await downloadPromise;
// Verify download
expect(download.suggestedFilename()).toBe('report.pdf');
await download.saveAs(`/tmp/${download.suggestedFilename()}`);
});
});
// ============================================================================
// MULTI-TAB/WINDOW TESTING
// ============================================================================
test.describe('Multi-Window', () => {
test('should handle popup windows', async ({ context, page }) => {
await page.goto('/');
// Wait for popup
const [popup] = await Promise.all([
context.waitForEvent('page'),
page.getByRole('button', { name: 'Open Help' }).click(),
]);
// Interact with popup
await expect(popup.getByText('Help Center')).toBeVisible();
await popup.close();
});
});
// ============================================================================
// BEST PRACTICES CHECKLIST
// ============================================================================
/*
✅ Page Object Model (POM)
✅ Test Fixtures for Setup/Teardown
✅ Descriptive Test Names
✅ Auto-Waiting (Playwright built-in)
✅ User-Centric Selectors (getByRole, getByLabel)
✅ API Mocking for Reliability
✅ Visual Regression Testing
✅ Accessibility Testing (axe-core)
✅ Mobile/Responsive Testing
✅ Performance Assertions
✅ File Upload/Download
✅ Multi-Tab/Window Handling
✅ Screenshot/Video on Failure (configured in playwright.config.ts)
✅ Parallel Execution (configured in playwright.config.ts)
✅ Cross-Browser Testing (configured in playwright.config.ts)
*/

507
agents/qa-engineer/templates/test-data-factory.ts Normal file
View File

@@ -0,0 +1,507 @@
/**
* Test Data Factory Template
*
* This template demonstrates best practices for creating reusable
* test data factories using the Factory pattern.
*
* Benefits:
* - Consistent test data generation
* - Easy to customize with overrides
* - Reduces test setup boilerplate
* - Type-safe with TypeScript
*/
import { faker } from '@faker-js/faker';
// ============================================================================
// TYPES
// ============================================================================
export interface User {
id: string;
email: string;
username: string;
firstName: string;
lastName: string;
role: 'admin' | 'user' | 'guest';
isActive: boolean;
createdAt: Date;
updatedAt: Date;
profile?: UserProfile;
}
export interface UserProfile {
bio: string;
avatar: string;
phoneNumber: string;
address: Address;
}
export interface Address {
street: string;
city: string;
state: string;
zipCode: string;
country: string;
}
export interface Product {
id: string;
name: string;
description: string;
price: number;
category: string;
inStock: boolean;
quantity: number;
imageUrl: string;
createdAt: Date;
}
export interface Order {
id: string;
userId: string;
items: OrderItem[];
subtotal: number;
tax: number;
total: number;
status: 'pending' | 'processing' | 'shipped' | 'delivered' | 'cancelled';
shippingAddress: Address;
createdAt: Date;
updatedAt: Date;
}
export interface OrderItem {
productId: string;
quantity: number;
price: number;
}
// ============================================================================
// FACTORY FUNCTIONS
// ============================================================================
/**
* User Factory
*
* Creates realistic user test data with sensible defaults
*/
export class UserFactory {
/**
* Create a single user
*
* @param overrides - Partial user object to override defaults
* @returns Complete user object
*
* @example
* ```ts
* const admin = UserFactory.create({ role: 'admin' });
* const inactiveUser = UserFactory.create({ isActive: false });
* ```
*/
static create(overrides: Partial<User> = {}): User {
const firstName = faker.person.firstName();
const lastName = faker.person.lastName();
const email =
overrides.email || faker.internet.email({ firstName, lastName });
return {
id: faker.string.uuid(),
email,
username: faker.internet.userName({ firstName, lastName }),
firstName,
lastName,
role: 'user',
isActive: true,
createdAt: faker.date.past(),
updatedAt: faker.date.recent(),
...overrides,
};
}
/**
* Create multiple users
*
* @param count - Number of users to create
* @param overrides - Partial user object to override defaults for all users
* @returns Array of user objects
*
* @example
* ```ts
* const users = UserFactory.createMany(5);
* const admins = UserFactory.createMany(3, { role: 'admin' });
* ```
*/
static createMany(count: number, overrides: Partial<User> = {}): User[] {
return Array.from({ length: count }, () => this.create(overrides));
}
/**
* Create admin user
*
* @param overrides - Partial user object to override defaults
* @returns Admin user object
*/
static createAdmin(overrides: Partial<User> = {}): User {
return this.create({
role: 'admin',
...overrides,
});
}
/**
* Create user with complete profile
*
* @param overrides - Partial user object to override defaults
* @returns User with profile object
*/
static createWithProfile(overrides: Partial<User> = {}): User {
return this.create({
profile: {
bio: faker.person.bio(),
avatar: faker.image.avatar(),
phoneNumber: faker.phone.number(),
address: AddressFactory.create(),
},
...overrides,
});
}
/**
* Create inactive user
*
* @param overrides - Partial user object to override defaults
* @returns Inactive user object
*/
static createInactive(overrides: Partial<User> = {}): User {
return this.create({
isActive: false,
...overrides,
});
}
}
/**
* Address Factory
*
* Creates realistic address test data
*/
export class AddressFactory {
static create(overrides: Partial<Address> = {}): Address {
return {
street: faker.location.streetAddress(),
city: faker.location.city(),
state: faker.location.state(),
zipCode: faker.location.zipCode(),
country: faker.location.country(),
...overrides,
};
}
static createUS(overrides: Partial<Address> = {}): Address {
return this.create({
country: 'United States',
zipCode: faker.location.zipCode('#####'),
state: faker.location.state({ abbreviated: true }),
...overrides,
});
}
}
/**
* Product Factory
*
* Creates realistic product test data
*/
export class ProductFactory {
static create(overrides: Partial<Product> = {}): Product {
return {
id: faker.string.uuid(),
name: faker.commerce.productName(),
description: faker.commerce.productDescription(),
price: parseFloat(faker.commerce.price()),
category: faker.commerce.department(),
inStock: true,
quantity: faker.number.int({ min: 0, max: 100 }),
imageUrl: faker.image.url(),
createdAt: faker.date.past(),
...overrides,
};
}
static createMany(count: number, overrides: Partial<Product> = {}): Product[] {
return Array.from({ length: count }, () => this.create(overrides));
}
static createOutOfStock(overrides: Partial<Product> = {}): Product {
return this.create({
inStock: false,
quantity: 0,
...overrides,
});
}
static createExpensive(overrides: Partial<Product> = {}): Product {
return this.create({
price: faker.number.int({ min: 1000, max: 10000 }),
...overrides,
});
}
}
/**
* Order Factory
*
* Creates realistic order test data with items
*/
export class OrderFactory {
static create(overrides: Partial<Order> = {}): Order {
const items = overrides.items || [
{
productId: faker.string.uuid(),
quantity: faker.number.int({ min: 1, max: 5 }),
price: parseFloat(faker.commerce.price()),
},
];
const subtotal = items.reduce((sum, item) => sum + item.price * item.quantity, 0);
const tax = subtotal * 0.08; // 8% tax
const total = subtotal + tax;
return {
id: faker.string.uuid(),
userId: faker.string.uuid(),
items,
subtotal,
tax,
total,
status: 'pending',
shippingAddress: AddressFactory.createUS(),
createdAt: faker.date.recent(),
updatedAt: faker.date.recent(),
...overrides,
};
}
static createMany(count: number, overrides: Partial<Order> = {}): Order[] {
return Array.from({ length: count }, () => this.create(overrides));
}
static createWithItems(items: OrderItem[], overrides: Partial<Order> = {}): Order {
return this.create({ items, ...overrides });
}
static createShipped(overrides: Partial<Order> = {}): Order {
return this.create({
status: 'shipped',
...overrides,
});
}
static createCancelled(overrides: Partial<Order> = {}): Order {
return this.create({
status: 'cancelled',
...overrides,
});
}
}
// ============================================================================
// BUILDER PATTERN (Advanced)
// ============================================================================
/**
* User Builder
*
* Provides a fluent interface for building complex user objects
*
* @example
* ```ts
* const user = new UserBuilder()
* .withEmail('admin@example.com')
* .withRole('admin')
* .withProfile()
* .build();
* ```
*/
export class UserBuilder {
private user: Partial<User> = {};
withId(id: string): this {
this.user.id = id;
return this;
}
withEmail(email: string): this {
this.user.email = email;
return this;
}
withUsername(username: string): this {
this.user.username = username;
return this;
}
withName(firstName: string, lastName: string): this {
this.user.firstName = firstName;
this.user.lastName = lastName;
return this;
}
withRole(role: User['role']): this {
this.user.role = role;
return this;
}
withProfile(profile?: UserProfile): this {
this.user.profile = profile || {
bio: faker.person.bio(),
avatar: faker.image.avatar(),
phoneNumber: faker.phone.number(),
address: AddressFactory.create(),
};
return this;
}
inactive(): this {
this.user.isActive = false;
return this;
}
active(): this {
this.user.isActive = true;
return this;
}
build(): User {
return UserFactory.create(this.user);
}
}
// ============================================================================
// USAGE EXAMPLES
// ============================================================================
/**
* Example: Simple user creation
*/
export function exampleSimpleUser() {
const user = UserFactory.create();
const admin = UserFactory.createAdmin();
const users = UserFactory.createMany(5);
return { user, admin, users };
}
/**
* Example: Customized user creation
*/
export function exampleCustomUser() {
const user = UserFactory.create({
email: 'custom@example.com',
role: 'admin',
isActive: false,
});
return user;
}
/**
* Example: Builder pattern
*/
export function exampleBuilder() {
const user = new UserBuilder()
.withEmail('builder@example.com')
.withName('John', 'Doe')
.withRole('admin')
.withProfile()
.active()
.build();
return user;
}
/**
* Example: Order with products
*/
export function exampleOrder() {
// Create products
const products = ProductFactory.createMany(3);
// Create order items from products
const items: OrderItem[] = products.map((product) => ({
productId: product.id,
quantity: faker.number.int({ min: 1, max: 3 }),
price: product.price,
}));
// Create order with items
const order = OrderFactory.createWithItems(items);
return { products, order };
}
// ============================================================================
// TEST USAGE
// ============================================================================
/**
* Example test using factories
*/
import { describe, it, expect } from 'vitest';
describe('UserService (using factories)', () => {
it('should create user', () => {
// ARRANGE
const userData = UserFactory.create();
// ACT
const result = userService.create(userData);
// ASSERT
expect(result.id).toBeDefined();
expect(result.email).toBe(userData.email);
});
it('should only allow admins to delete users', () => {
// ARRANGE
const admin = UserFactory.createAdmin();
const regularUser = UserFactory.create();
// ACT & ASSERT
expect(() => userService.deleteUser(regularUser.id, admin)).not.toThrow();
expect(() => userService.deleteUser(admin.id, regularUser)).toThrow('Unauthorized');
});
it('should calculate order total correctly', () => {
// ARRANGE
const order = OrderFactory.create({
items: [
{ productId: '1', quantity: 2, price: 50 },
{ productId: '2', quantity: 1, price: 30 },
],
});
// ACT
const total = orderService.calculateTotal(order);
// ASSERT
expect(total).toBe(140.4); // (50*2 + 30*1) * 1.08 tax
});
});
// ============================================================================
// BEST PRACTICES
// ============================================================================
/*
✅ Use realistic data (faker.js)
✅ Provide sensible defaults
✅ Allow overrides for customization
✅ Type-safe with TypeScript
✅ Create helper methods (createAdmin, createInactive, etc.)
✅ Builder pattern for complex objects
✅ Consistent naming (create, createMany, createWith...)
✅ Document with JSDoc
✅ Export all factories for reuse
✅ Keep factories simple and focused
*/

400
agents/qa-engineer/templates/vitest-unit-test.ts Normal file
View File

@@ -0,0 +1,400 @@
/**
* Vitest Unit Test Template
*
* This template demonstrates best practices for writing unit tests
* with Vitest for TypeScript projects.
*
* Features:
* - AAA pattern (Arrange-Act-Assert)
* - Test isolation
* - Mocking dependencies
* - Parametric testing
* - Error handling tests
*/
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
// Import the module under test
import { YourModule } from './YourModule';
// Import dependencies (to be mocked)
import { ExternalDependency } from './ExternalDependency';
// ============================================================================
// MOCKS
// ============================================================================
// Mock external dependencies
vi.mock('./ExternalDependency', () => ({
ExternalDependency: vi.fn().mockImplementation(() => ({
fetchData: vi.fn().mockResolvedValue({ data: 'mocked' }),
processData: vi.fn(),
})),
}));
// ============================================================================
// TEST SUITE
// ============================================================================
describe('YourModule', () => {
// ========================================================================
// SETUP & TEARDOWN
// ========================================================================
let instance: YourModule;
let mockDependency: any;
beforeEach(() => {
// ARRANGE: Set up fresh instance for each test
mockDependency = new ExternalDependency();
instance = new YourModule(mockDependency);
});
afterEach(() => {
// TEARDOWN: Clean up mocks
vi.clearAllMocks();
});
// ========================================================================
// HAPPY PATH TESTS
// ========================================================================
describe('methodName', () => {
it('should perform expected operation with valid input', () => {
// ARRANGE
const input = 'test-input';
const expectedOutput = 'test-output';
// ACT
const result = instance.methodName(input);
// ASSERT
expect(result).toBe(expectedOutput);
});
it('should call dependency with correct parameters', () => {
// ARRANGE
const input = 'test-input';
// ACT
instance.methodName(input);
// ASSERT
expect(mockDependency.processData).toHaveBeenCalledWith(input);
expect(mockDependency.processData).toHaveBeenCalledTimes(1);
});
});
// ========================================================================
// ASYNC TESTS
// ========================================================================
describe('asyncMethod', () => {
it('should resolve with data on success', async () => {
// ARRANGE
const mockData = { id: 1, value: 'test' };
mockDependency.fetchData.mockResolvedValue(mockData);
// ACT
const result = await instance.asyncMethod();
// ASSERT
expect(result).toEqual(mockData);
expect(mockDependency.fetchData).toHaveBeenCalled();
});
it('should handle async errors gracefully', async () => {
// ARRANGE
const error = new Error('Network error');
mockDependency.fetchData.mockRejectedValue(error);
// ACT & ASSERT
await expect(instance.asyncMethod()).rejects.toThrow('Network error');
});
});
// ========================================================================
// EDGE CASES
// ========================================================================
describe('edge cases', () => {
it('should handle empty input', () => {
// ACT
const result = instance.methodName('');
// ASSERT
expect(result).toBe('');
});
it('should handle null input', () => {
// ACT
const result = instance.methodName(null as any);
// ASSERT
expect(result).toBeNull();
});
it('should handle undefined input', () => {
// ACT
const result = instance.methodName(undefined as any);
// ASSERT
expect(result).toBeUndefined();
});
it('should handle very large input', () => {
// ARRANGE
const largeInput = 'x'.repeat(1000000);
// ACT
const result = instance.methodName(largeInput);
// ASSERT
expect(result).toBeDefined();
});
});
// ========================================================================
// ERROR HANDLING
// ========================================================================
describe('error handling', () => {
it('should throw error for invalid input', () => {
// ARRANGE
const invalidInput = -1;
// ACT & ASSERT
expect(() => instance.methodName(invalidInput)).toThrow(
'Input must be positive'
);
});
it('should throw specific error type', () => {
// ARRANGE
const invalidInput = 'invalid';
// ACT & ASSERT
expect(() => instance.methodName(invalidInput)).toThrow(ValidationError);
});
it('should include error details', () => {
// ARRANGE
const invalidInput = 'invalid';
// ACT
try {
instance.methodName(invalidInput);
fail('Expected error to be thrown');
} catch (error) {
// ASSERT
expect(error).toBeInstanceOf(ValidationError);
expect(error.message).toContain('invalid');
expect(error.code).toBe('INVALID_INPUT');
}
});
});
// ========================================================================
// PARAMETRIC TESTS (Table-Driven)
// ========================================================================
describe.each([
{ input: 1, expected: 2 },
{ input: 2, expected: 4 },
{ input: 3, expected: 6 },
{ input: 4, expected: 8 },
])('methodName($input)', ({ input, expected }) => {
it(`should return ${expected}`, () => {
// ACT
const result = instance.methodName(input);
// ASSERT
expect(result).toBe(expected);
});
});
// ========================================================================
// STATE MANAGEMENT TESTS
// ========================================================================
describe('state management', () => {
it('should update internal state correctly', () => {
// ARRANGE
const initialState = instance.getState();
// ACT
instance.updateState('new-value');
// ASSERT
const newState = instance.getState();
expect(newState).not.toBe(initialState);
expect(newState).toBe('new-value');
});
it('should emit events on state change', () => {
// ARRANGE
const eventHandler = vi.fn();
instance.on('stateChanged', eventHandler);
// ACT
instance.updateState('new-value');
// ASSERT
expect(eventHandler).toHaveBeenCalledWith('new-value');
});
});
// ========================================================================
// PERFORMANCE TESTS
// ========================================================================
describe('performance', () => {
it('should complete within reasonable time', () => {
// ARRANGE
const startTime = Date.now();
// ACT
instance.methodName('test');
// ASSERT
const endTime = Date.now();
const executionTime = endTime - startTime;
expect(executionTime).toBeLessThan(100); // 100ms threshold
});
it('should handle large datasets efficiently', () => {
// ARRANGE
const largeDataset = Array.from({ length: 10000 }, (_, i) => i);
// ACT
const startTime = Date.now();
const result = instance.processBatch(largeDataset);
const endTime = Date.now();
// ASSERT
expect(result).toHaveLength(10000);
expect(endTime - startTime).toBeLessThan(1000); // 1s threshold
});
});
// ========================================================================
// SNAPSHOT TESTS
// ========================================================================
describe('snapshots', () => {
it('should match snapshot for complex output', () => {
// ARRANGE
const input = { id: 1, name: 'Test', nested: { value: 42 } };
// ACT
const result = instance.transform(input);
// ASSERT
expect(result).toMatchSnapshot();
});
it('should match inline snapshot for simple output', () => {
// ARRANGE
const input = 'test';
// ACT
const result = instance.methodName(input);
// ASSERT
expect(result).toMatchInlineSnapshot('"test-output"');
});
});
// ========================================================================
// TIMER/DEBOUNCE TESTS
// ========================================================================
describe('timers', () => {
beforeEach(() => {
vi.useFakeTimers();
});
afterEach(() => {
vi.restoreAllTimers();
});
it('should debounce function calls', () => {
// ARRANGE
const callback = vi.fn();
const debounced = instance.debounce(callback, 1000);
// ACT
debounced();
debounced();
debounced();
// ASSERT (not called yet)
expect(callback).not.toHaveBeenCalled();
// Fast-forward time
vi.advanceTimersByTime(1000);
// ASSERT (called once)
expect(callback).toHaveBeenCalledTimes(1);
});
it('should throttle function calls', () => {
// ARRANGE
const callback = vi.fn();
const throttled = instance.throttle(callback, 1000);
// ACT
throttled(); // Called immediately
throttled(); // Ignored (within throttle window)
throttled(); // Ignored (within throttle window)
// ASSERT
expect(callback).toHaveBeenCalledTimes(1);
// Fast-forward time
vi.advanceTimersByTime(1000);
// ACT
throttled(); // Called after throttle window
// ASSERT
expect(callback).toHaveBeenCalledTimes(2);
});
});
});
// ============================================================================
// HELPER TYPES & CLASSES (For Examples Above)
// ============================================================================
class ValidationError extends Error {
constructor(
message: string,
public code: string
) {
super(message);
this.name = 'ValidationError';
}
}
// ============================================================================
// BEST PRACTICES CHECKLIST
// ============================================================================
/*
✅ AAA Pattern (Arrange-Act-Assert)
✅ Test Isolation (beforeEach creates fresh instance)
✅ Descriptive Test Names (should do X when Y)
✅ One Assertion Per Test (when possible)
✅ Mock External Dependencies
✅ Test Happy Path
✅ Test Edge Cases (null, undefined, empty, large)
✅ Test Error Handling
✅ Parametric Tests (test.each)
✅ Async Testing (async/await, rejects, resolves)
✅ Timer Testing (useFakeTimers)
✅ Performance Testing (execution time)
✅ Snapshot Testing (complex outputs)
✅ No Shared State Between Tests
✅ Fast Execution (< 1s per test)
*/

726
agents/qa-engineer/test-strategies.md Normal file
View File

@@ -0,0 +1,726 @@
# QA Engineer Agent - Test Strategies
Comprehensive guide to testing strategies for different application types and scenarios.
## Table of Contents
1. [Testing Pyramid Strategy](#testing-pyramid-strategy)
2. [Testing Trophy Strategy](#testing-trophy-strategy)
3. [TDD Red-Green-Refactor](#tdd-red-green-refactor)
4. [BDD Given-When-Then](#bdd-given-when-then)
5. [API Testing Strategy](#api-testing-strategy)
6. [Frontend Testing Strategy](#frontend-testing-strategy)
7. [Micro-Services Testing Strategy](#micro-services-testing-strategy)
8. [Performance Testing Strategy](#performance-testing-strategy)
9. [Security Testing Strategy](#security-testing-strategy)
10. [Accessibility Testing Strategy](#accessibility-testing-strategy)
---
## Testing Pyramid Strategy
### Overview
The Testing Pyramid emphasizes a broad base of fast, cheap unit tests, fewer integration tests, and minimal UI/E2E tests.
```
/\
/ \ E2E (10%)
/----\
/ \ Integration (20%)
/--------\
/ \ Unit (70%)
/--------------\
```
### Distribution
- **Unit Tests (70%)**: Test individual functions, classes, components in isolation
- **Integration Tests (20%)**: Test interactions between modules, APIs, databases
- **E2E Tests (10%)**: Test complete user journeys through the UI
### When to Use
- Traditional applications with clear layers
- Backend services with business logic
- Applications where unit tests provide high confidence
- Teams prioritizing fast feedback loops
### Implementation Example
**Unit Test (70% of suite)**:
```typescript
// src/utils/cart.test.ts
import { describe, it, expect } from 'vitest';
import { calculateTotal, applyDiscount } from './cart';
describe('Cart Utils', () => {
describe('calculateTotal', () => {
it('should sum item prices', () => {
const items = [
{ id: 1, price: 10 },
{ id: 2, price: 20 },
];
expect(calculateTotal(items)).toBe(30);
});
it('should return 0 for empty cart', () => {
expect(calculateTotal([])).toBe(0);
});
it('should handle decimal prices', () => {
const items = [
{ id: 1, price: 10.99 },
{ id: 2, price: 20.50 },
];
expect(calculateTotal(items)).toBeCloseTo(31.49, 2);
});
});
describe('applyDiscount', () => {
it('should apply percentage discount', () => {
expect(applyDiscount(100, 'SAVE20', { type: 'percentage', value: 20 })).toBe(80);
});
it('should apply fixed discount', () => {
expect(applyDiscount(100, 'SAVE10', { type: 'fixed', value: 10 })).toBe(90);
});
it('should not go below zero', () => {
expect(applyDiscount(50, 'SAVE100', { type: 'fixed', value: 100 })).toBe(0);
});
});
});
```
**Integration Test (20% of suite)**:
```typescript
// src/api/orders.integration.test.ts
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { createTestServer } from '../test-utils/server';
import { seedDatabase, clearDatabase } from '../test-utils/database';
describe('Orders API Integration', () => {
let server: TestServer;
beforeEach(async () => {
server = await createTestServer();
await seedDatabase();
});
afterEach(async () => {
await clearDatabase();
await server.close();
});
it('should create order and store in database', async () => {
const response = await server.request
.post('/api/orders')
.send({
userId: 'user-123',
items: [{ productId: 'prod-1', quantity: 2 }],
});
expect(response.status).toBe(201);
expect(response.body).toMatchObject({
id: expect.any(String),
userId: 'user-123',
status: 'pending',
});
// Verify database persistence
const order = await server.db.orders.findById(response.body.id);
expect(order).toBeTruthy();
expect(order.items).toHaveLength(1);
});
it('should send confirmation email on order creation', async () => {
const emailSpy = vi.spyOn(server.emailService, 'send');
await server.request
.post('/api/orders')
.send({
userId: 'user-123',
items: [{ productId: 'prod-1', quantity: 1 }],
});
expect(emailSpy).toHaveBeenCalledWith({
to: 'user@example.com',
subject: 'Order Confirmation',
template: 'order-confirmation',
});
});
});
```
**E2E Test (10% of suite)**:
```typescript
// e2e/checkout-flow.spec.ts
import { test, expect } from '@playwright/test';
test.describe('Checkout Flow', () => {
test('should complete purchase as guest user', async ({ page }) => {
// Navigate to product
await page.goto('/products/laptop-123');
await expect(page.getByRole('heading', { name: 'Gaming Laptop' })).toBeVisible();
// Add to cart
await page.getByRole('button', { name: 'Add to Cart' }).click();
await expect(page.getByText('Item added to cart')).toBeVisible();
// Go to checkout
await page.getByRole('link', { name: 'Cart (1)' }).click();
await page.getByRole('button', { name: 'Checkout' }).click();
// Fill shipping info
await page.getByLabel('Email').fill('guest@example.com');
await page.getByLabel('Full Name').fill('John Doe');
await page.getByLabel('Address').fill('123 Main St');
await page.getByLabel('City').fill('New York');
await page.getByLabel('Zip Code').fill('10001');
// Fill payment info (test mode)
await page.getByLabel('Card Number').fill('4242424242424242');
await page.getByLabel('Expiry Date').fill('12/25');
await page.getByLabel('CVC').fill('123');
// Submit order
await page.getByRole('button', { name: 'Place Order' }).click();
// Verify confirmation
await expect(page).toHaveURL(/\/order-confirmation/);
await expect(page.getByText('Order Confirmed!')).toBeVisible();
await expect(page.getByText(/Order #/)).toBeVisible();
});
});
```
### Coverage Targets
- **Unit Tests**: 80%+ line coverage, 75%+ branch coverage
- **Integration Tests**: 100% critical API endpoints
- **E2E Tests**: 100% critical user journeys
---
## Testing Trophy Strategy
### Overview
Modern approach that emphasizes integration tests over unit tests, with static analysis as the foundation.
```
/\
/ \ E2E (5%)
/----\
/ \ Integration (50%)
/--------\
/ \ Unit (25%)
/--------------\
/ \ Static (20%)
/------------------\
```
### Distribution
- **Static Analysis (20%)**: TypeScript, ESLint, Prettier
- **Unit Tests (25%)**: Pure functions, utilities, critical logic
- **Integration Tests (50%)**: Components with dependencies, API contracts
- **E2E Tests (5%)**: Critical business flows only
### When to Use
- Modern frontend applications (React, Vue, Angular)
- Applications with complex component interactions
- Teams using TypeScript and static analysis tools
- Applications where integration tests catch more bugs
### Implementation Example
**Static Analysis (20%)**:
```json
// tsconfig.json
{
"compilerOptions": {
"strict": true,
"noImplicitAny": true,
"strictNullChecks": true,
"noUnusedLocals": true,
"noUnusedParameters": true
}
}
```
```json
// .eslintrc.json
{
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/recommended",
"plugin:react-hooks/recommended",
"plugin:jsx-a11y/recommended"
],
"rules": {
"@typescript-eslint/no-explicit-any": "error",
"react-hooks/exhaustive-deps": "error"
}
}
```
**Unit Tests (25%)**:
```typescript
// src/utils/formatters.test.ts
import { describe, it, expect } from 'vitest';
import { formatCurrency, formatDate } from './formatters';
describe('Formatters (Pure Functions)', () => {
describe('formatCurrency', () => {
it('should format USD currency', () => {
expect(formatCurrency(1234.56, 'USD')).toBe('$1,234.56');
});
it('should handle negative amounts', () => {
expect(formatCurrency(-100, 'USD')).toBe('-$100.00');
});
});
describe('formatDate', () => {
it('should format ISO date', () => {
const date = new Date('2025-01-15');
expect(formatDate(date, 'short')).toBe('1/15/2025');
});
});
});
```
**Integration Tests (50%)**:
```typescript
// src/components/UserProfile.integration.test.tsx
import { describe, it, expect, beforeEach } from 'vitest';
import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { rest } from 'msw';
import { setupServer } from 'msw/node';
import { UserProfile } from './UserProfile';
const server = setupServer(
rest.get('/api/users/:id', (req, res, ctx) => {
return res(
ctx.json({
id: req.params.id,
name: 'John Doe',
email: 'john@example.com',
})
);
}),
rest.put('/api/users/:id', (req, res, ctx) => {
return res(ctx.json({ ...req.body, updatedAt: Date.now() }));
})
);
beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());
afterAll(() => server.close());
describe('UserProfile Integration', () => {
it('should load and display user data', async () => {
render(<UserProfile userId="123" />);
// Loading state
expect(screen.getByText('Loading...')).toBeInTheDocument();
// Wait for data to load
await waitFor(() => {
expect(screen.getByText('John Doe')).toBeInTheDocument();
});
expect(screen.getByText('john@example.com')).toBeInTheDocument();
});
it('should update user profile on form submit', async () => {
const user = userEvent.setup();
render(<UserProfile userId="123" />);
// Wait for initial load
await screen.findByText('John Doe');
// Edit name
const nameInput = screen.getByLabelText('Name');
await user.clear(nameInput);
await user.type(nameInput, 'Jane Smith');
// Submit form
await user.click(screen.getByRole('button', { name: 'Save' }));
// Verify success message
await waitFor(() => {
expect(screen.getByText('Profile updated successfully')).toBeInTheDocument();
});
// Verify updated name
expect(screen.getByText('Jane Smith')).toBeInTheDocument();
});
it('should handle API errors gracefully', async () => {
// Mock API error
server.use(
rest.get('/api/users/:id', (req, res, ctx) => {
return res(ctx.status(500), ctx.json({ error: 'Internal Server Error' }));
})
);
render(<UserProfile userId="123" />);
// Wait for error message
await waitFor(() => {
expect(screen.getByText(/Failed to load user/i)).toBeInTheDocument();
});
});
});
```
**E2E Tests (5%)**:
```typescript
// e2e/critical-path.spec.ts
import { test, expect } from '@playwright/test';
test('critical user journey: signup to first purchase', async ({ page }) => {
// Only test the MOST critical path
await page.goto('/');
// Signup
await page.getByRole('link', { name: 'Sign Up' }).click();
await page.getByLabel('Email').fill('newuser@example.com');
await page.getByLabel('Password').fill('SecurePass123!');
await page.getByRole('button', { name: 'Create Account' }).click();
// Verify logged in
await expect(page.getByText('Welcome, New User')).toBeVisible();
// Make purchase
await page.goto('/products/best-seller');
await page.getByRole('button', { name: 'Buy Now' }).click();
await page.getByLabel('Card Number').fill('4242424242424242');
await page.getByRole('button', { name: 'Complete Purchase' }).click();
// Verify success
await expect(page.getByText('Purchase Successful')).toBeVisible();
});
```
### Coverage Targets
- **Static Analysis**: 100% (TypeScript strict mode, zero ESLint errors)
- **Unit Tests**: 90%+ for pure functions and utilities
- **Integration Tests**: 80%+ for components with dependencies
- **E2E Tests**: 100% critical paths only
---
## TDD Red-Green-Refactor
### The Cycle
1. **RED**: Write a failing test that defines expected behavior
2. **GREEN**: Write minimal code to make the test pass
3. **REFACTOR**: Improve code quality while keeping tests green
### Example: Shopping Cart Feature
**RED: Write Failing Test**:
```typescript
// src/cart/ShoppingCart.test.ts
import { describe, it, expect } from 'vitest';
import { ShoppingCart } from './ShoppingCart';
describe('ShoppingCart', () => {
it('should add item to cart', () => {
const cart = new ShoppingCart();
cart.addItem({ id: 1, name: 'Laptop', price: 1000 });
expect(cart.getItemCount()).toBe(1);
});
});
```
**Run test**: ❌ FAIL (ShoppingCart doesn't exist)
**GREEN: Minimal Implementation**:
```typescript
// src/cart/ShoppingCart.ts
interface CartItem {
id: number;
name: string;
price: number;
}
export class ShoppingCart {
private items: CartItem[] = [];
addItem(item: CartItem): void {
this.items.push(item);
}
getItemCount(): number {
return this.items.length;
}
}
```
**Run test**: ✅ PASS
**Add Another Test (Triangulation)**:
```typescript
it('should calculate total price', () => {
const cart = new ShoppingCart();
cart.addItem({ id: 1, name: 'Laptop', price: 1000 });
cart.addItem({ id: 2, name: 'Mouse', price: 50 });
expect(cart.getTotal()).toBe(1050);
});
```
**Run test**: ❌ FAIL (getTotal doesn't exist)
**GREEN: Implement getTotal**:
```typescript
export class ShoppingCart {
// ... previous code ...
getTotal(): number {
return this.items.reduce((sum, item) => sum + item.price, 0);
}
}
```
**Run test**: ✅ PASS
**REFACTOR: Improve Design**:
```typescript
export class ShoppingCart {
private items: Map<number, CartItem> = new Map();
addItem(item: CartItem): void {
const existing = this.items.get(item.id);
if (existing) {
// Increment quantity instead of duplicating
existing.quantity = (existing.quantity || 1) + 1;
} else {
this.items.set(item.id, { ...item, quantity: 1 });
}
}
getItemCount(): number {
return Array.from(this.items.values()).reduce(
(count, item) => count + (item.quantity || 1),
0
);
}
getTotal(): number {
return Array.from(this.items.values()).reduce(
(sum, item) => sum + item.price * (item.quantity || 1),
0
);
}
}
```
**Run tests**: ✅ ALL PASS (refactoring didn't break anything!)
### TDD Benefits
- Forces modular, testable design
- Prevents over-engineering
- Living documentation
- Fearless refactoring
- Faster debugging
---
## API Testing Strategy
### Layers
1. **Unit Tests**: Test business logic in isolation
2. **Integration Tests**: Test API endpoints with real database
3. **Contract Tests**: Test API contracts (Pact)
4. **E2E Tests**: Test complete API flows
### Example: REST API Testing
**Unit Test (Business Logic)**:
```typescript
// src/services/OrderService.test.ts
import { describe, it, expect, vi } from 'vitest';
import { OrderService } from './OrderService';
describe('OrderService', () => {
it('should calculate order total with tax', () => {
const mockRepo = {
save: vi.fn(),
findById: vi.fn(),
};
const service = new OrderService(mockRepo);
const order = service.calculateTotal({
items: [{ price: 100, quantity: 2 }],
taxRate: 0.08,
});
expect(order.subtotal).toBe(200);
expect(order.tax).toBeCloseTo(16, 2);
expect(order.total).toBeCloseTo(216, 2);
});
});
```
**Integration Test (API + Database)**:
```typescript
// tests/integration/orders.test.ts
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import supertest from 'supertest';
import { createTestApp } from '../utils/test-app';
describe('Orders API', () => {
let app;
let request;
beforeAll(async () => {
app = await createTestApp();
request = supertest(app);
});
afterAll(async () => {
await app.close();
});
it('POST /api/orders should create order', async () => {
const response = await request
.post('/api/orders')
.send({
userId: 'user-123',
items: [
{ productId: 'prod-1', quantity: 2, price: 50 },
],
})
.expect(201);
expect(response.body).toMatchObject({
id: expect.any(String),
userId: 'user-123',
status: 'pending',
total: 100,
});
// Verify database persistence
const order = await app.db.orders.findById(response.body.id);
expect(order).toBeTruthy();
});
it('GET /api/orders/:id should return order', async () => {
// Create order first
const createResponse = await request
.post('/api/orders')
.send({ userId: 'user-123', items: [] });
const orderId = createResponse.body.id;
// Fetch order
const response = await request
.get(`/api/orders/${orderId}`)
.expect(200);
expect(response.body.id).toBe(orderId);
});
it('PUT /api/orders/:id/status should update status', async () => {
const createResponse = await request
.post('/api/orders')
.send({ userId: 'user-123', items: [] });
const orderId = createResponse.body.id;
const response = await request
.put(`/api/orders/${orderId}/status`)
.send({ status: 'shipped' })
.expect(200);
expect(response.body.status).toBe('shipped');
});
});
```
**Contract Test (Pact)**:
```typescript
// tests/contract/orders-consumer.test.ts
import { PactV3, MatchersV3 } from '@pact-foundation/pact';
import { OrdersClient } from '@/api/orders-client';
const provider = new PactV3({
consumer: 'OrdersConsumer',
provider: 'OrdersAPI',
});
describe('Orders API Contract', () => {
it('should get order by ID', async () => {
await provider
.given('order with ID 123 exists')
.uponReceiving('a request for order 123')
.withRequest({
method: 'GET',
path: '/api/orders/123',
})
.willRespondWith({
status: 200,
headers: { 'Content-Type': 'application/json' },
body: {
id: MatchersV3.string('123'),
userId: MatchersV3.string('user-456'),
status: MatchersV3.regex('pending|shipped|delivered', 'pending'),
total: MatchersV3.decimal(100.50),
},
})
.executeTest(async (mockServer) => {
const client = new OrdersClient(mockServer.url);
const order = await client.getOrder('123');
expect(order.id).toBe('123');
expect(order.status).toMatch(/pending|shipped|delivered/);
});
});
});
```
### Coverage Targets
- Unit: 90%+ business logic
- Integration: 100% API endpoints
- Contract: 100% API contracts
- E2E: Critical flows only
[Document continues with 6 more comprehensive testing strategies...]
---
## Summary Table
| Strategy | Best For | Coverage Target | Execution Time |
|----------|----------|-----------------|----------------|
| Pyramid | Backend services, traditional apps | 80%+ unit, 100% critical | < 5 min |
| Trophy | Modern frontends (React, Vue) | 80%+ integration | < 3 min |
| TDD | New features, greenfield projects | 90%+ | Continuous |
| BDD | Stakeholder-driven development | 100% acceptance | Variable |
| API | REST/GraphQL services | 90%+ endpoints | < 2 min |
| Frontend | SPAs, component libraries | 85%+ components | < 4 min |
| Micro-Services | Distributed systems | 80%+ per service | < 10 min |
| Performance | High-traffic applications | Critical paths | 30 min |
| Security | Sensitive data, compliance | 100% attack vectors | 1 hour |
| Accessibility | Public-facing websites | WCAG AA 100% | 15 min |

1081
commands/e2e-setup.md Normal file
View File

File diff suppressed because it is too large Load Diff

979
commands/test-coverage.md Normal file
View File

@@ -0,0 +1,979 @@
# /specweave-testing:test-coverage
Comprehensive test coverage analysis, reporting, and quality metrics for modern test suites.
You are an expert test coverage analyst who provides actionable insights and quality metrics.
## Your Task
Analyze test coverage, identify gaps, generate reports, and provide recommendations for improving test quality.
### 1. Coverage Tools Stack
**Vitest Coverage (v8/istanbul)**:
- Line coverage
- Branch coverage
- Function coverage
- Statement coverage
- Per-file analysis
- HTML/LCOV/JSON reports
**Additional Tools**:
- Codecov integration
- SonarQube analysis
- Custom coverage badges
- Historical trending
- Coverage gates
### 2. Vitest Coverage Configuration
**vitest.config.ts** (Comprehensive Coverage):
```typescript
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
coverage: {
provider: 'v8', // or 'istanbul'
reporter: [
'text', // Console output
'text-summary', // Summary in console
'html', // HTML report
'lcov', // LCOV format (for Codecov)
'json', // JSON format
'json-summary', // Summary JSON
'clover', // Clover XML
],
// Files to include
include: ['src/**/*.{ts,tsx,js,jsx}'],
// Files to exclude
exclude: [
'node_modules/',
'dist/',
'build/',
'coverage/',
'**/*.d.ts',
'**/*.config.*',
'**/*.setup.*',
'**/mockData/**',
'**/types/**',
'**/__tests__/**',
'**/*.test.*',
'**/*.spec.*',
],
// Coverage thresholds (fail if below)
thresholds: {
lines: 80,
functions: 80,
branches: 80,
statements: 80,
// Per-file thresholds
perFile: true,
// Auto-update thresholds
autoUpdate: false,
// Threshold enforcement
100: false, // Don't require 100%
},
// Report all files (even untested)
all: true,
// Skip coverage for specific files
ignoreClassMethods: ['toString', 'toJSON'],
// Clean coverage directory before run
clean: true,
// Watermarks for coloring
watermarks: {
statements: [50, 80],
functions: [50, 80],
branches: [50, 80],
lines: [50, 80],
},
},
},
});
```
### 3. Coverage Analysis Script
**scripts/analyze-coverage.ts**:
```typescript
import fs from 'fs';
import path from 'path';
import chalk from 'chalk';
interface FileCoverage {
lines: { total: number; covered: number; skipped: number; pct: number };
statements: { total: number; covered: number; skipped: number; pct: number };
functions: { total: number; covered: number; skipped: number; pct: number };
branches: { total: number; covered: number; skipped: number; pct: number };
}
interface CoverageSummary {
total: FileCoverage;
[file: string]: FileCoverage;
}
export class CoverageAnalyzer {
private coveragePath: string;
private summary: CoverageSummary;
constructor(coveragePath = 'coverage/coverage-summary.json') {
this.coveragePath = coveragePath;
this.summary = JSON.parse(fs.readFileSync(coveragePath, 'utf-8'));
}
// Overall coverage summary
printSummary() {
const { total } = this.summary;
console.log(chalk.bold('\n📊 Coverage Summary\n'));
console.log(this.formatCoverageLine('Lines', total.lines));
console.log(this.formatCoverageLine('Statements', total.statements));
console.log(this.formatCoverageLine('Functions', total.functions));
console.log(this.formatCoverageLine('Branches', total.branches));
}
// Files with low coverage
findLowCoverageFiles(threshold = 80) {
const lowCoverageFiles: Array<{
file: string;
type: string;
coverage: number;
gap: number;
}> = [];
Object.entries(this.summary).forEach(([file, data]) => {
if (file === 'total') return;
const checks = [
{ type: 'lines', pct: data.lines.pct },
{ type: 'functions', pct: data.functions.pct },
{ type: 'branches', pct: data.branches.pct },
{ type: 'statements', pct: data.statements.pct },
];
checks.forEach(({ type, pct }) => {
if (pct < threshold) {
lowCoverageFiles.push({
file: this.shortenPath(file),
type,
coverage: pct,
gap: threshold - pct,
});
}
});
});
return lowCoverageFiles.sort((a, b) => b.gap - a.gap);
}
// Uncovered files
findUncoveredFiles() {
const uncovered: string[] = [];
Object.entries(this.summary).forEach(([file, data]) => {
if (file === 'total') return;
if (data.statements.pct === 0) {
uncovered.push(this.shortenPath(file));
}
});
return uncovered;
}
// Files with perfect coverage
findPerfectCoverage() {
const perfect: string[] = [];
Object.entries(this.summary).forEach(([file, data]) => {
if (file === 'total') return;
const isPerfect =
data.lines.pct === 100 &&
data.functions.pct === 100 &&
data.branches.pct === 100 &&
data.statements.pct === 100;
if (isPerfect) {
perfect.push(this.shortenPath(file));
}
});
return perfect;
}
// Coverage trends (requires historical data)
analyzeTrends(previousCoveragePath: string) {
const previousSummary: CoverageSummary = JSON.parse(
fs.readFileSync(previousCoveragePath, 'utf-8')
);
const trends = {
lines: this.summary.total.lines.pct - previousSummary.total.lines.pct,
statements: this.summary.total.statements.pct - previousSummary.total.statements.pct,
functions: this.summary.total.functions.pct - previousSummary.total.functions.pct,
branches: this.summary.total.branches.pct - previousSummary.total.branches.pct,
};
console.log(chalk.bold('\n📈 Coverage Trends\n'));
Object.entries(trends).forEach(([type, change]) => {
const arrow = change > 0 ? '↗' : change < 0 ? '↘' : '→';
const color = change > 0 ? chalk.green : change < 0 ? chalk.red : chalk.gray;
console.log(
`${type.padEnd(12)} ${color(arrow)} ${color(`${change > 0 ? '+' : ''}${change.toFixed(2)}%`)}`
);
});
}
// Generate coverage report
generateReport() {
const lowCoverage = this.findLowCoverageFiles();
const uncovered = this.findUncoveredFiles();
const perfect = this.findPerfectCoverage();
console.log(chalk.bold('\n🎯 Coverage Analysis Report\n'));
// Summary
this.printSummary();
// Perfect coverage
if (perfect.length > 0) {
console.log(chalk.bold.green(`\n✓ Perfect Coverage (${perfect.length} files)`));
perfect.slice(0, 5).forEach((file) => {
console.log(chalk.green(` • ${file}`));
});
if (perfect.length > 5) {
console.log(chalk.gray(` ... and ${perfect.length - 5} more`));
}
}
// Uncovered files
if (uncovered.length > 0) {
console.log(chalk.bold.red(`\n✗ Uncovered Files (${uncovered.length})`));
uncovered.slice(0, 10).forEach((file) => {
console.log(chalk.red(` • ${file}`));
});
if (uncovered.length > 10) {
console.log(chalk.gray(` ... and ${uncovered.length - 10} more`));
}
}
// Low coverage files
if (lowCoverage.length > 0) {
console.log(chalk.bold.yellow(`\n⚠ Low Coverage Areas (${lowCoverage.length})`));
lowCoverage.slice(0, 10).forEach(({ file, type, coverage, gap }) => {
console.log(
chalk.yellow(
` • ${file} - ${type}: ${coverage.toFixed(1)}% (gap: ${gap.toFixed(1)}%)`
)
);
});
if (lowCoverage.length > 10) {
console.log(chalk.gray(` ... and ${lowCoverage.length - 10} more`));
}
}
// Recommendations
this.printRecommendations(lowCoverage, uncovered);
}
// Recommendations
private printRecommendations(lowCoverage: any[], uncovered: string[]) {
console.log(chalk.bold('\n💡 Recommendations\n'));
if (uncovered.length > 0) {
console.log(chalk.yellow('1. Add tests for uncovered files'));
console.log(` Priority: ${uncovered.slice(0, 3).join(', ')}`);
}
if (lowCoverage.length > 0) {
const topGap = lowCoverage[0];
console.log(chalk.yellow(`2. Improve ${topGap.type} coverage in ${topGap.file}`));
console.log(` Current: ${topGap.coverage.toFixed(1)}%, Target: 80%`);
}
const { total } = this.summary;
if (total.branches.pct < 80) {
console.log(chalk.yellow('3. Focus on branch coverage'));
console.log(' Add tests for conditional logic and edge cases');
}
if (total.functions.pct < 80) {
console.log(chalk.yellow('4. Increase function coverage'));
console.log(' Test all exported functions and methods');
}
}
// Helpers
private formatCoverageLine(label: string, data: FileCoverage['lines']) {
const percentage = data.pct.toFixed(2);
const color =
data.pct >= 80 ? chalk.green : data.pct >= 50 ? chalk.yellow : chalk.red;
return `${label.padEnd(12)} ${color(percentage.padStart(6))}% ${chalk.gray(
`(${data.covered}/${data.total})`
)}`;
}
private shortenPath(file: string) {
return file.replace(process.cwd(), '').replace(/^\//, '');
}
}
// CLI usage
if (require.main === module) {
const analyzer = new CoverageAnalyzer();
analyzer.generateReport();
}
```
### 4. Coverage Badge Generation
**scripts/generate-coverage-badge.ts**:
```typescript
import fs from 'fs';
import path from 'path';
interface CoverageSummary {
total: {
lines: { pct: number };
};
}
export function generateCoverageBadge() {
const summary: CoverageSummary = JSON.parse(
fs.readFileSync('coverage/coverage-summary.json', 'utf-8')
);
const coverage = summary.total.lines.pct;
const color = coverage >= 80 ? 'brightgreen' : coverage >= 50 ? 'yellow' : 'red';
const badge = `https://img.shields.io/badge/coverage-${coverage.toFixed(0)}%25-${color}`;
// Update README.md
const readme = fs.readFileSync('README.md', 'utf-8');
const updatedReadme = readme.replace(
/!\[Coverage\]\(.*?\)/,
`![Coverage](${badge})`
);
fs.writeFileSync('README.md', updatedReadme);
console.log(`✓ Updated coverage badge: ${coverage.toFixed(2)}%`);
}
```
### 5. Coverage Gates
**scripts/enforce-coverage-gates.ts**:
```typescript
import fs from 'fs';
import chalk from 'chalk';
interface CoverageThresholds {
lines: number;
statements: number;
functions: number;
branches: number;
}
export class CoverageGate {
private summary: any;
private thresholds: CoverageThresholds;
constructor(
summaryPath = 'coverage/coverage-summary.json',
thresholds: CoverageThresholds = {
lines: 80,
statements: 80,
functions: 80,
branches: 80,
}
) {
this.summary = JSON.parse(fs.readFileSync(summaryPath, 'utf-8'));
this.thresholds = thresholds;
}
enforce(): boolean {
const { total } = this.summary;
const failures: string[] = [];
console.log(chalk.bold('\n🚦 Coverage Gate Enforcement\n'));
Object.entries(this.thresholds).forEach(([metric, threshold]) => {
const actual = total[metric].pct;
const passed = actual >= threshold;
const status = passed ? chalk.green('✓') : chalk.red('✗');
const color = passed ? chalk.green : chalk.red;
console.log(
`${status} ${metric.padEnd(12)} ${color(
`${actual.toFixed(2)}%`
)} ${chalk.gray(`(threshold: ${threshold}%)`)}`
);
if (!passed) {
failures.push(`${metric}: ${actual.toFixed(2)}% < ${threshold}%`);
}
});
if (failures.length > 0) {
console.log(chalk.bold.red('\n✗ Coverage gate failed!'));
console.log(chalk.red('\nFailures:'));
failures.forEach((failure) => console.log(chalk.red(` • ${failure}`)));
return false;
}
console.log(chalk.bold.green('\n✓ Coverage gate passed!'));
return true;
}
}
// CLI usage
if (require.main === module) {
const gate = new CoverageGate();
const passed = gate.enforce();
process.exit(passed ? 0 : 1);
}
```
### 6. Diff Coverage Analysis
**scripts/analyze-diff-coverage.ts**:
```typescript
import { execSync } from 'child_process';
import fs from 'fs';
import chalk from 'chalk';
export class DiffCoverageAnalyzer {
private baseBranch: string;
constructor(baseBranch = 'main') {
this.baseBranch = baseBranch;
}
// Get changed files
getChangedFiles(): string[] {
const output = execSync(`git diff ${this.baseBranch} --name-only`, {
encoding: 'utf-8',
});
return output
.split('\n')
.filter((file) => file.endsWith('.ts') || file.endsWith('.tsx'))
.filter((file) => file.startsWith('src/'));
}
// Get coverage for changed files
analyzeDiffCoverage() {
const changedFiles = this.getChangedFiles();
const coverage = JSON.parse(
fs.readFileSync('coverage/coverage-summary.json', 'utf-8')
);
console.log(chalk.bold('\n📝 Diff Coverage Analysis\n'));
console.log(chalk.gray(`Base branch: ${this.baseBranch}\n`));
const results = changedFiles.map((file) => {
const fullPath = `${process.cwd()}/${file}`;
const fileCoverage = coverage[fullPath];
if (!fileCoverage) {
return {
file,
covered: false,
lines: 0,
statements: 0,
functions: 0,
branches: 0,
};
}
return {
file,
covered: true,
lines: fileCoverage.lines.pct,
statements: fileCoverage.statements.pct,
functions: fileCoverage.functions.pct,
branches: fileCoverage.branches.pct,
};
});
// Print results
results.forEach((result) => {
if (!result.covered) {
console.log(chalk.red(`✗ ${result.file} - No coverage`));
} else {
const avgCoverage =
(result.lines + result.statements + result.functions + result.branches) / 4;
const color = avgCoverage >= 80 ? chalk.green : chalk.yellow;
console.log(
`${color('✓')} ${result.file} - ${color(`${avgCoverage.toFixed(1)}%`)}`
);
console.log(
chalk.gray(
` Lines: ${result.lines.toFixed(1)}%, Functions: ${result.functions.toFixed(1)}%, Branches: ${result.branches.toFixed(1)}%`
)
);
}
});
// Summary
const uncovered = results.filter((r) => !r.covered).length;
const lowCoverage = results.filter(
(r) =>
r.covered &&
(r.lines < 80 || r.statements < 80 || r.functions < 80 || r.branches < 80)
).length;
console.log(chalk.bold('\n📊 Diff Coverage Summary\n'));
console.log(`Total changed files: ${results.length}`);
console.log(chalk.red(`Uncovered: ${uncovered}`));
console.log(chalk.yellow(`Low coverage: ${lowCoverage}`));
console.log(
chalk.green(`Good coverage: ${results.length - uncovered - lowCoverage}`)
);
return uncovered === 0 && lowCoverage === 0;
}
}
// CLI usage
if (require.main === module) {
const analyzer = new DiffCoverageAnalyzer();
const passed = analyzer.analyzeDiffCoverage();
process.exit(passed ? 0 : 1);
}
```
### 7. Uncovered Lines Reporter
**scripts/report-uncovered-lines.ts**:
```typescript
import fs from 'fs';
import chalk from 'chalk';
interface UncoveredRange {
start: number;
end: number;
}
interface FileCoverageDetail {
path: string;
statementMap: Record<string, any>;
s: Record<string, number>;
branchMap: Record<string, any>;
b: Record<string, number[]>;
fnMap: Record<string, any>;
f: Record<string, number>;
}
export class UncoveredLinesReporter {
private coverageData: Record<string, FileCoverageDetail>;
constructor(coveragePath = 'coverage/coverage-final.json') {
this.coverageData = JSON.parse(fs.readFileSync(coveragePath, 'utf-8'));
}
reportUncoveredLines(targetFile?: string) {
const files = targetFile
? [targetFile]
: Object.keys(this.coverageData).filter((f) => !f.includes('node_modules'));
console.log(chalk.bold('\n🔍 Uncovered Lines Report\n'));
files.forEach((file) => {
const coverage = this.coverageData[file];
if (!coverage) return;
const uncoveredStatements = this.getUncoveredStatements(coverage);
const uncoveredBranches = this.getUncoveredBranches(coverage);
const uncoveredFunctions = this.getUncoveredFunctions(coverage);
if (
uncoveredStatements.length === 0 &&
uncoveredBranches.length === 0 &&
uncoveredFunctions.length === 0
) {
return; // Skip files with perfect coverage
}
console.log(chalk.bold(this.shortenPath(file)));
if (uncoveredStatements.length > 0) {
console.log(chalk.yellow(' Uncovered statements:'));
uncoveredStatements.forEach((range) => {
console.log(chalk.gray(` Lines ${range.start}-${range.end}`));
});
}
if (uncoveredBranches.length > 0) {
console.log(chalk.yellow(' Uncovered branches:'));
uncoveredBranches.forEach(({ line, type }) => {
console.log(chalk.gray(` Line ${line} (${type})`));
});
}
if (uncoveredFunctions.length > 0) {
console.log(chalk.yellow(' Uncovered functions:'));
uncoveredFunctions.forEach(({ name, line }) => {
console.log(chalk.gray(` ${name} (line ${line})`));
});
}
console.log();
});
}
private getUncoveredStatements(coverage: FileCoverageDetail): UncoveredRange[] {
const uncovered: UncoveredRange[] = [];
Object.entries(coverage.s).forEach(([key, count]) => {
if (count === 0) {
const statement = coverage.statementMap[key];
uncovered.push({
start: statement.start.line,
end: statement.end.line,
});
}
});
return uncovered;
}
private getUncoveredBranches(coverage: FileCoverageDetail) {
const uncovered: Array<{ line: number; type: string }> = [];
Object.entries(coverage.b).forEach(([key, branches]) => {
const branch = coverage.branchMap[key];
branches.forEach((count, idx) => {
if (count === 0) {
uncovered.push({
line: branch.loc.start.line,
type: branch.type,
});
}
});
});
return uncovered;
}
private getUncoveredFunctions(coverage: FileCoverageDetail) {
const uncovered: Array<{ name: string; line: number }> = [];
Object.entries(coverage.f).forEach(([key, count]) => {
if (count === 0) {
const fn = coverage.fnMap[key];
uncovered.push({
name: fn.name || '(anonymous)',
line: fn.loc.start.line,
});
}
});
return uncovered;
}
private shortenPath(file: string) {
return file.replace(process.cwd(), '').replace(/^\//, '');
}
}
// CLI usage
if (require.main === module) {
const reporter = new UncoveredLinesReporter();
reporter.reportUncoveredLines();
}
```
### 8. Coverage Comparison Tool
**scripts/compare-coverage.ts**:
```typescript
import fs from 'fs';
import chalk from 'chalk';
interface CoverageSummary {
total: {
lines: { pct: number };
statements: { pct: number };
functions: { pct: number };
branches: { pct: number };
};
}
export function compareCoverage(
beforePath: string,
afterPath: string = 'coverage/coverage-summary.json'
) {
const before: CoverageSummary = JSON.parse(fs.readFileSync(beforePath, 'utf-8'));
const after: CoverageSummary = JSON.parse(fs.readFileSync(afterPath, 'utf-8'));
console.log(chalk.bold('\n📊 Coverage Comparison\n'));
const metrics = ['lines', 'statements', 'functions', 'branches'] as const;
metrics.forEach((metric) => {
const beforePct = before.total[metric].pct;
const afterPct = after.total[metric].pct;
const diff = afterPct - beforePct;
const arrow = diff > 0 ? '↗' : diff < 0 ? '↘' : '→';
const color = diff > 0 ? chalk.green : diff < 0 ? chalk.red : chalk.gray;
console.log(
`${metric.padEnd(12)} ${beforePct.toFixed(2)}% ${arrow} ${afterPct.toFixed(2)}% ${color(
`(${diff > 0 ? '+' : ''}${diff.toFixed(2)}%)`
)}`
);
});
// Recommendation
if (after.total.lines.pct < before.total.lines.pct) {
console.log(chalk.bold.red('\n⚠ Coverage decreased! Consider adding tests.'));
} else if (after.total.lines.pct > before.total.lines.pct) {
console.log(chalk.bold.green('\n✓ Coverage improved!'));
} else {
console.log(chalk.gray('\n→ Coverage unchanged.'));
}
}
```
### 9. CI/CD Integration
**GitHub Actions (.github/workflows/coverage.yml)**:
```yaml
name: Coverage
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Needed for diff coverage
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run tests with coverage
run: npm run test:coverage
- name: Analyze coverage
run: npm run coverage:analyze
- name: Enforce coverage gates
run: npm run coverage:gate
- name: Analyze diff coverage
if: github.event_name == 'pull_request'
run: npm run coverage:diff
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:
files: ./coverage/lcov.info
flags: unittests
name: codecov-umbrella
fail_ci_if_error: true
- name: Upload coverage reports
uses: actions/upload-artifact@v3
with:
name: coverage-report
path: |
coverage/
!coverage/**/*.json
- name: Comment PR with coverage
if: github.event_name == 'pull_request'
uses: romeovs/lcov-reporter-action@v0.3.1
with:
lcov-file: ./coverage/lcov.info
github-token: ${{ secrets.GITHUB_TOKEN }}
delete-old-comments: true
- name: Generate coverage badge
if: github.ref == 'refs/heads/main'
run: npm run coverage:badge
```
### 10. Package Scripts
```json
{
"scripts": {
"test:coverage": "vitest run --coverage",
"coverage:analyze": "tsx scripts/analyze-coverage.ts",
"coverage:gate": "tsx scripts/enforce-coverage-gates.ts",
"coverage:diff": "tsx scripts/analyze-diff-coverage.ts",
"coverage:uncovered": "tsx scripts/report-uncovered-lines.ts",
"coverage:compare": "tsx scripts/compare-coverage.ts",
"coverage:badge": "tsx scripts/generate-coverage-badge.ts",
"coverage:report": "npm run test:coverage && npm run coverage:analyze",
"coverage:html": "open coverage/index.html"
},
"devDependencies": {
"@vitest/coverage-v8": "^1.0.4",
"chalk": "^5.3.0",
"tsx": "^4.7.0"
}
}
```
### 11. SonarQube Integration
**sonar-project.properties**:
```properties
sonar.projectKey=my-project
sonar.projectName=My Project
sonar.projectVersion=1.0.0
# Source
sonar.sources=src
sonar.tests=tests
# Language
sonar.language=ts
sonar.sourceEncoding=UTF-8
# Coverage
sonar.javascript.lcov.reportPaths=coverage/lcov.info
sonar.coverage.exclusions=**/*.test.ts,**/*.spec.ts,**/mockData/**
# Quality gates
sonar.qualitygate.wait=true
```
### 12. Coverage Dashboard HTML
**scripts/generate-coverage-dashboard.ts**:
```typescript
import fs from 'fs';
export function generateDashboard() {
const summary = JSON.parse(
fs.readFileSync('coverage/coverage-summary.json', 'utf-8')
);
const html = `
<!DOCTYPE html>
<html>
<head>
<title>Coverage Dashboard</title>
<style>
body { font-family: system-ui; padding: 20px; max-width: 1200px; margin: 0 auto; }
.metric { display: inline-block; margin: 10px; padding: 20px; border-radius: 8px; }
.high { background: #d4edda; }
.medium { background: #fff3cd; }
.low { background: #f8d7da; }
h1 { color: #333; }
.percentage { font-size: 2em; font-weight: bold; }
</style>
</head>
<body>
<h1>Test Coverage Dashboard</h1>
<div class="metrics">
${generateMetricCard('Lines', summary.total.lines.pct)}
${generateMetricCard('Statements', summary.total.statements.pct)}
${generateMetricCard('Functions', summary.total.functions.pct)}
${generateMetricCard('Branches', summary.total.branches.pct)}
</div>
</body>
</html>
`;
fs.writeFileSync('coverage/dashboard.html', html);
}
function generateMetricCard(name: string, pct: number) {
const className = pct >= 80 ? 'high' : pct >= 50 ? 'medium' : 'low';
return `
<div class="metric ${className}">
<div>${name}</div>
<div class="percentage">${pct.toFixed(1)}%</div>
</div>
`;
}
```
### 13. Best Practices
**Coverage Goals**:
- 80%+ overall coverage (all metrics)
- 90%+ for critical paths
- 100% for utility functions
- No uncovered files
**What to Focus On**:
- Business logic (highest priority)
- Error handling
- Edge cases
- Conditional branches
- Integration points
**What to Skip**:
- Type definitions
- Configuration files
- Mock data
- Test utilities
- Build artifacts
**Quality over Quantity**:
- Meaningful tests > high coverage
- Test behavior, not implementation
- Focus on user journeys
- Verify error scenarios
- Check accessibility
## Workflow
1. Ask about coverage requirements and current state
2. Run coverage analysis with Vitest
3. Generate comprehensive coverage report
4. Identify low coverage areas and gaps
5. Analyze diff coverage for PRs
6. Report uncovered lines with specific locations
7. Enforce coverage gates
8. Generate badges and dashboards
9. Set up CI/CD integration
10. Provide actionable recommendations
11. Track coverage trends over time
## When to Use
- Checking current test coverage
- Identifying coverage gaps
- Enforcing coverage standards
- Setting up CI/CD coverage gates
- Analyzing coverage trends
- Generating coverage reports
- Creating coverage dashboards
- Improving code quality
Achieve comprehensive test coverage with actionable insights!

1156
commands/test-generate.md Normal file
View File

File diff suppressed because it is too large Load Diff

409
commands/test-init.md Normal file
View File

@@ -0,0 +1,409 @@
# /specweave-testing:test-init
Initialize comprehensive testing infrastructure with Vitest, Playwright, and testing best practices.
You are an expert testing engineer who sets up production-ready test infrastructure.
## Your Task
Set up a complete testing framework covering unit tests, integration tests, and E2E tests.
### 1. Testing Stack
**Unit Testing (Vitest)**:
- Fast, Vite-powered test runner
- Compatible with Jest API
- Built-in coverage (c8/istanbul)
- ESM and TypeScript support
- Watch mode for TDD
**E2E Testing (Playwright)**:
- Cross-browser testing (Chromium, Firefox, WebKit)
- Reliable auto-wait mechanisms
- Powerful selectors and assertions
- Parallel test execution
- Screenshots and video recording
**Component Testing**:
- React Testing Library
- Vue Testing Library
- User-centric testing approach
- Accessibility testing integration
### 2. Vitest Configuration
**vitest.config.ts**:
```typescript
import { defineConfig } from 'vitest/config';
import react from '@vitejs/plugin-react';
import path from 'path';
export default defineConfig({
plugins: [react()],
test: {
globals: true,
environment: 'jsdom',
setupFiles: ['./tests/setup.ts'],
include: ['**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}'],
exclude: ['node_modules', 'dist', '.idea', '.git', '.cache'],
coverage: {
provider: 'v8',
reporter: ['text', 'json', 'html', 'lcov'],
exclude: [
'node_modules/',
'tests/',
'**/*.d.ts',
'**/*.config.*',
'**/mockData',
],
all: true,
lines: 80,
functions: 80,
branches: 80,
statements: 80,
},
testTimeout: 10000,
},
resolve: {
alias: {
'@': path.resolve(__dirname, './src'),
},
},
});
```
**tests/setup.ts**:
```typescript
import '@testing-library/jest-dom';
import { cleanup } from '@testing-library/react';
import { afterEach, vi } from 'vitest';
// Cleanup after each test
afterEach(() => {
cleanup();
});
// Mock window.matchMedia
Object.defineProperty(window, 'matchMedia', {
writable: true,
value: vi.fn().mockImplementation((query) => ({
matches: false,
media: query,
onchange: null,
addListener: vi.fn(),
removeListener: vi.fn(),
addEventListener: vi.fn(),
removeEventListener: vi.fn(),
dispatchEvent: vi.fn(),
})),
});
// Mock IntersectionObserver
global.IntersectionObserver = class IntersectionObserver {
constructor() {}
disconnect() {}
observe() {}
takeRecords() {
return [];
}
unobserve() {}
} as any;
```
### 3. Playwright Configuration
**playwright.config.ts**:
```typescript
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests/e2e',
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: 'html',
use: {
baseURL: 'http://localhost:3000',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
video: 'retain-on-failure',
},
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
// Mobile viewports
{
name: 'Mobile Chrome',
use: { ...devices['Pixel 5'] },
},
{
name: 'Mobile Safari',
use: { ...devices['iPhone 12'] },
},
],
webServer: {
command: 'npm run dev',
url: 'http://localhost:3000',
reuseExistingServer: !process.env.CI,
},
});
```
### 4. Test Utilities
**tests/utils/test-utils.tsx** (React):
```typescript
import { render, RenderOptions } from '@testing-library/react';
import { ReactElement } from 'react';
import { BrowserRouter } from 'react-router-dom';
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
const queryClient = new QueryClient({
defaultOptions: {
queries: { retry: false },
mutations: { retry: false },
},
});
function AllTheProviders({ children }: { children: React.ReactNode }) {
return (
<QueryClientProvider client={queryClient}>
<BrowserRouter>
{children}
</BrowserRouter>
</QueryClientProvider>
);
}
function customRender(ui: ReactElement, options?: Omit<RenderOptions, 'wrapper'>) {
return render(ui, { wrapper: AllTheProviders, ...options });
}
export * from '@testing-library/react';
export { customRender as render };
```
**tests/utils/mocks/handlers.ts** (MSW):
```typescript
import { http, HttpResponse } from 'msw';
export const handlers = [
http.get('/api/users', () => {
return HttpResponse.json([
{ id: '1', name: 'John Doe' },
{ id: '2', name: 'Jane Smith' },
]);
}),
http.post('/api/login', async ({ request }) => {
const { email, password } = await request.json();
if (email === 'test@example.com' && password === 'password') {
return HttpResponse.json({
token: 'mock-jwt-token',
user: { id: '1', email },
});
}
return HttpResponse.json(
{ error: 'Invalid credentials' },
{ status: 401 }
);
}),
];
```
**tests/utils/mocks/server.ts**:
```typescript
import { setupServer } from 'msw/node';
import { handlers } from './handlers';
export const server = setupServer(...handlers);
```
### 5. Package Dependencies
```json
{
"devDependencies": {
"@playwright/test": "^1.40.0",
"@testing-library/jest-dom": "^6.1.5",
"@testing-library/react": "^14.1.2",
"@testing-library/user-event": "^14.5.1",
"@vitest/coverage-v8": "^1.0.4",
"@vitest/ui": "^1.0.4",
"jsdom": "^23.0.1",
"msw": "^2.0.0",
"vitest": "^1.0.4"
}
}
```
### 6. NPM Scripts
```json
{
"scripts": {
"test": "vitest",
"test:ui": "vitest --ui",
"test:coverage": "vitest --coverage",
"test:e2e": "playwright test",
"test:e2e:ui": "playwright test --ui",
"test:e2e:debug": "playwright test --debug",
"test:all": "npm run test:coverage && npm run test:e2e"
}
}
```
### 7. CI/CD Configuration
**GitHub Actions (.github/workflows/test.yml)**:
```yaml
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run unit tests
run: npm run test:coverage
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Run E2E tests
run: npm run test:e2e
- name: Upload coverage
uses: codecov/codecov-action@v3
with:
files: ./coverage/lcov.info
- name: Upload test results
if: always()
uses: actions/upload-artifact@v3
with:
name: playwright-report
path: playwright-report/
```
### 8. Testing Best Practices
**Unit Test Example**:
```typescript
import { render, screen, fireEvent } from './utils/test-utils';
import { LoginForm } from '@/components/LoginForm';
describe('LoginForm', () => {
it('renders login form correctly', () => {
render(<LoginForm />);
expect(screen.getByLabelText(/email/i)).toBeInTheDocument();
expect(screen.getByLabelText(/password/i)).toBeInTheDocument();
expect(screen.getByRole('button', { name: /login/i })).toBeInTheDocument();
});
it('shows validation errors for empty fields', async () => {
render(<LoginForm />);
fireEvent.click(screen.getByRole('button', { name: /login/i }));
expect(await screen.findByText(/email is required/i)).toBeInTheDocument();
expect(await screen.findByText(/password is required/i)).toBeInTheDocument();
});
it('submits form with valid data', async () => {
const onSubmit = vi.fn();
render(<LoginForm onSubmit={onSubmit} />);
fireEvent.change(screen.getByLabelText(/email/i), {
target: { value: 'test@example.com' },
});
fireEvent.change(screen.getByLabelText(/password/i), {
target: { value: 'password123' },
});
fireEvent.click(screen.getByRole('button', { name: /login/i }));
await waitFor(() => {
expect(onSubmit).toHaveBeenCalledWith({
email: 'test@example.com',
password: 'password123',
});
});
});
});
```
**E2E Test Example**:
```typescript
import { test, expect } from '@playwright/test';
test.describe('Authentication Flow', () => {
test('should allow user to login', async ({ page }) => {
await page.goto('/login');
await page.fill('input[name="email"]', 'test@example.com');
await page.fill('input[name="password"]', 'password123');
await page.click('button[type="submit"]');
await expect(page).toHaveURL('/dashboard');
await expect(page.locator('h1')).toContainText('Dashboard');
});
test('should show error for invalid credentials', async ({ page }) => {
await page.goto('/login');
await page.fill('input[name="email"]', 'wrong@example.com');
await page.fill('input[name="password"]', 'wrongpassword');
await page.click('button[type="submit"]');
await expect(page.locator('[role="alert"]')).toContainText(
'Invalid credentials'
);
});
});
```
## Workflow
1. Ask about testing requirements and existing setup
2. Install testing dependencies (Vitest, Playwright, Testing Library)
3. Create Vitest configuration
4. Create Playwright configuration
5. Set up test utilities and helpers
6. Configure MSW for API mocking
7. Add test scripts to package.json
8. Create example tests
9. Set up CI/CD workflow
10. Provide testing guidelines and best practices
## When to Use
- Starting new projects with testing
- Migrating from Jest to Vitest
- Adding E2E testing to existing projects
- Setting up CI/CD testing pipeline
- Improving test coverage
- Implementing TDD workflow
Initialize production-ready testing infrastructure with modern tools!

93
plugin.lock.json Normal file
View File

@@ -0,0 +1,93 @@
{
"$schema": "internal://schemas/plugin.lock.v1.json",
"pluginId": "gh:anton-abyzov/specweave:plugins/specweave-testing",
"normalized": {
"repo": null,
"ref": "refs/tags/v20251128.0",
"commit": "c8de26e77e446b620ab5e00c02813964463314db",
"treeHash": "118fe82de0f25d768d3f4974f9ae13081be959fc40f2b3adfa7deff79a7994ec",
"generatedAt": "2025-11-28T10:13:53.752945Z",
"toolVersion": "publish_plugins.py@0.2.0"
},
"origin": {
"remote": "git@github.com:zhongweili/42plugin-data.git",
"branch": "master",
"commit": "aa1497ed0949fd50e99e70d6324a29c5b34f9390",
"repoRoot": "/Users/zhongweili/projects/openmind/42plugin-data"
},
"manifest": {
"name": "specweave-testing",
"description": "Comprehensive testing tools for modern web applications. Includes Playwright E2E testing, Vitest unit testing, test generation, and coverage analysis. Focus on test-driven development and quality assurance.",
"version": "0.24.0"
},
"content": {
"files": [
{
"path": "README.md",
"sha256": "89da915c6556480d1e74c92d8ab118555fdae3375c6eab80dc7ac3b6251ec17d"
},
{
"path": "agents/qa-engineer/AGENT.md",
"sha256": "76bb0a1b5edf1d3b418e960a47c130afe1bcbdf76c69d479e95ec816ab264bc7"
},
{
"path": "agents/qa-engineer/test-strategies.md",
"sha256": "1f17a955fe06f3edfbb00f11224dc3ee60d5ca75f07f9de5d468011f57d24399"
},
{
"path": "agents/qa-engineer/README.md",
"sha256": "a31b49a047a0524ff0eb26092d9dfdc25d6f982a91ec95915f5c58df714ae065"
},
{
"path": "agents/qa-engineer/templates/vitest-unit-test.ts",
"sha256": "7c1fcf6bd837ddd3601c2967cfdfcf92ebc9696104d2c2c4d134170248f173de"
},
{
"path": "agents/qa-engineer/templates/playwright-e2e-test.ts",
"sha256": "6afd11822f6826befd236217b5d006421edf9f8c615f5d5c227cdf9fe3430bce"
},
{
"path": "agents/qa-engineer/templates/test-data-factory.ts",
"sha256": "ea86f5b366b9e87ba62588ef81b382169af294a57088111b01e1587cbbf0e6e4"
},
{
"path": ".claude-plugin/plugin.json",
"sha256": "9c0e285c63b72808625b20d9bb6e94da7b3d8b0c42974a2579dbbc49315ce929"
},
{
"path": "commands/e2e-setup.md",
"sha256": "48e2637e3da9c3d91de9948458b261f96b80fdc82705def79a118f885bc77417"
},
{
"path": "commands/test-init.md",
"sha256": "ec81224a620a64112d1cc05b224d98bb4fa194f50dfb3b27c9ebae5093841460"
},
{
"path": "commands/test-generate.md",
"sha256": "7f2f48240dedf6e0fcaaacd1b460bfb7f223424c6d348e491d5fc565273bfe16"
},
{
"path": "commands/test-coverage.md",
"sha256": "0ecfd14db6c43b71d4dce722570ed9c02e7b6a8d8adf470a7c2bf0f7d73322b6"
},
{
"path": "skills/tdd-expert/SKILL.md",
"sha256": "53aac6fe1e71f26e6558fdc243926284ebc6861db7c0390c40ca142ac342633b"
},
{
"path": "skills/e2e-playwright/SKILL.md",
"sha256": "f36b83a4cfa77e70fd2e397af9bc7e6a6016c207abf7f1e061ea1efde019381b"
},
{
"path": "skills/unit-testing-expert/SKILL.md",
"sha256": "92c284b6d3c099d937ef3cf1ee456b1499cb08a84b3600df5ffc01ea4c23e775"
}
],
"dirSha256": "118fe82de0f25d768d3f4974f9ae13081be959fc40f2b3adfa7deff79a7994ec"
},
"security": {
"scannedAt": null,
"scannerVersion": null,
"flags": []
}
}

769
skills/e2e-playwright/SKILL.md Normal file
View File

@@ -0,0 +1,769 @@
---
name: e2e-playwright
description: Comprehensive Playwright end-to-end testing expertise covering browser automation, cross-browser testing, visual regression, API testing, mobile emulation, accessibility testing, test architecture, page object models, fixtures, parallel execution, CI/CD integration, debugging strategies, and production-grade E2E test patterns. Activates for playwright, e2e testing, end-to-end testing, browser automation, cross-browser testing, visual testing, screenshot testing, API testing, mobile testing, accessibility testing, test fixtures, page object model, POM, test architecture, parallel testing, playwright config, trace viewer, codegen, test debugging, flaky tests, CI integration, playwright best practices.
---
# E2E Playwright Testing Expert
## Core Expertise
### 1. Playwright Fundamentals
**Browser Automation**:
- Multi-browser support (Chromium, Firefox, WebKit)
- Context isolation and parallel execution
- Auto-waiting and actionability checks
- Network interception and mocking
- File downloads and uploads
- Geolocation and permissions
- Authentication state management
**Test Structure**:
```typescript
import { test, expect } from '@playwright/test';
test.describe('Authentication Flow', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/login');
});
test('should login successfully', async ({ page }) => {
await page.getByLabel('Email').fill('user@example.com');
await page.getByLabel('Password').fill('password123');
await page.getByRole('button', { name: 'Login' }).click();
await expect(page).toHaveURL('/dashboard');
await expect(page.getByText('Welcome back')).toBeVisible();
});
test('should show validation errors', async ({ page }) => {
await page.getByRole('button', { name: 'Login' }).click();
await expect(page.getByText('Email is required')).toBeVisible();
await expect(page.getByText('Password is required')).toBeVisible();
});
});
```
### 2. Page Object Model (POM)
**Pattern**: Encapsulate page interactions for maintainability
```typescript
// pages/LoginPage.ts
import { Page, Locator } from '@playwright/test';
export class LoginPage {
readonly page: Page;
readonly emailInput: Locator;
readonly passwordInput: Locator;
readonly loginButton: Locator;
readonly errorMessage: Locator;
constructor(page: Page) {
this.page = page;
this.emailInput = page.getByLabel('Email');
this.passwordInput = page.getByLabel('Password');
this.loginButton = page.getByRole('button', { name: 'Login' });
this.errorMessage = page.getByRole('alert');
}
async goto() {
await this.page.goto('/login');
}
async login(email: string, password: string) {
await this.emailInput.fill(email);
await this.passwordInput.fill(password);
await this.loginButton.click();
}
async loginWithGoogle() {
await this.page.getByRole('button', { name: 'Continue with Google' }).click();
// Handle OAuth popup
}
async expectError(message: string) {
await expect(this.errorMessage).toContainText(message);
}
}
// Usage in tests
test('login flow', async ({ page }) => {
const loginPage = new LoginPage(page);
await loginPage.goto();
await loginPage.login('user@example.com', 'password123');
await expect(page).toHaveURL('/dashboard');
});
```
### 3. Test Fixtures & Custom Contexts
**Fixtures**: Reusable setup/teardown logic
```typescript
// fixtures/auth.fixture.ts
import { test as base } from '@playwright/test';
import { LoginPage } from '../pages/LoginPage';
type AuthFixtures = {
authenticatedPage: Page;
loginPage: LoginPage;
};
export const test = base.extend<AuthFixtures>({
authenticatedPage: async ({ page }, use) => {
// Setup: Login before test
await page.goto('/login');
await page.getByLabel('Email').fill('user@example.com');
await page.getByLabel('Password').fill('password123');
await page.getByRole('button', { name: 'Login' }).click();
await page.waitForURL('/dashboard');
await use(page);
// Teardown: Logout after test
await page.getByRole('button', { name: 'Logout' }).click();
},
loginPage: async ({ page }, use) => {
const loginPage = new LoginPage(page);
await loginPage.goto();
await use(loginPage);
},
});
export { expect } from '@playwright/test';
// Usage
test('authenticated user can view profile', async ({ authenticatedPage }) => {
await authenticatedPage.goto('/profile');
await expect(authenticatedPage.getByText('Profile Settings')).toBeVisible();
});
```
### 4. API Testing with Playwright
**Pattern**: Test backend APIs alongside E2E flows
```typescript
import { test, expect } from '@playwright/test';
test.describe('API Testing', () => {
test('should fetch user data', async ({ request }) => {
const response = await request.get('/api/users/123');
expect(response.ok()).toBeTruthy();
expect(response.status()).toBe(200);
const data = await response.json();
expect(data).toMatchObject({
id: 123,
email: expect.any(String),
name: expect.any(String),
});
});
test('should handle authentication', async ({ request }) => {
const response = await request.post('/api/auth/login', {
data: {
email: 'user@example.com',
password: 'password123',
},
});
expect(response.ok()).toBeTruthy();
const { token } = await response.json();
expect(token).toBeTruthy();
// Use token in subsequent requests
const profileResponse = await request.get('/api/profile', {
headers: {
Authorization: `Bearer ${token}`,
},
});
expect(profileResponse.ok()).toBeTruthy();
});
test('should mock API responses', async ({ page }) => {
await page.route('/api/users', async (route) => {
await route.fulfill({
status: 200,
body: JSON.stringify([
{ id: 1, name: 'John Doe' },
{ id: 2, name: 'Jane Smith' },
]),
});
});
await page.goto('/users');
await expect(page.getByText('John Doe')).toBeVisible();
await expect(page.getByText('Jane Smith')).toBeVisible();
});
});
```
### 5. Visual Regression Testing
**Pattern**: Screenshot comparison for UI changes
```typescript
import { test, expect } from '@playwright/test';
test.describe('Visual Regression', () => {
test('homepage matches baseline', async ({ page }) => {
await page.goto('/');
await expect(page).toHaveScreenshot('homepage.png', {
fullPage: true,
animations: 'disabled',
});
});
test('component states', async ({ page }) => {
await page.goto('/components');
// Default state
const button = page.getByRole('button', { name: 'Submit' });
await expect(button).toHaveScreenshot('button-default.png');
// Hover state
await button.hover();
await expect(button).toHaveScreenshot('button-hover.png');
// Disabled state
await page.evaluate(() => {
document.querySelector('button')?.setAttribute('disabled', 'true');
});
await expect(button).toHaveScreenshot('button-disabled.png');
});
test('responsive screenshots', async ({ page }) => {
await page.goto('/');
// Desktop
await page.setViewportSize({ width: 1920, height: 1080 });
await expect(page).toHaveScreenshot('homepage-desktop.png');
// Tablet
await page.setViewportSize({ width: 768, height: 1024 });
await expect(page).toHaveScreenshot('homepage-tablet.png');
// Mobile
await page.setViewportSize({ width: 375, height: 667 });
await expect(page).toHaveScreenshot('homepage-mobile.png');
});
});
```
### 6. Mobile Emulation & Device Testing
**Pattern**: Test responsive behavior and touch interactions
```typescript
import { test, expect, devices } from '@playwright/test';
test.use(devices['iPhone 13 Pro']);
test.describe('Mobile Experience', () => {
test('should render mobile navigation', async ({ page }) => {
await page.goto('/');
// Mobile menu should be visible
await expect(page.getByRole('button', { name: 'Menu' })).toBeVisible();
// Desktop nav should be hidden
await expect(page.getByRole('navigation').first()).toBeHidden();
});
test('touch gestures', async ({ page }) => {
await page.goto('/gallery');
const image = page.getByRole('img').first();
// Swipe left
await image.dispatchEvent('touchstart', { touches: [{ clientX: 300, clientY: 200 }] });
await image.dispatchEvent('touchmove', { touches: [{ clientX: 100, clientY: 200 }] });
await image.dispatchEvent('touchend');
await expect(page.getByText('Next Image')).toBeVisible();
});
test('landscape orientation', async ({ page }) => {
await page.setViewportSize({ width: 812, height: 375 }); // iPhone landscape
await page.goto('/video');
await expect(page.locator('video')).toHaveCSS('width', '100%');
});
});
// Test across multiple devices
for (const deviceName of ['iPhone 13', 'Pixel 5', 'iPad Pro']) {
test.describe(`Device: ${deviceName}`, () => {
test.use(devices[deviceName]);
test('critical user flow', async ({ page }) => {
await page.goto('/');
// Test critical flow on each device
});
});
}
```
### 7. Accessibility Testing
**Pattern**: Automated accessibility checks
```typescript
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
test.describe('Accessibility', () => {
test('should not have accessibility violations', async ({ page }) => {
await page.goto('/');
const accessibilityScanResults = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
});
test('keyboard navigation', async ({ page }) => {
await page.goto('/form');
// Tab through form fields
await page.keyboard.press('Tab');
await expect(page.getByLabel('Email')).toBeFocused();
await page.keyboard.press('Tab');
await expect(page.getByLabel('Password')).toBeFocused();
await page.keyboard.press('Tab');
await expect(page.getByRole('button', { name: 'Submit' })).toBeFocused();
// Submit with Enter
await page.keyboard.press('Enter');
});
test('screen reader support', async ({ page }) => {
await page.goto('/');
// Check ARIA labels
await expect(page.getByRole('navigation', { name: 'Main' })).toBeVisible();
await expect(page.getByRole('main')).toHaveAttribute('aria-label', 'Main content');
// Check alt text
const images = page.getByRole('img');
for (const img of await images.all()) {
await expect(img).toHaveAttribute('alt');
}
});
});
```
### 8. Performance Testing
**Pattern**: Monitor performance metrics
```typescript
import { test, expect } from '@playwright/test';
test.describe('Performance', () => {
test('page load performance', async ({ page }) => {
await page.goto('/');
const performanceMetrics = await page.evaluate(() => {
const perfData = window.performance.timing;
return {
loadTime: perfData.loadEventEnd - perfData.navigationStart,
domContentLoaded: perfData.domContentLoadedEventEnd - perfData.navigationStart,
firstPaint: performance.getEntriesByType('paint')[0]?.startTime || 0,
};
});
expect(performanceMetrics.loadTime).toBeLessThan(3000); // 3s max
expect(performanceMetrics.domContentLoaded).toBeLessThan(2000); // 2s max
});
test('Core Web Vitals', async ({ page }) => {
await page.goto('/');
const vitals = await page.evaluate(() => {
return new Promise((resolve) => {
new PerformanceObserver((list) => {
const entries = list.getEntries();
const lcp = entries.find(e => e.entryType === 'largest-contentful-paint');
const fid = entries.find(e => e.entryType === 'first-input');
const cls = entries.find(e => e.entryType === 'layout-shift');
resolve({ lcp: lcp?.startTime, fid: fid?.processingStart, cls: cls?.value });
}).observe({ entryTypes: ['largest-contentful-paint', 'first-input', 'layout-shift'] });
});
});
expect(vitals.lcp).toBeLessThan(2500); // Good LCP
expect(vitals.fid).toBeLessThan(100); // Good FID
expect(vitals.cls).toBeLessThan(0.1); // Good CLS
});
});
```
### 9. Advanced Configuration
**playwright.config.ts**:
```typescript
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './e2e',
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: [
['html'],
['junit', { outputFile: 'test-results/junit.xml' }],
['json', { outputFile: 'test-results/results.json' }],
],
use: {
baseURL: process.env.BASE_URL || 'http://localhost:3000',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
video: 'retain-on-failure',
},
projects: [
// Desktop browsers
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
// Mobile browsers
{
name: 'Mobile Chrome',
use: { ...devices['Pixel 5'] },
},
{
name: 'Mobile Safari',
use: { ...devices['iPhone 13'] },
},
// Tablet browsers
{
name: 'iPad',
use: { ...devices['iPad Pro'] },
},
],
webServer: {
command: 'npm run dev',
url: 'http://localhost:3000',
reuseExistingServer: !process.env.CI,
timeout: 120 * 1000,
},
});
```
### 10. CI/CD Integration
**GitHub Actions**:
```yaml
name: E2E Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18
- name: Install dependencies
run: npm ci
- name: Install Playwright browsers
run: npx playwright install --with-deps
- name: Run E2E tests
run: npm run test:e2e
env:
BASE_URL: https://staging.example.com
- name: Upload test results
if: always()
uses: actions/upload-artifact@v3
with:
name: playwright-report
path: playwright-report/
retention-days: 30
- name: Upload traces
if: failure()
uses: actions/upload-artifact@v3
with:
name: playwright-traces
path: test-results/
```
### 11. Debugging Strategies
**Tools & Techniques**:
```typescript
// 1. Debug mode (headed browser + slow motion)
test('debug example', async ({ page }) => {
await page.goto('/');
await page.pause(); // Pauses execution, opens inspector
});
// 2. Console logs
test('capture console', async ({ page }) => {
page.on('console', msg => console.log(`Browser: ${msg.text()}`));
await page.goto('/');
});
// 3. Network inspection
test('inspect network', async ({ page }) => {
page.on('request', request => console.log('Request:', request.url()));
page.on('response', response => console.log('Response:', response.status()));
await page.goto('/');
});
// 4. Screenshots on failure
test.afterEach(async ({ page }, testInfo) => {
if (testInfo.status !== testInfo.expectedStatus) {
await page.screenshot({
path: `screenshots/${testInfo.title}.png`,
fullPage: true
});
}
});
// 5. Trace viewer
// Run: npx playwright test --trace on
// View: npx playwright show-trace trace.zip
```
**Common Debugging Commands**:
```bash
# Run in headed mode (see browser)
npx playwright test --headed
# Run with UI mode (interactive debugging)
npx playwright test --ui
# Run single test
npx playwright test tests/login.spec.ts
# Debug specific test
npx playwright test tests/login.spec.ts --debug
# Generate test code
npx playwright codegen http://localhost:3000
```
### 12. Handling Flaky Tests
**Patterns for Reliability**:
```typescript
// 1. Proper waiting strategies
test('wait for content', async ({ page }) => {
await page.goto('/');
// ❌ BAD: Fixed delays
// await page.waitForTimeout(5000);
// ✅ GOOD: Wait for specific conditions
await page.waitForLoadState('networkidle');
await page.waitForSelector('.content', { state: 'visible' });
await page.getByText('Welcome').waitFor();
});
// 2. Retry logic for external dependencies
test('api with retry', async ({ page }) => {
await page.goto('/');
let retries = 3;
while (retries > 0) {
try {
const response = await page.waitForResponse(
response => response.url().includes('/api/data') && response.ok(),
{ timeout: 5000 }
);
expect(response.ok()).toBeTruthy();
break;
} catch (error) {
retries--;
if (retries === 0) throw error;
await page.reload();
}
}
});
// 3. Test isolation
test.describe.configure({ mode: 'parallel' });
test.beforeEach(async ({ page }) => {
// Clear state before each test
await page.context().clearCookies();
await page.context().clearPermissions();
});
// 4. Deterministic test data
test('use fixtures', async ({ page }) => {
// Seed database with known data
await page.request.post('/api/test/seed', {
data: { userId: 'test-123', email: 'test@example.com' }
});
await page.goto('/users/test-123');
await expect(page.getByText('test@example.com')).toBeVisible();
// Cleanup
await page.request.delete('/api/test/users/test-123');
});
```
## Best Practices
### Test Organization
```
e2e/
├── fixtures/
│ ├── auth.fixture.ts
│ ├── data.fixture.ts
│ └── mock.fixture.ts
├── pages/
│ ├── LoginPage.ts
│ ├── DashboardPage.ts
│ └── ProfilePage.ts
├── tests/
│ ├── auth/
│ │ ├── login.spec.ts
│ │ ├── signup.spec.ts
│ │ └── logout.spec.ts
│ ├── user/
│ │ ├── profile.spec.ts
│ │ └── settings.spec.ts
│ └── api/
│ ├── users.spec.ts
│ └── posts.spec.ts
└── playwright.config.ts
```
### Naming Conventions
- Test files: `*.spec.ts` or `*.test.ts`
- Page objects: `*Page.ts`
- Fixtures: `*.fixture.ts`
- Descriptive test names: `should allow user to login with valid credentials`
### Performance Optimization
1. **Parallel execution**: Run tests in parallel across workers
2. **Test sharding**: Split tests across CI machines
3. **Selective testing**: Use tags/annotations for smoke tests
4. **Reuse authentication**: Save auth state, reuse across tests
5. **Mock external APIs**: Reduce network latency and flakiness
### Security Considerations
- Never commit credentials in test files
- Use environment variables for sensitive data
- Isolate test data from production
- Clear cookies/storage between tests
- Use disposable test accounts
## Common Patterns
### Authentication State Reuse
```typescript
// global-setup.ts
import { chromium, FullConfig } from '@playwright/test';
async function globalSetup(config: FullConfig) {
const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('http://localhost:3000/login');
await page.getByLabel('Email').fill('user@example.com');
await page.getByLabel('Password').fill('password123');
await page.getByRole('button', { name: 'Login' }).click();
await page.waitForURL('http://localhost:3000/dashboard');
// Save signed-in state
await page.context().storageState({ path: 'auth.json' });
await browser.close();
}
export default globalSetup;
// playwright.config.ts
export default defineConfig({
globalSetup: require.resolve('./global-setup'),
use: {
storageState: 'auth.json',
},
});
```
### Multi-Tab/Window Testing
```typescript
test('open in new tab', async ({ context }) => {
const page = await context.newPage();
await page.goto('/');
const [newPage] = await Promise.all([
context.waitForEvent('page'),
page.getByRole('link', { name: 'Open in new tab' }).click()
]);
await newPage.waitForLoadState();
await expect(newPage).toHaveURL('/new-page');
});
```
### File Upload/Download
```typescript
test('upload file', async ({ page }) => {
await page.goto('/upload');
const fileChooserPromise = page.waitForEvent('filechooser');
await page.getByRole('button', { name: 'Upload' }).click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles('path/to/file.pdf');
await expect(page.getByText('file.pdf uploaded')).toBeVisible();
});
test('download file', async ({ page }) => {
await page.goto('/downloads');
const downloadPromise = page.waitForEvent('download');
await page.getByRole('link', { name: 'Download Report' }).click();
const download = await downloadPromise;
await download.saveAs(`/tmp/${download.suggestedFilename()}`);
expect(download.suggestedFilename()).toBe('report.pdf');
});
```
## Troubleshooting
### Common Issues
1. **Timeouts**: Increase timeout, use proper wait strategies
2. **Flaky selectors**: Use stable locators (roles, labels, test IDs)
3. **Race conditions**: Wait for network idle, use explicit waits
4. **Authentication failures**: Clear cookies, check auth state
5. **Screenshot mismatches**: Update baselines, disable animations
### Debug Checklist
- [ ] Test passes locally in headed mode?
- [ ] Network requests succeed (check DevTools)?
- [ ] Selectors are stable and unique?
- [ ] Proper waits before assertions?
- [ ] Test data is deterministic?
- [ ] No race conditions with async operations?
- [ ] Traces/screenshots captured on failure?
## Resources
- **Official Docs**: https://playwright.dev
- **API Reference**: https://playwright.dev/docs/api/class-playwright
- **Best Practices**: https://playwright.dev/docs/best-practices
- **Examples**: https://github.com/microsoft/playwright/tree/main/examples
- **Community**: https://github.com/microsoft/playwright/discussions

454
skills/tdd-expert/SKILL.md Normal file
View File

@@ -0,0 +1,454 @@
---
name: tdd-expert
description: Test-Driven Development (TDD) expertise covering red-green-refactor cycle, behavior-driven development, test-first design, refactoring with confidence, TDD best practices, TDD workflow, unit testing strategies, mock-driven development, test doubles, TDD patterns, SOLID principles through testing, emergent design, incremental development, TDD anti-patterns, and production-grade TDD practices. Activates for TDD, test-driven development, red-green-refactor, test-first, behavior-driven, BDD, refactoring, test doubles, mock-driven, test design, SOLID principles, emergent design, incremental development, TDD workflow, TDD best practices, TDD patterns, Kent Beck, Robert Martin, Uncle Bob, test-first design.
---
# Test-Driven Development (TDD) Expert
**Self-contained TDD expertise for ANY user project.**
---
## The TDD Cycle: Red-Green-Refactor
### 1. RED Phase: Write Failing Test
**Goal**: Define expected behavior through a failing test
```typescript
import { describe, it, expect } from 'vitest';
import { Calculator } from './Calculator';
describe('Calculator', () => {
it('should add two numbers', () => {
const calculator = new Calculator();
expect(calculator.add(2, 3)).toBe(5); // WILL FAIL - Calculator doesn't exist
});
});
```
**RED Checklist**:
- [ ] Test describes ONE specific behavior
- [ ] Test fails for RIGHT reason (not syntax error)
- [ ] Test name is clear
- [ ] Expected behavior obvious
### 2. GREEN Phase: Minimal Implementation
**Goal**: Simplest code that makes test pass
```typescript
// Calculator.ts
export class Calculator {
add(a: number, b: number): number {
return a + b; // Minimal implementation
}
}
```
**GREEN Checklist**:
- [ ] Test passes
- [ ] Code is simplest possible
- [ ] No premature optimization
- [ ] No extra features
### 3. REFACTOR Phase: Improve Design
**Goal**: Improve code quality without changing behavior
```typescript
// Refactor: Support variable arguments
export class Calculator {
add(...numbers: number[]): number {
return numbers.reduce((sum, n) => sum + n, 0);
}
}
// Tests still pass!
```
**REFACTOR Checklist**:
- [ ] All tests still pass
- [ ] Code is more readable
- [ ] Removed duplication
- [ ] Better design patterns
---
## TDD Benefits
**Design Benefits**:
- Forces modular, testable code
- Reveals design problems early
- Encourages SOLID principles
- Promotes simple solutions
**Quality Benefits**:
- 100% test coverage (by definition)
- Tests document behavior
- Regression safety net
- Faster debugging
**Productivity Benefits**:
- Less time debugging
- Confidence to refactor
- Faster iterations
- Clearer requirements
---
## BDD: Behavior-Driven Development
**Extension of TDD with natural language tests**
### Given-When-Then Pattern
```typescript
describe('Shopping Cart', () => {
it('should apply 10% discount when total exceeds $100', () => {
// Given: A cart with $120 worth of items
const cart = new ShoppingCart();
cart.addItem({ price: 120, quantity: 1 });
// When: Getting the total
const total = cart.getTotal();
// Then: 10% discount applied
expect(total).toBe(108); // $120 - $12 (10%)
});
});
```
**BDD Benefits**:
- Tests readable by non-developers
- Clear business requirements
- Better stakeholder communication
- Executable specifications
---
## TDD Patterns
### Pattern 1: Test List
Before coding, list all tests needed:
```markdown
Calculator Tests:
- [ ] add two positive numbers
- [ ] add negative numbers
- [ ] add zero
- [ ] add multiple numbers
- [ ] multiply two numbers
- [ ] divide two numbers
- [ ] divide by zero (error)
```
Work through list one by one.
### Pattern 2: Fake It Till You Make It
Start with hardcoded returns, generalize later:
```typescript
// Test 1: add(2, 3) = 5
add(a, b) { return 5; } // Hardcoded!
// Test 2: add(5, 7) = 12
add(a, b) { return a + b; } // Generalized
```
### Pattern 3: Triangulation
Use multiple tests to force generalization:
```typescript
// Test 1
expect(fizzbuzz(3)).toBe('Fizz');
// Test 2
expect(fizzbuzz(5)).toBe('Buzz');
// Test 3
expect(fizzbuzz(15)).toBe('FizzBuzz');
// Forces complete implementation
```
### Pattern 4: Test Data Builders
Create test helpers for complex objects:
```typescript
class UserBuilder {
private user = { name: 'Test', email: 'test@example.com', role: 'user' };
withName(name: string) {
this.user.name = name;
return this;
}
withRole(role: string) {
this.user.role = role;
return this;
}
build() {
return this.user;
}
}
// Usage
const admin = new UserBuilder().withRole('admin').build();
```
---
## Refactoring with Confidence
**The TDD Safety Net**
### Refactoring Types
**1. Extract Method**:
```typescript
// Before
function processOrder(order) {
const total = order.items.reduce((sum, item) => sum + item.price, 0);
const tax = total * 0.1;
return total + tax;
}
// After (refactored with test safety)
function calculateTotal(items) {
return items.reduce((sum, item) => sum + item.price, 0);
}
function calculateTax(total) {
return total * 0.1;
}
function processOrder(order) {
const total = calculateTotal(order.items);
const tax = calculateTax(total);
return total + tax;
}
```
**2. Remove Duplication**:
```typescript
// Tests force you to see duplication
it('should validate email', () => {
expect(validateEmail('test@example.com')).toBe(true);
expect(validateEmail('invalid')).toBe(false);
});
it('should validate phone', () => {
expect(validatePhone('+1-555-0100')).toBe(true);
expect(validatePhone('invalid')).toBe(false);
});
// Extract common validation pattern
```
### Refactoring Workflow
```
1. All tests GREEN? → Continue
2. Identify code smell
3. Make small refactoring
4. Run tests → GREEN? → Continue
5. Repeat until satisfied
6. Commit
```
---
## TDD Anti-Patterns
### ❌ Testing Implementation Details
```typescript
// BAD: Testing private method
it('should call _validateEmail internally', () => {
spyOn(service, '_validateEmail');
service.createUser({ email: 'test@example.com' });
expect(service._validateEmail).toHaveBeenCalled();
});
// GOOD: Testing behavior
it('should reject invalid email', () => {
expect(() => service.createUser({ email: 'invalid' }))
.toThrow('Invalid email');
});
```
### ❌ Writing Tests After Code
```typescript
// Wrong order!
1. Write implementation
2. Write tests
// Correct TDD:
1. Write test (RED)
2. Write implementation (GREEN)
3. Refactor
```
### ❌ Large Tests
```typescript
// BAD: Testing multiple behaviors
it('should handle user lifecycle', () => {
const user = createUser();
updateUser(user, { name: 'New Name' });
deleteUser(user);
// Too much in one test!
});
// GOOD: One behavior per test
it('should create user', () => {
const user = createUser();
expect(user).toBeDefined();
});
it('should update user name', () => {
const user = createUser();
updateUser(user, { name: 'New Name' });
expect(user.name).toBe('New Name');
});
```
### ❌ Skipping Refactor Phase
```typescript
// Don't skip refactoring!
RED → GREEN → REFACTOR → RED → GREEN → REFACTOR
↑________________↑
Always refactor!
```
---
## Mock-Driven TDD
**When testing with external dependencies**
### Strategy 1: Dependency Injection
```typescript
class UserService {
constructor(private db: Database) {} // Inject dependency
async getUser(id: string) {
return this.db.query('SELECT * FROM users WHERE id = ?', [id]);
}
}
// Test with mock
const mockDb = { query: vi.fn().mockResolvedValue({ id: '123' }) };
const service = new UserService(mockDb);
```
### Strategy 2: Interface-Based Mocking
```typescript
interface EmailService {
send(to: string, subject: string, body: string): Promise<void>;
}
class MockEmailService implements EmailService {
sent: any[] = [];
async send(to: string, subject: string, body: string) {
this.sent.push({ to, subject, body });
}
}
// Test with mock
const mockEmail = new MockEmailService();
const service = new UserService(mockEmail);
await service.registerUser({ email: 'test@example.com' });
expect(mockEmail.sent).toHaveLength(1);
```
---
## SOLID Principles Through TDD
**TDD naturally leads to SOLID design**
### Single Responsibility (SRP)
Tests reveal when class does too much:
```typescript
// Many tests for one class? Split it!
describe('UserManager', () => {
// 20+ tests here → Too many responsibilities
});
// Refactor to multiple classes
describe('UserCreator', () => { /* 5 tests */ });
describe('UserValidator', () => { /* 5 tests */ });
describe('UserNotifier', () => { /* 5 tests */ });
```
### Open/Closed (OCP)
Tests enable extension without modification:
```typescript
// Testable, extensible design
interface PaymentProcessor {
process(amount: number): Promise<void>;
}
class StripeProcessor implements PaymentProcessor { }
class PayPalProcessor implements PaymentProcessor { }
```
### Dependency Inversion (DIP)
TDD requires dependency injection:
```typescript
// Testable: Depends on abstraction
class OrderService {
constructor(private payment: PaymentProcessor) {}
}
// Easy to test with mocks
const mockPayment = new MockPaymentProcessor();
const service = new OrderService(mockPayment);
```
---
## Quick Reference
### TDD Workflow
```
1. Write test (RED) → Fails ✅
2. Minimal code (GREEN) → Passes ✅
3. Refactor → Still passes ✅
4. Repeat
```
### Test Smells
- Test too long (>20 lines)
- Multiple assertions (>3)
- Testing implementation
- Unclear test name
- Slow tests (>100ms)
- Flaky tests
### When to Use TDD
✅ New features
✅ Bug fixes (add test first)
✅ Refactoring
✅ Complex logic
✅ Public APIs
❌ Throwaway prototypes
❌ UI layout (use E2E instead)
❌ Highly experimental code
---
**This skill is self-contained and works in ANY user project.**

519
skills/unit-testing-expert/SKILL.md Normal file
View File

@@ -0,0 +1,519 @@
---
name: unit-testing-expert
description: Comprehensive unit testing expertise covering Vitest, Jest, test-driven development (TDD), mocking strategies, test coverage, snapshot testing, test architecture, testing patterns, dependency injection, test doubles (mocks, stubs, spies, fakes), async testing, error handling tests, parametric testing, test organization, code coverage analysis, mutation testing, and production-grade unit testing best practices. Activates for unit testing, vitest, jest, test-driven development, TDD, red-green-refactor, mocking, stubbing, spying, test doubles, test coverage, snapshot testing, test architecture, dependency injection, async testing, test patterns, code coverage, mutation testing, test isolation, test fixtures, AAA pattern, given-when-then, test organization, testing best practices, vi.fn, vi.mock, vi.spyOn, describe, it, expect, beforeEach, afterEach.
---
# Unit Testing Expert
**Self-contained unit testing expertise for Vitest/Jest in ANY user project.**
---
## Test-Driven Development (TDD)
**Red-Green-Refactor Cycle**:
```typescript
// 1. RED: Write failing test
describe('Calculator', () => {
it('should add two numbers', () => {
const calc = new Calculator();
expect(calc.add(2, 3)).toBe(5);
});
});
// 2. GREEN: Minimal implementation
class Calculator {
add(a: number, b: number): number {
return a + b;
}
}
// 3. REFACTOR: Improve code
class Calculator {
add(...numbers: number[]): number {
return numbers.reduce((sum, n) => sum + n, 0);
}
}
```
**TDD Benefits**:
- Better design (testable code)
- Living documentation
- Faster debugging
- Higher confidence
---
## Vitest/Jest Fundamentals
### Basic Test Structure
```typescript
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import { UserService } from './UserService';
describe('UserService', () => {
let service: UserService;
beforeEach(() => {
service = new UserService();
});
afterEach(() => {
vi.clearAllMocks();
});
it('should create user', () => {
const user = service.create({ name: 'John', email: 'john@test.com' });
expect(user).toMatchObject({
id: expect.any(String),
name: 'John',
email: 'john@test.com'
});
});
it('should throw for invalid email', () => {
expect(() => {
service.create({ name: 'John', email: 'invalid' });
}).toThrow('Invalid email');
});
});
```
### Async Testing
```typescript
it('should fetch user from API', async () => {
const user = await api.fetchUser('user-123');
expect(user).toEqual({
id: 'user-123',
name: 'John Doe'
});
});
// Testing async errors
it('should handle API errors', async () => {
await expect(api.fetchUser('invalid')).rejects.toThrow('User not found');
});
```
---
## Mocking Strategies
### 1. Mock Functions
```typescript
// Mock a function
const mockFn = vi.fn();
mockFn.mockReturnValue(42);
expect(mockFn()).toBe(42);
// Mock with implementation
const mockAdd = vi.fn((a, b) => a + b);
expect(mockAdd(2, 3)).toBe(5);
// Verify calls
expect(mockFn).toHaveBeenCalledTimes(1);
expect(mockFn).toHaveBeenCalledWith(expected);
```
### 2. Mock Modules
```typescript
// Mock entire module
vi.mock('./database', () => ({
query: vi.fn().mockResolvedValue([{ id: 1, name: 'Test' }])
}));
import { query } from './database';
it('should fetch users from database', async () => {
const users = await query('SELECT * FROM users');
expect(users).toHaveLength(1);
});
```
### 3. Spies
```typescript
// Spy on existing method
const spy = vi.spyOn(console, 'log');
myFunction();
expect(spy).toHaveBeenCalledWith('Expected message');
spy.mockRestore();
```
### 4. Mock Dependencies
```typescript
class UserService {
constructor(private db: Database) {}
async getUser(id: string) {
return this.db.query('SELECT * FROM users WHERE id = ?', [id]);
}
}
// Test with mock
const mockDb = {
query: vi.fn().mockResolvedValue({ id: '123', name: 'John' })
};
const service = new UserService(mockDb);
const user = await service.getUser('123');
expect(mockDb.query).toHaveBeenCalledWith(
'SELECT * FROM users WHERE id = ?',
['123']
);
```
---
## Test Patterns
### AAA Pattern (Arrange-Act-Assert)
```typescript
it('should calculate total price', () => {
// Arrange
const cart = new ShoppingCart();
cart.addItem({ price: 10, quantity: 2 });
cart.addItem({ price: 5, quantity: 3 });
// Act
const total = cart.getTotal();
// Assert
expect(total).toBe(35);
});
```
### Given-When-Then (BDD)
```typescript
describe('Shopping Cart', () => {
it('should apply discount when total exceeds $100', () => {
// Given: A cart with items totaling $120
const cart = new ShoppingCart();
cart.addItem({ price: 120, quantity: 1 });
// When: Getting the total
const total = cart.getTotal();
// Then: 10% discount applied
expect(total).toBe(108); // $120 - $12 (10%)
});
});
```
### Parametric Testing
```typescript
describe.each([
[2, 3, 5],
[10, 5, 15],
[-1, 1, 0],
[0, 0, 0]
])('Calculator.add(%i, %i)', (a, b, expected) => {
it(`should return ${expected}`, () => {
const calc = new Calculator();
expect(calc.add(a, b)).toBe(expected);
});
});
```
---
## Test Doubles
### Mocks vs Stubs vs Spies vs Fakes
**Mock**: Verifies behavior (calls, arguments)
```typescript
const mock = vi.fn();
mock('test');
expect(mock).toHaveBeenCalledWith('test');
```
**Stub**: Returns predefined values
```typescript
const stub = vi.fn().mockReturnValue(42);
expect(stub()).toBe(42);
```
**Spy**: Observes real function
```typescript
const spy = vi.spyOn(obj, 'method');
obj.method();
expect(spy).toHaveBeenCalled();
```
**Fake**: Working implementation (simplified)
```typescript
class FakeDatabase {
private data = new Map();
async save(key, value) {
this.data.set(key, value);
}
async get(key) {
return this.data.get(key);
}
}
```
---
## Coverage Analysis
### Running Coverage
```bash
# Vitest
vitest --coverage
# Jest
jest --coverage
```
### Coverage Thresholds
```javascript
// vitest.config.ts
export default {
test: {
coverage: {
provider: 'v8',
reporter: ['text', 'html', 'lcov'],
lines: 80,
functions: 80,
branches: 80,
statements: 80
}
}
};
```
### Coverage Best Practices
**✅ DO**:
- Aim for 80-90% coverage
- Focus on business logic
- Test edge cases
- Test error paths
**❌ DON'T**:
- Chase 100% coverage
- Test getters/setters only
- Test framework code
- Write tests just for coverage
---
## Snapshot Testing
### When to Use Snapshots
**Good use cases**:
- UI component output
- API responses
- Configuration objects
- Error messages
```typescript
it('should render user card', () => {
const card = renderUserCard({ name: 'John', role: 'Admin' });
expect(card).toMatchSnapshot();
});
// Update snapshots: vitest -u
```
**Avoid snapshots for**:
- Dates/timestamps
- Random values
- Large objects (prefer specific assertions)
---
## Test Organization
### File Structure
```
src/
├── services/
│ ├── UserService.ts
│ └── UserService.test.ts ← Co-located
tests/
├── unit/
│ └── utils.test.ts
├── integration/
│ └── api.test.ts
└── fixtures/
└── users.json
```
### Test Naming
**✅ GOOD**:
```typescript
describe('UserService.create', () => {
it('should create user with valid email', () => {});
it('should throw error for invalid email', () => {});
it('should generate unique ID', () => {});
});
```
**❌ BAD**:
```typescript
describe('UserService', () => {
it('test1', () => {});
it('should work', () => {});
});
```
---
## Error Handling Tests
```typescript
// Synchronous errors
it('should throw for negative numbers', () => {
expect(() => sqrt(-1)).toThrow('Cannot compute square root of negative');
});
// Async errors
it('should reject for invalid ID', async () => {
await expect(fetchUser('invalid')).rejects.toThrow('Invalid ID');
});
// Error types
it('should throw TypeError', () => {
expect(() => doSomething()).toThrow(TypeError);
});
// Custom errors
it('should throw ValidationError', () => {
expect(() => validate()).toThrow(ValidationError);
});
```
---
## Test Isolation
### Reset State Between Tests
```typescript
let service: UserService;
beforeEach(() => {
service = new UserService();
vi.clearAllMocks();
});
afterEach(() => {
vi.restoreAllMocks();
});
```
### Avoid Test Interdependence
**❌ BAD**:
```typescript
let user;
it('should create user', () => {
user = createUser(); // Shared state
});
it('should update user', () => {
updateUser(user); // Depends on previous test
});
```
**✅ GOOD**:
```typescript
it('should update user', () => {
const user = createUser();
updateUser(user);
expect(user.updated).toBe(true);
});
```
---
## Best Practices Summary
**✅ DO**:
- Write tests before code (TDD)
- Test behavior, not implementation
- One assertion per test (when possible)
- Clear test names (should...)
- Mock external dependencies
- Test edge cases and errors
- Keep tests fast (<100ms each)
- Use descriptive variable names
- Clean up after tests
**❌ DON'T**:
- Test private methods directly
- Share state between tests
- Use real databases/APIs
- Test framework code
- Write fragile tests (implementation-dependent)
- Skip error cases
- Use magic numbers
- Leave commented-out tests
---
## Quick Reference
### Assertions
```typescript
expect(value).toBe(expected); // ===
expect(value).toEqual(expected); // Deep equality
expect(value).toBeTruthy(); // Boolean true
expect(value).toBeFalsy(); // Boolean false
expect(array).toHaveLength(3); // Array length
expect(array).toContain(item); // Array includes
expect(string).toMatch(/pattern/); // Regex match
expect(fn).toThrow(Error); // Throws error
expect(obj).toHaveProperty('key'); // Has property
expect(value).toBeCloseTo(0.3, 5); // Float comparison
```
### Lifecycle Hooks
```typescript
beforeAll(() => {}); // Once before all tests
beforeEach(() => {}); // Before each test
afterEach(() => {}); // After each test
afterAll(() => {}); // Once after all tests
```
### Mock Utilities
```typescript
vi.fn() // Create mock
vi.fn().mockReturnValue(x) // Return value
vi.fn().mockResolvedValue(x) // Async return
vi.fn().mockRejectedValue(e) // Async error
vi.mock('./module') // Mock module
vi.spyOn(obj, 'method') // Spy on method
vi.clearAllMocks() // Clear call history
vi.resetAllMocks() // Reset + clear
vi.restoreAllMocks() // Restore originals
```
---
**This skill is self-contained and works in ANY user project with Vitest/Jest.**