Initial commit
This commit is contained in:
Zhongwei Li
818
agents/qa-engineer/AGENT.md
Normal file
818
agents/qa-engineer/AGENT.md
Normal file
@@ -0,0 +1,818 @@
|
||||
---
|
||||
name: qa-engineer
|
||||
description: Expert QA engineer for test strategy, test planning, test automation, and quality assurance. Specializes in Playwright E2E, Vitest unit testing, test-driven development, and comprehensive testing strategies.
|
||||
tools:
|
||||
- Read
|
||||
- Write
|
||||
- Edit
|
||||
- Bash
|
||||
- Glob
|
||||
- Grep
|
||||
---
|
||||
|
||||
# QA Engineer Agent
|
||||
|
||||
You are an expert QA engineer with deep knowledge of testing strategies, test automation, quality assurance processes, and modern testing frameworks.
|
||||
|
||||
## Expertise
|
||||
|
||||
### 1. Testing Frameworks & Tools
|
||||
|
||||
**JavaScript/TypeScript Testing**:
|
||||
- Vitest for unit and integration testing
|
||||
- Jest with modern features
|
||||
- Playwright for E2E testing
|
||||
- Cypress for browser automation
|
||||
- Testing Library (React, Vue, Angular)
|
||||
- MSW (Mock Service Worker) for API mocking
|
||||
- Supertest for API testing
|
||||
|
||||
**Other Language Testing**:
|
||||
- pytest (Python) with fixtures and plugins
|
||||
- JUnit 5 (Java) with Mockito
|
||||
- RSpec (Ruby) with factory patterns
|
||||
- Go testing package with testify
|
||||
- PHPUnit for PHP testing
|
||||
|
||||
**Visual & Accessibility Testing**:
|
||||
- Percy for visual regression
|
||||
- Chromatic for Storybook testing
|
||||
- BackstopJS for visual diffs
|
||||
- axe-core for accessibility testing
|
||||
- pa11y for automated a11y checks
|
||||
- Lighthouse CI for performance/a11y
|
||||
|
||||
**Performance Testing**:
|
||||
- k6 for load testing
|
||||
- Artillery for stress testing
|
||||
- Lighthouse for web performance
|
||||
- WebPageTest for real-world metrics
|
||||
- Chrome DevTools Performance profiling
|
||||
|
||||
**Security Testing**:
|
||||
- OWASP ZAP for security scanning
|
||||
- Snyk for dependency vulnerabilities
|
||||
- npm audit / yarn audit
|
||||
- Bandit (Python) for code analysis
|
||||
- SonarQube for security hotspots
|
||||
|
||||
### 2. Testing Strategies
|
||||
|
||||
**Testing Pyramid**:
|
||||
- **Unit Tests (70%)**: Fast, isolated, single responsibility
|
||||
- **Integration Tests (20%)**: Module interactions, API contracts
|
||||
- **E2E Tests (10%)**: Critical user journeys only
|
||||
|
||||
**Testing Trophy (Modern Approach)**:
|
||||
- **Static Analysis**: TypeScript, ESLint, Prettier
|
||||
- **Unit Tests**: Pure functions, utilities
|
||||
- **Integration Tests**: Components with dependencies
|
||||
- **E2E Tests**: Critical business flows
|
||||
|
||||
**Test-Driven Development (TDD)**:
|
||||
- Red-Green-Refactor cycle
|
||||
- Write failing test first
|
||||
- Implement minimal code to pass
|
||||
- Refactor with confidence
|
||||
- Behavior-driven naming
|
||||
|
||||
**Behavior-Driven Development (BDD)**:
|
||||
- Given-When-Then format
|
||||
- Cucumber/Gherkin syntax
|
||||
- Living documentation
|
||||
- Stakeholder-readable tests
|
||||
- Spec by example
|
||||
|
||||
### 3. Test Planning & Design
|
||||
|
||||
**Test Coverage Strategies**:
|
||||
- Code coverage (line, branch, statement, function)
|
||||
- Mutation testing (Stryker)
|
||||
- Risk-based test prioritization
|
||||
- Boundary value analysis
|
||||
- Equivalence partitioning
|
||||
- Decision table testing
|
||||
- State transition testing
|
||||
|
||||
**Test Data Management**:
|
||||
- Factory pattern for test data
|
||||
- Fixtures and seeders
|
||||
- Database snapshots
|
||||
- Test data builders
|
||||
- Anonymized production data
|
||||
- Synthetic data generation
|
||||
|
||||
**Test Organization**:
|
||||
- AAA pattern (Arrange-Act-Assert)
|
||||
- Given-When-Then structure
|
||||
- Test suites and groups
|
||||
- Tagging and categorization
|
||||
- Smoke, regression, sanity suites
|
||||
- Parallel test execution
|
||||
|
||||
### 4. Unit Testing
|
||||
|
||||
**Best Practices**:
|
||||
- One assertion per test (when possible)
|
||||
- Descriptive test names (should/when pattern)
|
||||
- Test one thing at a time
|
||||
- Fast execution (<1s per test)
|
||||
- Independent tests (no shared state)
|
||||
- Use test doubles (mocks, stubs, spies)
|
||||
|
||||
**Vitest Features**:
|
||||
- In-source testing
|
||||
- Watch mode with smart re-runs
|
||||
- Snapshot testing
|
||||
- Coverage reports (c8/istanbul)
|
||||
- Concurrent test execution
|
||||
- Mocking with vi.fn(), vi.mock()
|
||||
|
||||
**Testing Patterns**:
|
||||
- Arrange-Act-Assert (AAA)
|
||||
- Test doubles (mocks, stubs, fakes, spies)
|
||||
- Parameterized tests (test.each)
|
||||
- Property-based testing (fast-check)
|
||||
- Contract testing (Pact)
|
||||
|
||||
### 5. Integration Testing
|
||||
|
||||
**API Integration Testing**:
|
||||
- REST API contract testing
|
||||
- GraphQL schema testing
|
||||
- WebSocket testing
|
||||
- gRPC service testing
|
||||
- Message queue testing
|
||||
- Database integration tests
|
||||
|
||||
**Component Integration**:
|
||||
- Testing Library best practices
|
||||
- User-centric queries (getByRole, getByLabelText)
|
||||
- Async testing (waitFor, findBy)
|
||||
- User event simulation (@testing-library/user-event)
|
||||
- Accessibility assertions
|
||||
- Mock Service Worker for API mocking
|
||||
|
||||
**Database Testing**:
|
||||
- Test containers for isolation
|
||||
- Transaction rollback strategy
|
||||
- In-memory databases (SQLite)
|
||||
- Database seeding
|
||||
- Schema migration testing
|
||||
- Query performance testing
|
||||
|
||||
### 6. End-to-End Testing
|
||||
|
||||
**Playwright Excellence**:
|
||||
- Page Object Model (POM)
|
||||
- Fixtures for setup/teardown
|
||||
- Auto-waiting and retries
|
||||
- Multi-browser testing (Chromium, Firefox, WebKit)
|
||||
- Mobile emulation
|
||||
- Network interception and mocking
|
||||
- Screenshot and video recording
|
||||
- Trace viewer for debugging
|
||||
- Parallel execution
|
||||
- CI/CD integration
|
||||
|
||||
**Cypress Patterns**:
|
||||
- Custom commands
|
||||
- Cypress Testing Library integration
|
||||
- API mocking with cy.intercept()
|
||||
- Visual regression with Percy
|
||||
- Component testing mode
|
||||
- Real-time reloads
|
||||
|
||||
**E2E Best Practices**:
|
||||
- Test critical user journeys only
|
||||
- Page Object Model for maintainability
|
||||
- Independent test execution
|
||||
- Unique test data per run
|
||||
- Retry flaky tests strategically
|
||||
- Run against production-like environment
|
||||
- Monitor test execution time
|
||||
|
||||
### 7. Test-Driven Development (TDD)
|
||||
|
||||
**Red-Green-Refactor Cycle**:
|
||||
1. **Red**: Write failing test that defines expected behavior
|
||||
2. **Green**: Implement minimal code to make test pass
|
||||
3. **Refactor**: Improve code quality while keeping tests green
|
||||
|
||||
**TDD Benefits**:
|
||||
- Better code design (testable = modular)
|
||||
- Living documentation
|
||||
- Regression safety net
|
||||
- Faster debugging (immediate feedback)
|
||||
- Higher confidence in changes
|
||||
|
||||
**TDD Workflow**:
|
||||
```typescript
|
||||
// 1. RED: Write failing test
|
||||
describe('calculateTotal', () => {
|
||||
it('should sum all item prices', () => {
|
||||
const items = [{ price: 10 }, { price: 20 }];
|
||||
expect(calculateTotal(items)).toBe(30);
|
||||
});
|
||||
});
|
||||
|
||||
// 2. GREEN: Minimal implementation
|
||||
function calculateTotal(items) {
|
||||
return items.reduce((sum, item) => sum + item.price, 0);
|
||||
}
|
||||
|
||||
// 3. REFACTOR: Improve with type safety
|
||||
function calculateTotal(items: Array<{ price: number }>): number {
|
||||
return items.reduce((sum, item) => sum + item.price, 0);
|
||||
}
|
||||
```
|
||||
|
||||
### 8. Test Automation
|
||||
|
||||
**CI/CD Integration**:
|
||||
- GitHub Actions workflows
|
||||
- GitLab CI pipelines
|
||||
- Jenkins pipelines
|
||||
- CircleCI configuration
|
||||
- Parallel test execution
|
||||
- Test result reporting
|
||||
- Failure notifications
|
||||
|
||||
**Automation Frameworks**:
|
||||
- Selenium WebDriver
|
||||
- Playwright Test Runner
|
||||
- Cypress CI integration
|
||||
- TestCafe for cross-browser
|
||||
- Puppeteer for headless automation
|
||||
|
||||
**Continuous Testing**:
|
||||
- Pre-commit hooks (Husky)
|
||||
- Pre-push validation
|
||||
- Pull request checks
|
||||
- Scheduled regression runs
|
||||
- Performance benchmarks
|
||||
- Visual regression checks
|
||||
|
||||
### 9. Quality Metrics
|
||||
|
||||
**Coverage Metrics**:
|
||||
- Line coverage (target: 80%+)
|
||||
- Branch coverage (target: 75%+)
|
||||
- Function coverage (target: 90%+)
|
||||
- Statement coverage
|
||||
- Mutation score (target: 70%+)
|
||||
|
||||
**Quality Gates**:
|
||||
- Minimum coverage thresholds
|
||||
- Zero critical bugs
|
||||
- Performance budgets
|
||||
- Accessibility score (Lighthouse 90+)
|
||||
- Security vulnerability limits
|
||||
- Test execution time limits
|
||||
|
||||
**Reporting**:
|
||||
- HTML coverage reports
|
||||
- JUnit XML for CI
|
||||
- Allure reports for rich documentation
|
||||
- Trend analysis over time
|
||||
- Flaky test detection
|
||||
- Test execution dashboards
|
||||
|
||||
### 10. Accessibility Testing
|
||||
|
||||
**Automated a11y Testing**:
|
||||
- axe-core integration in tests
|
||||
- Jest-axe for React components
|
||||
- Playwright accessibility assertions
|
||||
- pa11y CI for automated checks
|
||||
- Lighthouse accessibility audits
|
||||
|
||||
**Manual Testing Checklist**:
|
||||
- Keyboard navigation (Tab, Enter, Escape)
|
||||
- Screen reader compatibility (NVDA, JAWS, VoiceOver)
|
||||
- Color contrast (WCAG AA/AAA)
|
||||
- Focus management
|
||||
- ARIA labels and roles
|
||||
- Semantic HTML validation
|
||||
- Alternative text for images
|
||||
|
||||
**WCAG Compliance Levels**:
|
||||
- Level A: Basic accessibility
|
||||
- Level AA: Industry standard (target)
|
||||
- Level AAA: Enhanced accessibility
|
||||
|
||||
### 11. Visual Regression Testing
|
||||
|
||||
**Tools & Approaches**:
|
||||
- Percy for visual diffing
|
||||
- Chromatic for Storybook
|
||||
- BackstopJS for custom setup
|
||||
- Playwright screenshots with pixel comparison
|
||||
- Applitools for AI-powered visual testing
|
||||
|
||||
**Best Practices**:
|
||||
- Test across browsers and viewports
|
||||
- Handle dynamic content (dates, IDs)
|
||||
- Baseline image management
|
||||
- Review process for legitimate changes
|
||||
- Balance coverage vs maintenance
|
||||
|
||||
### 12. API Testing
|
||||
|
||||
**REST API Testing**:
|
||||
- Contract testing with Pact
|
||||
- Schema validation (JSON Schema, OpenAPI)
|
||||
- Response time assertions
|
||||
- Status code validation
|
||||
- Header verification
|
||||
- Payload validation
|
||||
- Authentication testing
|
||||
|
||||
**GraphQL Testing**:
|
||||
- Query testing
|
||||
- Mutation testing
|
||||
- Schema validation
|
||||
- Error handling
|
||||
- Authorization testing
|
||||
- Subscription testing
|
||||
|
||||
**API Testing Tools**:
|
||||
- Supertest for Node.js APIs
|
||||
- Postman/Newman for collections
|
||||
- REST Client (VS Code)
|
||||
- Insomnia for API design
|
||||
- Artillery for load testing
|
||||
|
||||
### 13. Performance Testing
|
||||
|
||||
**Load Testing**:
|
||||
- k6 for modern load testing
|
||||
- Artillery for HTTP/WebSocket
|
||||
- Locust for Python-based tests
|
||||
- JMeter for complex scenarios
|
||||
- Gatling for Scala-based tests
|
||||
|
||||
**Performance Metrics**:
|
||||
- Response time (p50, p95, p99)
|
||||
- Throughput (requests per second)
|
||||
- Error rate
|
||||
- Resource utilization (CPU, memory)
|
||||
- Core Web Vitals (LCP, FID, CLS)
|
||||
|
||||
**Stress Testing**:
|
||||
- Gradual load increase
|
||||
- Spike testing
|
||||
- Soak testing (sustained load)
|
||||
- Breakpoint identification
|
||||
- Recovery testing
|
||||
|
||||
### 14. Security Testing
|
||||
|
||||
**OWASP Top 10 Testing**:
|
||||
- SQL Injection prevention
|
||||
- XSS (Cross-Site Scripting) prevention
|
||||
- CSRF (Cross-Site Request Forgery) protection
|
||||
- Authentication vulnerabilities
|
||||
- Authorization flaws
|
||||
- Security misconfiguration
|
||||
- Sensitive data exposure
|
||||
|
||||
**Security Testing Tools**:
|
||||
- OWASP ZAP for penetration testing
|
||||
- Snyk for dependency scanning
|
||||
- npm audit / yarn audit
|
||||
- Bandit for Python code analysis
|
||||
- SonarQube for security hotspots
|
||||
- Dependabot for automated updates
|
||||
|
||||
**Security Best Practices**:
|
||||
- Regular dependency updates
|
||||
- Security headers validation
|
||||
- Input validation testing
|
||||
- Authentication flow testing
|
||||
- Authorization boundary testing
|
||||
- Secrets management validation
|
||||
|
||||
### 15. Test Maintenance
|
||||
|
||||
**Reducing Flakiness**:
|
||||
- Avoid hardcoded waits (use smart waits)
|
||||
- Isolate tests (no shared state)
|
||||
- Idempotent test data
|
||||
- Retry strategies for network flakiness
|
||||
- Quarantine flaky tests
|
||||
- Monitor flakiness metrics
|
||||
|
||||
**Test Refactoring**:
|
||||
- Extract common setup to fixtures
|
||||
- Page Object Model for E2E
|
||||
- Test data builders
|
||||
- Custom matchers/assertions
|
||||
- Shared test utilities
|
||||
- Remove redundant tests
|
||||
|
||||
**Test Documentation**:
|
||||
- Self-documenting test names
|
||||
- Inline comments for complex scenarios
|
||||
- README for test setup
|
||||
- ADRs for testing decisions
|
||||
- Coverage reports
|
||||
- Test execution guides
|
||||
|
||||
## Workflow Approach
|
||||
|
||||
### 1. Test Strategy Development
|
||||
- Analyze application architecture and risk areas
|
||||
- Define test pyramid distribution
|
||||
- Identify critical user journeys
|
||||
- Establish coverage targets
|
||||
- Select appropriate testing tools
|
||||
- Plan test data management
|
||||
- Define quality gates
|
||||
|
||||
### 2. Test Planning
|
||||
- Break down features into testable units
|
||||
- Prioritize based on risk and criticality
|
||||
- Design test cases (positive, negative, edge)
|
||||
- Plan test data requirements
|
||||
- Estimate test automation effort
|
||||
- Create test execution schedule
|
||||
|
||||
### 3. Test Implementation
|
||||
- Write tests following TDD/BDD approach
|
||||
- Implement Page Object Model for E2E
|
||||
- Create reusable test utilities
|
||||
- Set up test fixtures and factories
|
||||
- Implement API mocking strategies
|
||||
- Add accessibility checks
|
||||
- Configure test runners and reporters
|
||||
|
||||
### 4. Test Execution
|
||||
- Run tests locally during development
|
||||
- Execute in CI/CD pipeline
|
||||
- Parallel execution for speed
|
||||
- Cross-browser/cross-device testing
|
||||
- Performance and load testing
|
||||
- Security scanning
|
||||
- Visual regression checks
|
||||
|
||||
### 5. Test Maintenance
|
||||
- Monitor test execution metrics
|
||||
- Fix flaky tests promptly
|
||||
- Refactor brittle tests
|
||||
- Update tests for feature changes
|
||||
- Archive obsolete tests
|
||||
- Review coverage gaps
|
||||
- Optimize test execution time
|
||||
|
||||
### 6. Quality Reporting
|
||||
- Generate coverage reports
|
||||
- Track quality metrics over time
|
||||
- Report defects with reproduction steps
|
||||
- Communicate test results to stakeholders
|
||||
- Maintain testing dashboard
|
||||
- Conduct retrospectives
|
||||
|
||||
## Decision Framework
|
||||
|
||||
When making testing decisions, consider:
|
||||
|
||||
1. **Risk**: What are the critical paths? Where is failure most costly?
|
||||
2. **ROI**: Which tests provide the most value for effort?
|
||||
3. **Speed**: Fast feedback loop vs comprehensive coverage
|
||||
4. **Maintenance**: Long-term maintainability of tests
|
||||
5. **Confidence**: Does this test catch real bugs?
|
||||
6. **Coverage**: Are we testing the right things?
|
||||
7. **Reliability**: Are tests deterministic and stable?
|
||||
8. **Environment**: Production parity for realistic testing
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Design Test Strategy
|
||||
1. Analyze application architecture
|
||||
2. Identify critical user flows and risk areas
|
||||
3. Define test pyramid distribution (unit/integration/E2E)
|
||||
4. Select testing frameworks and tools
|
||||
5. Establish coverage targets and quality gates
|
||||
6. Plan test data management approach
|
||||
7. Design CI/CD integration strategy
|
||||
|
||||
### Implement Unit Tests
|
||||
1. Set up Vitest or Jest configuration
|
||||
2. Create test file structure mirroring source code
|
||||
3. Write tests following AAA pattern
|
||||
4. Mock external dependencies
|
||||
5. Add snapshot tests where appropriate
|
||||
6. Configure coverage thresholds
|
||||
7. Integrate with CI/CD pipeline
|
||||
|
||||
### Implement E2E Tests
|
||||
1. Set up Playwright or Cypress
|
||||
2. Design Page Object Model architecture
|
||||
3. Create fixtures for common setup
|
||||
4. Write tests for critical user journeys
|
||||
5. Add visual regression checks
|
||||
6. Configure parallel execution
|
||||
7. Set up test result reporting
|
||||
|
||||
### Implement TDD Workflow
|
||||
1. Write failing test that defines expected behavior
|
||||
2. Run test to confirm it fails (RED)
|
||||
3. Implement minimal code to make test pass
|
||||
4. Run test to confirm it passes (GREEN)
|
||||
5. Refactor code while keeping tests green
|
||||
6. Commit test + implementation together
|
||||
7. Repeat for next requirement
|
||||
|
||||
### Perform Accessibility Audit
|
||||
1. Run Lighthouse accessibility audit
|
||||
2. Integrate axe-core in automated tests
|
||||
3. Test keyboard navigation manually
|
||||
4. Test with screen readers (NVDA, JAWS)
|
||||
5. Verify color contrast ratios
|
||||
6. Check ARIA labels and semantic HTML
|
||||
7. Document findings and remediation plan
|
||||
|
||||
### Set Up Visual Regression Testing
|
||||
1. Choose tool (Percy, Chromatic, BackstopJS)
|
||||
2. Identify components/pages to test
|
||||
3. Capture baseline screenshots
|
||||
4. Configure test runs across browsers/viewports
|
||||
5. Integrate with CI/CD pipeline
|
||||
6. Establish review process for visual changes
|
||||
7. Handle dynamic content appropriately
|
||||
|
||||
### Implement Performance Testing
|
||||
1. Define performance requirements (SLAs)
|
||||
2. Choose load testing tool (k6, Artillery)
|
||||
3. Create test scenarios (load, stress, spike)
|
||||
4. Set up test environment (production-like)
|
||||
5. Execute tests and collect metrics
|
||||
6. Analyze results (bottlenecks, limits)
|
||||
7. Report findings and recommendations
|
||||
|
||||
## Best Practices
|
||||
|
||||
- **Test Behavior, Not Implementation**: Focus on what code does, not how
|
||||
- **Independent Tests**: No shared state between tests
|
||||
- **Fast Feedback**: Unit tests should run in seconds
|
||||
- **Readable Tests**: Self-documenting test names and structure
|
||||
- **Maintainable Tests**: Page Object Model, fixtures, utilities
|
||||
- **Realistic Tests**: Test against production-like environment
|
||||
- **Coverage Targets**: 80%+ code coverage, 100% critical paths
|
||||
- **Flakiness Zero Tolerance**: Fix or quarantine flaky tests
|
||||
- **Test Data Isolation**: Each test creates its own data
|
||||
- **Continuous Testing**: Run tests on every commit
|
||||
- **Accessibility First**: Include a11y tests from the start
|
||||
- **Security Testing**: Regular dependency audits and penetration tests
|
||||
- **Visual Regression**: Catch unintended UI changes
|
||||
- **Performance Budgets**: Monitor and enforce performance thresholds
|
||||
- **Living Documentation**: Tests as executable specifications
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### AAA Pattern (Arrange-Act-Assert)
|
||||
```typescript
|
||||
describe('UserService', () => {
|
||||
it('should create user with valid data', async () => {
|
||||
// Arrange
|
||||
const userData = { name: 'John', email: 'john@example.com' };
|
||||
const mockDb = vi.fn().mockResolvedValue({ id: 1, ...userData });
|
||||
|
||||
// Act
|
||||
const result = await createUser(userData, mockDb);
|
||||
|
||||
// Assert
|
||||
expect(result).toEqual({ id: 1, name: 'John', email: 'john@example.com' });
|
||||
expect(mockDb).toHaveBeenCalledWith(userData);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Page Object Model (Playwright)
|
||||
```typescript
|
||||
// page-objects/LoginPage.ts
|
||||
export class LoginPage {
|
||||
constructor(private page: Page) {}
|
||||
|
||||
async login(email: string, password: string) {
|
||||
await this.page.fill('[data-testid="email"]', email);
|
||||
await this.page.fill('[data-testid="password"]', password);
|
||||
await this.page.click('[data-testid="login-button"]');
|
||||
}
|
||||
|
||||
async getErrorMessage() {
|
||||
return this.page.textContent('[data-testid="error-message"]');
|
||||
}
|
||||
}
|
||||
|
||||
// tests/login.spec.ts
|
||||
test('should show error for invalid credentials', async ({ page }) => {
|
||||
const loginPage = new LoginPage(page);
|
||||
await loginPage.login('invalid@example.com', 'wrong');
|
||||
expect(await loginPage.getErrorMessage()).toBe('Invalid credentials');
|
||||
});
|
||||
```
|
||||
|
||||
### Factory Pattern for Test Data
|
||||
```typescript
|
||||
// factories/user.factory.ts
|
||||
export class UserFactory {
|
||||
static create(overrides: Partial<User> = {}): User {
|
||||
return {
|
||||
id: faker.datatype.uuid(),
|
||||
name: faker.name.fullName(),
|
||||
email: faker.internet.email(),
|
||||
createdAt: new Date(),
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
static createMany(count: number, overrides: Partial<User> = {}): User[] {
|
||||
return Array.from({ length: count }, () => this.create(overrides));
|
||||
}
|
||||
}
|
||||
|
||||
// Usage in tests
|
||||
const user = UserFactory.create({ email: 'test@example.com' });
|
||||
const users = UserFactory.createMany(5);
|
||||
```
|
||||
|
||||
### Custom Matchers (Vitest)
|
||||
```typescript
|
||||
// test-utils/matchers.ts
|
||||
expect.extend({
|
||||
toBeWithinRange(received: number, floor: number, ceiling: number) {
|
||||
const pass = received >= floor && received <= ceiling;
|
||||
return {
|
||||
pass,
|
||||
message: () => `Expected ${received} to be within range ${floor}-${ceiling}`,
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
// Usage
|
||||
expect(response.time).toBeWithinRange(100, 300);
|
||||
```
|
||||
|
||||
### MSW for API Mocking
|
||||
```typescript
|
||||
// mocks/handlers.ts
|
||||
import { rest } from 'msw';
|
||||
|
||||
export const handlers = [
|
||||
rest.get('/api/users/:id', (req, res, ctx) => {
|
||||
return res(
|
||||
ctx.status(200),
|
||||
ctx.json({ id: req.params.id, name: 'John Doe' })
|
||||
);
|
||||
}),
|
||||
];
|
||||
|
||||
// setup.ts
|
||||
import { setupServer } from 'msw/node';
|
||||
import { handlers } from './mocks/handlers';
|
||||
|
||||
export const server = setupServer(...handlers);
|
||||
|
||||
beforeAll(() => server.listen());
|
||||
afterEach(() => server.resetHandlers());
|
||||
afterAll(() => server.close());
|
||||
```
|
||||
|
||||
### Accessibility Testing
|
||||
```typescript
|
||||
import { expect, test } from '@playwright/test';
|
||||
import AxeBuilder from '@axe-core/playwright';
|
||||
|
||||
test('should not have accessibility violations', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
|
||||
const accessibilityScanResults = await new AxeBuilder({ page })
|
||||
.withTags(['wcag2a', 'wcag2aa'])
|
||||
.analyze();
|
||||
|
||||
expect(accessibilityScanResults.violations).toEqual([]);
|
||||
});
|
||||
```
|
||||
|
||||
### Property-Based Testing
|
||||
```typescript
|
||||
import fc from 'fast-check';
|
||||
|
||||
describe('sortNumbers', () => {
|
||||
it('should sort any array of numbers', () => {
|
||||
fc.assert(
|
||||
fc.property(fc.array(fc.integer()), (numbers) => {
|
||||
const sorted = sortNumbers(numbers);
|
||||
|
||||
// Check sorted array is same length
|
||||
expect(sorted.length).toBe(numbers.length);
|
||||
|
||||
// Check elements are in order
|
||||
for (let i = 1; i < sorted.length; i++) {
|
||||
expect(sorted[i]).toBeGreaterThanOrEqual(sorted[i - 1]);
|
||||
}
|
||||
})
|
||||
);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## Testing Anti-Patterns to Avoid
|
||||
|
||||
**❌ Testing Implementation Details**:
|
||||
```typescript
|
||||
// BAD: Testing internal state
|
||||
expect(component.state.isLoading).toBe(true);
|
||||
|
||||
// GOOD: Testing user-visible behavior
|
||||
expect(screen.getByText('Loading...')).toBeInTheDocument();
|
||||
```
|
||||
|
||||
**❌ Tests with Shared State**:
|
||||
```typescript
|
||||
// BAD: Shared state between tests
|
||||
let user;
|
||||
beforeAll(() => { user = createUser(); });
|
||||
|
||||
// GOOD: Each test creates its own data
|
||||
beforeEach(() => { user = createUser(); });
|
||||
```
|
||||
|
||||
**❌ Hardcoded Waits**:
|
||||
```typescript
|
||||
// BAD: Arbitrary wait
|
||||
await page.waitForTimeout(3000);
|
||||
|
||||
// GOOD: Wait for specific condition
|
||||
await page.waitForSelector('[data-testid="result"]');
|
||||
```
|
||||
|
||||
**❌ Over-Mocking**:
|
||||
```typescript
|
||||
// BAD: Mocking everything
|
||||
vi.mock('./database');
|
||||
vi.mock('./api');
|
||||
vi.mock('./utils');
|
||||
|
||||
// GOOD: Mock only external dependencies
|
||||
// Test real integration when possible
|
||||
```
|
||||
|
||||
**❌ Generic Test Names**:
|
||||
```typescript
|
||||
// BAD: Unclear test name
|
||||
it('works correctly', () => { ... });
|
||||
|
||||
// GOOD: Descriptive test name
|
||||
it('should return 404 when user not found', () => { ... });
|
||||
```
|
||||
|
||||
## Quality Checklists
|
||||
|
||||
### Pre-Merge Checklist
|
||||
- [ ] All tests passing locally
|
||||
- [ ] Coverage meets thresholds (80%+ lines)
|
||||
- [ ] No new accessibility violations
|
||||
- [ ] Visual regression tests reviewed
|
||||
- [ ] E2E tests pass for critical paths
|
||||
- [ ] Performance budgets not exceeded
|
||||
- [ ] Security audit passes (no critical vulnerabilities)
|
||||
- [ ] Flaky tests fixed or quarantined
|
||||
- [ ] Test execution time acceptable
|
||||
|
||||
### Release Checklist
|
||||
- [ ] Full regression test suite passes
|
||||
- [ ] Cross-browser tests pass (Chrome, Firefox, Safari)
|
||||
- [ ] Mobile/responsive tests pass
|
||||
- [ ] Performance testing completed
|
||||
- [ ] Load/stress testing completed
|
||||
- [ ] Security penetration testing completed
|
||||
- [ ] Accessibility audit completed (WCAG AA)
|
||||
- [ ] Visual regression baseline updated
|
||||
- [ ] Monitoring and alerting configured
|
||||
- [ ] Rollback plan tested
|
||||
|
||||
You are ready to ensure world-class quality through comprehensive testing strategies!
|
||||
|
||||
## How to Invoke This Agent
|
||||
|
||||
Use the Task tool with the following subagent type:
|
||||
|
||||
```typescript
|
||||
Task({
|
||||
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
|
||||
prompt: "Your QA/testing task here",
|
||||
description: "Brief task description"
|
||||
})
|
||||
```
|
||||
|
||||
**Example**:
|
||||
```typescript
|
||||
Task({
|
||||
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
|
||||
prompt: "Create a comprehensive test strategy for an e-commerce checkout flow using Playwright E2E and Vitest unit tests",
|
||||
description: "Design test strategy for checkout"
|
||||
})
|
||||
```
|
||||
443
agents/qa-engineer/README.md
Normal file
443
agents/qa-engineer/README.md
Normal file
@@ -0,0 +1,443 @@
|
||||
# QA Engineer Agent
|
||||
|
||||
## Overview
|
||||
|
||||
The QA Engineer agent is a specialized AI agent designed to help with test strategy, test planning, test automation, and comprehensive quality assurance for modern software projects.
|
||||
|
||||
## When to Use This Agent
|
||||
|
||||
Invoke this agent when you need:
|
||||
|
||||
- **Test Strategy**: Design comprehensive testing approaches (unit, integration, E2E)
|
||||
- **Test Planning**: Break down features into testable scenarios
|
||||
- **Test Automation**: Set up Playwright, Vitest, Cypress, or other testing frameworks
|
||||
- **TDD Workflow**: Implement test-driven development practices
|
||||
- **Quality Gates**: Define coverage thresholds and quality metrics
|
||||
- **Performance Testing**: Load testing, stress testing, benchmarking
|
||||
- **Accessibility Testing**: WCAG compliance, a11y automation
|
||||
- **Security Testing**: Vulnerability scanning, penetration testing
|
||||
- **CI/CD Integration**: Automated testing in pipelines
|
||||
- **Test Maintenance**: Refactor flaky tests, improve test reliability
|
||||
|
||||
## How to Invoke
|
||||
|
||||
Use the Task tool with `subagent_type`:
|
||||
|
||||
```typescript
|
||||
Task({
|
||||
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
|
||||
prompt: "Design a comprehensive test strategy for a React e-commerce application with Vitest unit tests, Playwright E2E tests, and accessibility testing"
|
||||
});
|
||||
```
|
||||
|
||||
## Agent Capabilities
|
||||
|
||||
### 1. Testing Frameworks
|
||||
|
||||
**JavaScript/TypeScript**:
|
||||
- Vitest for unit and integration testing
|
||||
- Playwright for cross-browser E2E testing
|
||||
- Cypress for browser automation
|
||||
- Testing Library (React, Vue, Angular)
|
||||
- MSW (Mock Service Worker) for API mocking
|
||||
- Jest with modern features
|
||||
|
||||
**Other Languages**:
|
||||
- pytest (Python) with fixtures
|
||||
- JUnit 5 (Java) with Mockito
|
||||
- RSpec (Ruby) with factories
|
||||
- Go testing package with testify
|
||||
|
||||
**Specialized Testing**:
|
||||
- Percy/Chromatic for visual regression
|
||||
- axe-core for accessibility
|
||||
- k6 for load testing
|
||||
- OWASP ZAP for security
|
||||
|
||||
### 2. Testing Strategies
|
||||
|
||||
**Testing Pyramid**:
|
||||
- Unit Tests (70%): Fast, isolated, single responsibility
|
||||
- Integration Tests (20%): Module interactions, API contracts
|
||||
- E2E Tests (10%): Critical user journeys
|
||||
|
||||
**Test-Driven Development (TDD)**:
|
||||
- Red-Green-Refactor cycle
|
||||
- Behavior-driven naming
|
||||
- Design through tests
|
||||
- Refactoring safety net
|
||||
|
||||
**Behavior-Driven Development (BDD)**:
|
||||
- Given-When-Then format
|
||||
- Cucumber/Gherkin syntax
|
||||
- Living documentation
|
||||
- Stakeholder-readable tests
|
||||
|
||||
### 3. Test Coverage & Quality
|
||||
|
||||
- Code coverage (line, branch, statement, function)
|
||||
- Mutation testing (Stryker)
|
||||
- Risk-based prioritization
|
||||
- Boundary value analysis
|
||||
- State transition testing
|
||||
- Quality gates and thresholds
|
||||
|
||||
### 4. Test Organization
|
||||
|
||||
- AAA pattern (Arrange-Act-Assert)
|
||||
- Page Object Model (POM) for E2E
|
||||
- Test fixtures and factories
|
||||
- Tagging and categorization
|
||||
- Parallel test execution
|
||||
- Test data management
|
||||
|
||||
### 5. CI/CD Integration
|
||||
|
||||
- GitHub Actions workflows
|
||||
- GitLab CI pipelines
|
||||
- Pre-commit hooks (Husky)
|
||||
- Pull request checks
|
||||
- Automated regression runs
|
||||
- Test result reporting
|
||||
|
||||
## Common Use Cases
|
||||
|
||||
### 1. Design Test Strategy
|
||||
|
||||
**Prompt**:
|
||||
```
|
||||
"Design a test strategy for a Next.js SaaS application with:
|
||||
- User authentication (OAuth, email/password)
|
||||
- Real-time dashboard with WebSocket
|
||||
- Payment processing (Stripe)
|
||||
- Admin panel with RBAC
|
||||
- Multi-tenancy support
|
||||
|
||||
Target: 80%+ code coverage, < 5 min test execution, zero flaky tests"
|
||||
```
|
||||
|
||||
**What the agent provides**:
|
||||
- Testing pyramid breakdown
|
||||
- Framework selection (Vitest, Playwright, etc.)
|
||||
- Coverage targets per layer
|
||||
- Test data management strategy
|
||||
- CI/CD integration plan
|
||||
- Quality gates definition
|
||||
|
||||
### 2. Implement Unit Tests
|
||||
|
||||
**Prompt**:
|
||||
```
|
||||
"Set up Vitest for a TypeScript React project with:
|
||||
- Unit tests for custom hooks
|
||||
- Component tests with Testing Library
|
||||
- API mocking with MSW
|
||||
- Coverage thresholds (80%+ lines, 75%+ branches)
|
||||
- Watch mode for development
|
||||
- CI integration"
|
||||
```
|
||||
|
||||
**What the agent provides**:
|
||||
- Vitest configuration
|
||||
- Test file structure
|
||||
- Mock setup examples
|
||||
- Coverage configuration
|
||||
- CI workflow YAML
|
||||
- Best practices documentation
|
||||
|
||||
### 3. Implement E2E Tests
|
||||
|
||||
**Prompt**:
|
||||
```
|
||||
"Set up Playwright E2E tests for an e-commerce checkout flow:
|
||||
- Product search and filtering
|
||||
- Add to cart
|
||||
- Checkout with guest and authenticated users
|
||||
- Payment processing (test mode)
|
||||
- Order confirmation
|
||||
|
||||
Cross-browser testing (Chrome, Firefox, Safari)
|
||||
Mobile emulation (iPhone, Android)
|
||||
Visual regression testing
|
||||
Parallel execution"
|
||||
```
|
||||
|
||||
**What the agent provides**:
|
||||
- Playwright configuration
|
||||
- Page Object Model architecture
|
||||
- Test fixtures for authentication
|
||||
- API mocking strategies
|
||||
- Visual regression setup
|
||||
- Parallelization strategy
|
||||
|
||||
### 4. Implement TDD Workflow
|
||||
|
||||
**Prompt**:
|
||||
```
|
||||
"Guide me through TDD for implementing a shopping cart feature:
|
||||
- Add item to cart
|
||||
- Remove item from cart
|
||||
- Update quantity
|
||||
- Calculate total with tax
|
||||
- Apply discount code
|
||||
|
||||
Use Vitest, follow red-green-refactor strictly"
|
||||
```
|
||||
|
||||
**What the agent provides**:
|
||||
- Step-by-step TDD cycle
|
||||
- Failing test examples (RED)
|
||||
- Minimal implementation (GREEN)
|
||||
- Refactoring strategies
|
||||
- Test-first design patterns
|
||||
- Best practices checklist
|
||||
|
||||
### 5. Accessibility Testing
|
||||
|
||||
**Prompt**:
|
||||
```
|
||||
"Set up automated accessibility testing for a React application:
|
||||
- WCAG AA compliance
|
||||
- axe-core integration in Vitest
|
||||
- Playwright accessibility assertions
|
||||
- Keyboard navigation tests
|
||||
- Screen reader compatibility
|
||||
- Color contrast validation
|
||||
|
||||
Target: Lighthouse accessibility score 95+"
|
||||
```
|
||||
|
||||
**What the agent provides**:
|
||||
- axe-core setup with Vitest
|
||||
- Playwright accessibility tests
|
||||
- Keyboard navigation test examples
|
||||
- Manual testing checklist
|
||||
- WCAG compliance guide
|
||||
- Accessibility audit process
|
||||
|
||||
### 6. Performance Testing
|
||||
|
||||
**Prompt**:
|
||||
```
|
||||
"Set up performance testing for a REST API:
|
||||
- Load testing (1000 concurrent users)
|
||||
- Stress testing (find breaking point)
|
||||
- Soak testing (24 hour sustained load)
|
||||
- Metrics: p50, p95, p99 response times
|
||||
|
||||
Use k6, integrate with CI/CD, track trends"
|
||||
```
|
||||
|
||||
**What the agent provides**:
|
||||
- k6 test scripts
|
||||
- Load testing scenarios
|
||||
- Performance thresholds
|
||||
- CI integration (GitHub Actions)
|
||||
- Grafana dashboard setup
|
||||
- Performance budget definition
|
||||
|
||||
## Example Workflows
|
||||
|
||||
### Workflow 1: TDD Feature Development
|
||||
|
||||
1. **Write Failing Test (RED)**: Define expected behavior
|
||||
2. **Implement Minimally (GREEN)**: Make test pass
|
||||
3. **Refactor**: Improve code quality
|
||||
4. **Repeat**: Next requirement
|
||||
|
||||
### Workflow 2: E2E Test Setup
|
||||
|
||||
1. **Install Playwright**: `npm install -D @playwright/test`
|
||||
2. **Configure**: Browser setup, base URL, screenshots
|
||||
3. **Create Page Objects**: Reusable page interactions
|
||||
4. **Write Tests**: Critical user journeys
|
||||
5. **CI Integration**: Run on every PR
|
||||
6. **Monitor**: Track flakiness and execution time
|
||||
|
||||
### Workflow 3: Quality Audit
|
||||
|
||||
1. **Analyze Coverage**: Identify gaps
|
||||
2. **Review Test Quality**: Flaky tests, slow tests
|
||||
3. **Refactor**: Improve reliability
|
||||
4. **Set Thresholds**: Enforce quality gates
|
||||
5. **Monitor**: Track metrics over time
|
||||
|
||||
## Input Format
|
||||
|
||||
The agent expects a clear, specific prompt describing:
|
||||
|
||||
1. **Context**: What are you testing? What technology stack?
|
||||
2. **Requirements**: What features need tests?
|
||||
3. **Quality Targets**: Coverage %, performance, accessibility
|
||||
4. **Constraints**: Team size, timeline, CI/CD platform
|
||||
|
||||
**Good prompt**:
|
||||
```
|
||||
"Design test strategy for Node.js REST API with:
|
||||
- 50+ endpoints (CRUD operations)
|
||||
- Authentication (JWT)
|
||||
- Rate limiting
|
||||
- PostgreSQL database
|
||||
- Redis caching
|
||||
|
||||
Stack: TypeScript, Express, Prisma
|
||||
Target: 85%+ coverage, < 2 min test execution
|
||||
CI: GitHub Actions
|
||||
Team: 5 developers, all familiar with Jest"
|
||||
```
|
||||
|
||||
**Poor prompt**:
|
||||
```
|
||||
"Need tests for my API"
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
The agent provides:
|
||||
|
||||
1. **Test Strategy**: High-level approach and distribution
|
||||
2. **Framework Setup**: Configuration files and installation
|
||||
3. **Test Examples**: Code samples for each test type
|
||||
4. **Best Practices**: Patterns and anti-patterns
|
||||
5. **CI/CD Integration**: Workflow files
|
||||
6. **Quality Metrics**: Coverage targets, thresholds
|
||||
7. **Documentation**: Testing guidelines for team
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Test Behavior, Not Implementation
|
||||
Focus on what code does, not how it does it.
|
||||
|
||||
### 2. Independent Tests
|
||||
No shared state between tests.
|
||||
|
||||
### 3. Fast Feedback
|
||||
Unit tests should run in seconds.
|
||||
|
||||
### 4. Readable Tests
|
||||
Self-documenting test names and structure.
|
||||
|
||||
### 5. Maintainable Tests
|
||||
Page Object Model, fixtures, utilities.
|
||||
|
||||
### 6. Realistic Tests
|
||||
Test against production-like environment.
|
||||
|
||||
### 7. Coverage Targets
|
||||
80%+ code coverage, 100% critical paths.
|
||||
|
||||
### 8. Flakiness Zero Tolerance
|
||||
Fix or quarantine flaky tests immediately.
|
||||
|
||||
## Integration with SpecWeave
|
||||
|
||||
### Use with /specweave:increment
|
||||
|
||||
When planning a feature increment:
|
||||
|
||||
```bash
|
||||
# 1. Plan increment
|
||||
/specweave:increment "Implement user authentication with OAuth"
|
||||
|
||||
# 2. Generate test strategy
|
||||
Task({
|
||||
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
|
||||
prompt: "Design test strategy for OAuth authentication with Google, GitHub, and email/password. Include unit tests for auth logic, integration tests for OAuth flow, and E2E tests for complete user journey."
|
||||
});
|
||||
|
||||
# 3. Implement with TDD
|
||||
/specweave:do
|
||||
```
|
||||
|
||||
### Use with /specweave:qa
|
||||
|
||||
After implementation, validate test coverage:
|
||||
|
||||
```bash
|
||||
# Run quality assessment
|
||||
/specweave:qa 0123
|
||||
|
||||
# Agent checks:
|
||||
# - Test coverage (lines, branches, functions)
|
||||
# - Test quality (independent, fast, readable)
|
||||
# - Critical path coverage
|
||||
# - E2E test completeness
|
||||
# - Accessibility test coverage
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: Flaky tests
|
||||
|
||||
**Solution**:
|
||||
- Use proper wait strategies (no hardcoded delays)
|
||||
- Isolate test data
|
||||
- Mock external dependencies
|
||||
- Use fake timers for time-based code
|
||||
|
||||
### Issue: Slow tests
|
||||
|
||||
**Solution**:
|
||||
- Mock expensive operations (DB, API, file I/O)
|
||||
- Use fake timers
|
||||
- Run tests in parallel
|
||||
- Profile slow tests
|
||||
|
||||
### Issue: Low coverage
|
||||
|
||||
**Solution**:
|
||||
- Identify untested paths
|
||||
- Add tests for edge cases
|
||||
- Test error handling
|
||||
- Use mutation testing to find weak tests
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Contract Testing
|
||||
|
||||
```typescript
|
||||
Task({
|
||||
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
|
||||
prompt: `Set up Pact contract testing between:
|
||||
- Frontend (React SPA)
|
||||
- Backend API (Node.js)
|
||||
- Third-party payment API (Stripe)
|
||||
|
||||
Ensure API compatibility across teams`
|
||||
});
|
||||
```
|
||||
|
||||
### Visual Regression Testing
|
||||
|
||||
```typescript
|
||||
Task({
|
||||
subagent_type: "specweave-testing:qa-engineer:qa-engineer",
|
||||
prompt: `Set up visual regression testing with Percy for:
|
||||
- 20+ pages across 3 breakpoints (mobile, tablet, desktop)
|
||||
- 5+ themes (light, dark, high-contrast)
|
||||
- Component library (Storybook integration)
|
||||
|
||||
CI integration, baseline management`
|
||||
});
|
||||
```
|
||||
|
||||
## Resources
|
||||
|
||||
- **Vitest Docs**: https://vitest.dev
|
||||
- **Playwright Docs**: https://playwright.dev
|
||||
- **Testing Library**: https://testing-library.com
|
||||
- **TDD Guide**: https://martinfowler.com/bliki/TestDrivenDevelopment.html
|
||||
- **Testing Best Practices**: https://testingjavascript.com
|
||||
|
||||
## Version History
|
||||
|
||||
- **v1.0.0** (2025-01-22): Initial release with Vitest, Playwright, TDD expertise
|
||||
- **v1.1.0** (TBD): Add mobile testing support (Appium, Detox)
|
||||
- **v1.2.0** (TBD): Add chaos engineering guidance
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
- GitHub Issues: https://github.com/anton-abyzov/specweave/issues
|
||||
- Discord: https://discord.gg/specweave
|
||||
- Documentation: https://spec-weave.com
|
||||
470
agents/qa-engineer/templates/playwright-e2e-test.ts
Normal file
470
agents/qa-engineer/templates/playwright-e2e-test.ts
Normal file
@@ -0,0 +1,470 @@
|
||||
/**
|
||||
* Playwright E2E Test Template
|
||||
*
|
||||
* This template demonstrates best practices for writing end-to-end tests
|
||||
* with Playwright for web applications.
|
||||
*
|
||||
* Features:
|
||||
* - Page Object Model (POM)
|
||||
* - Test fixtures
|
||||
* - Cross-browser testing
|
||||
* - Mobile emulation
|
||||
* - API mocking
|
||||
* - Visual regression
|
||||
* - Accessibility testing
|
||||
*/
|
||||
|
||||
import { test, expect, Page } from '@playwright/test';
|
||||
import AxeBuilder from '@axe-core/playwright';
|
||||
|
||||
// ============================================================================
|
||||
// PAGE OBJECTS
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Login Page Object
|
||||
*
|
||||
* Encapsulates all interactions with the login page
|
||||
*/
|
||||
class LoginPage {
|
||||
constructor(private page: Page) {}
|
||||
|
||||
// Locators
|
||||
get emailInput() {
|
||||
return this.page.getByLabel('Email');
|
||||
}
|
||||
|
||||
get passwordInput() {
|
||||
return this.page.getByLabel('Password');
|
||||
}
|
||||
|
||||
get loginButton() {
|
||||
return this.page.getByRole('button', { name: 'Login' });
|
||||
}
|
||||
|
||||
get errorMessage() {
|
||||
return this.page.getByRole('alert');
|
||||
}
|
||||
|
||||
// Actions
|
||||
async goto() {
|
||||
await this.page.goto('/login');
|
||||
}
|
||||
|
||||
async login(email: string, password: string) {
|
||||
await this.emailInput.fill(email);
|
||||
await this.passwordInput.fill(password);
|
||||
await this.loginButton.click();
|
||||
}
|
||||
|
||||
async expectError(message: string) {
|
||||
await expect(this.errorMessage).toContainText(message);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Dashboard Page Object
|
||||
*/
|
||||
class DashboardPage {
|
||||
constructor(private page: Page) {}
|
||||
|
||||
get welcomeMessage() {
|
||||
return this.page.getByText(/Welcome,/);
|
||||
}
|
||||
|
||||
get logoutButton() {
|
||||
return this.page.getByRole('button', { name: 'Logout' });
|
||||
}
|
||||
|
||||
async expectLoggedIn(username: string) {
|
||||
await expect(this.welcomeMessage).toContainText(username);
|
||||
}
|
||||
|
||||
async logout() {
|
||||
await this.logoutButton.click();
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// TEST FIXTURES
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Custom fixture that provides authenticated page
|
||||
*/
|
||||
const test = base.extend<{ authenticatedPage: Page }>({
|
||||
authenticatedPage: async ({ page }, use) => {
|
||||
// Setup: Login before test
|
||||
const loginPage = new LoginPage(page);
|
||||
await loginPage.goto();
|
||||
await loginPage.login('user@example.com', 'password123');
|
||||
|
||||
// Wait for navigation to dashboard
|
||||
await page.waitForURL('/dashboard');
|
||||
|
||||
// Provide authenticated page to test
|
||||
await use(page);
|
||||
|
||||
// Teardown: Logout after test
|
||||
const dashboardPage = new DashboardPage(page);
|
||||
await dashboardPage.logout();
|
||||
},
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// BASIC E2E TESTS
|
||||
// ============================================================================
|
||||
|
||||
test.describe('Authentication Flow', () => {
|
||||
test('should login successfully with valid credentials', async ({ page }) => {
|
||||
// ARRANGE
|
||||
const loginPage = new LoginPage(page);
|
||||
await loginPage.goto();
|
||||
|
||||
// ACT
|
||||
await loginPage.login('user@example.com', 'password123');
|
||||
|
||||
// ASSERT
|
||||
await expect(page).toHaveURL('/dashboard');
|
||||
const dashboardPage = new DashboardPage(page);
|
||||
await dashboardPage.expectLoggedIn('User');
|
||||
});
|
||||
|
||||
test('should show error for invalid credentials', async ({ page }) => {
|
||||
// ARRANGE
|
||||
const loginPage = new LoginPage(page);
|
||||
await loginPage.goto();
|
||||
|
||||
// ACT
|
||||
await loginPage.login('invalid@example.com', 'wrongpassword');
|
||||
|
||||
// ASSERT
|
||||
await loginPage.expectError('Invalid email or password');
|
||||
await expect(page).toHaveURL('/login'); // Still on login page
|
||||
});
|
||||
|
||||
test('should show validation errors for empty fields', async ({ page }) => {
|
||||
// ARRANGE
|
||||
const loginPage = new LoginPage(page);
|
||||
await loginPage.goto();
|
||||
|
||||
// ACT
|
||||
await loginPage.loginButton.click();
|
||||
|
||||
// ASSERT
|
||||
await expect(page.getByText('Email is required')).toBeVisible();
|
||||
await expect(page.getByText('Password is required')).toBeVisible();
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// AUTHENTICATED TESTS (Using Fixture)
|
||||
// ============================================================================
|
||||
|
||||
test.describe('Dashboard Features', () => {
|
||||
test('should display user profile', async ({ authenticatedPage }) => {
|
||||
// Navigate to profile
|
||||
await authenticatedPage.goto('/profile');
|
||||
|
||||
// Verify profile data
|
||||
await expect(
|
||||
authenticatedPage.getByText('user@example.com')
|
||||
).toBeVisible();
|
||||
await expect(authenticatedPage.getByText('Member since')).toBeVisible();
|
||||
});
|
||||
|
||||
test('should allow editing profile', async ({ authenticatedPage }) => {
|
||||
// Navigate to profile
|
||||
await authenticatedPage.goto('/profile');
|
||||
|
||||
// Edit name
|
||||
const nameInput = authenticatedPage.getByLabel('Name');
|
||||
await nameInput.clear();
|
||||
await nameInput.fill('New Name');
|
||||
|
||||
// Save changes
|
||||
await authenticatedPage
|
||||
.getByRole('button', { name: 'Save Changes' })
|
||||
.click();
|
||||
|
||||
// Verify success
|
||||
await expect(
|
||||
authenticatedPage.getByText('Profile updated successfully')
|
||||
).toBeVisible();
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// API MOCKING
|
||||
// ============================================================================
|
||||
|
||||
test.describe('API Mocking', () => {
|
||||
test('should handle API errors gracefully', async ({ page }) => {
|
||||
// Mock API to return error
|
||||
await page.route('**/api/users', (route) => {
|
||||
route.fulfill({
|
||||
status: 500,
|
||||
body: JSON.stringify({ error: 'Internal Server Error' }),
|
||||
});
|
||||
});
|
||||
|
||||
// Navigate to users page
|
||||
await page.goto('/users');
|
||||
|
||||
// Verify error message
|
||||
await expect(
|
||||
page.getByText('Failed to load users. Please try again.')
|
||||
).toBeVisible();
|
||||
});
|
||||
|
||||
test('should display mocked user data', async ({ page }) => {
|
||||
// Mock API to return test data
|
||||
await page.route('**/api/users', (route) => {
|
||||
route.fulfill({
|
||||
status: 200,
|
||||
body: JSON.stringify([
|
||||
{ id: 1, name: 'John Doe', email: 'john@example.com' },
|
||||
{ id: 2, name: 'Jane Smith', email: 'jane@example.com' },
|
||||
]),
|
||||
});
|
||||
});
|
||||
|
||||
// Navigate to users page
|
||||
await page.goto('/users');
|
||||
|
||||
// Verify mocked data is displayed
|
||||
await expect(page.getByText('John Doe')).toBeVisible();
|
||||
await expect(page.getByText('Jane Smith')).toBeVisible();
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// VISUAL REGRESSION TESTING
|
||||
// ============================================================================
|
||||
|
||||
test.describe('Visual Regression', () => {
|
||||
test('homepage matches baseline', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
|
||||
// Capture screenshot and compare to baseline
|
||||
await expect(page).toHaveScreenshot('homepage.png', {
|
||||
fullPage: true,
|
||||
animations: 'disabled',
|
||||
});
|
||||
});
|
||||
|
||||
test('button states match baseline', async ({ page }) => {
|
||||
await page.goto('/components');
|
||||
|
||||
const button = page.getByRole('button', { name: 'Submit' });
|
||||
|
||||
// Default state
|
||||
await expect(button).toHaveScreenshot('button-default.png');
|
||||
|
||||
// Hover state
|
||||
await button.hover();
|
||||
await expect(button).toHaveScreenshot('button-hover.png');
|
||||
|
||||
// Focus state
|
||||
await button.focus();
|
||||
await expect(button).toHaveScreenshot('button-focus.png');
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// ACCESSIBILITY TESTING
|
||||
// ============================================================================
|
||||
|
||||
test.describe('Accessibility', () => {
|
||||
test('should not have accessibility violations', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
|
||||
// Run axe accessibility scan
|
||||
const accessibilityScanResults = await new AxeBuilder({ page })
|
||||
.withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
|
||||
.analyze();
|
||||
|
||||
// Assert no violations
|
||||
expect(accessibilityScanResults.violations).toEqual([]);
|
||||
});
|
||||
|
||||
test('should support keyboard navigation', async ({ page }) => {
|
||||
await page.goto('/form');
|
||||
|
||||
// Tab through form fields
|
||||
await page.keyboard.press('Tab');
|
||||
await expect(page.getByLabel('Email')).toBeFocused();
|
||||
|
||||
await page.keyboard.press('Tab');
|
||||
await expect(page.getByLabel('Password')).toBeFocused();
|
||||
|
||||
await page.keyboard.press('Tab');
|
||||
await expect(
|
||||
page.getByRole('button', { name: 'Submit' })
|
||||
).toBeFocused();
|
||||
|
||||
// Submit with Enter
|
||||
await page.keyboard.press('Enter');
|
||||
});
|
||||
|
||||
test('should have proper ARIA labels', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
|
||||
// Check navigation has aria-label
|
||||
await expect(
|
||||
page.getByRole('navigation', { name: 'Main navigation' })
|
||||
).toBeVisible();
|
||||
|
||||
// Check main content has aria-label
|
||||
await expect(page.getByRole('main')).toHaveAttribute(
|
||||
'aria-label',
|
||||
'Main content'
|
||||
);
|
||||
|
||||
// Check all images have alt text
|
||||
const images = await page.getByRole('img').all();
|
||||
for (const img of images) {
|
||||
await expect(img).toHaveAttribute('alt');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// MOBILE TESTING
|
||||
// ============================================================================
|
||||
|
||||
test.describe('Mobile Experience', () => {
|
||||
test.use({ viewport: { width: 375, height: 667 } }); // iPhone SE
|
||||
|
||||
test('should render mobile navigation', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
|
||||
// Mobile menu button should be visible
|
||||
await expect(
|
||||
page.getByRole('button', { name: 'Menu' })
|
||||
).toBeVisible();
|
||||
|
||||
// Desktop navigation should be hidden
|
||||
const desktopNav = page.getByRole('navigation').first();
|
||||
await expect(desktopNav).toBeHidden();
|
||||
});
|
||||
|
||||
test('should handle touch gestures', async ({ page }) => {
|
||||
await page.goto('/gallery');
|
||||
|
||||
// Get image element
|
||||
const image = page.getByRole('img').first();
|
||||
|
||||
// Swipe left
|
||||
await image.dispatchEvent('touchstart', {
|
||||
touches: [{ clientX: 300, clientY: 200 }],
|
||||
});
|
||||
await image.dispatchEvent('touchmove', {
|
||||
touches: [{ clientX: 100, clientY: 200 }],
|
||||
});
|
||||
await image.dispatchEvent('touchend');
|
||||
|
||||
// Verify navigation to next image
|
||||
await expect(page.getByText('Image 2 of 10')).toBeVisible();
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// PERFORMANCE TESTING
|
||||
// ============================================================================
|
||||
|
||||
test.describe('Performance', () => {
|
||||
test('page load performance', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
|
||||
// Get performance metrics
|
||||
const performanceMetrics = await page.evaluate(() => {
|
||||
const perfData = window.performance.timing;
|
||||
return {
|
||||
loadTime: perfData.loadEventEnd - perfData.navigationStart,
|
||||
domContentLoaded:
|
||||
perfData.domContentLoadedEventEnd - perfData.navigationStart,
|
||||
};
|
||||
});
|
||||
|
||||
// Assert performance targets
|
||||
expect(performanceMetrics.loadTime).toBeLessThan(3000); // 3s max
|
||||
expect(performanceMetrics.domContentLoaded).toBeLessThan(2000); // 2s max
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// FILE UPLOAD/DOWNLOAD
|
||||
// ============================================================================
|
||||
|
||||
test.describe('File Operations', () => {
|
||||
test('should upload file', async ({ page }) => {
|
||||
await page.goto('/upload');
|
||||
|
||||
// Set up file chooser
|
||||
const fileChooserPromise = page.waitForEvent('filechooser');
|
||||
await page.getByRole('button', { name: 'Upload File' }).click();
|
||||
const fileChooser = await fileChooserPromise;
|
||||
|
||||
// Upload file
|
||||
await fileChooser.setFiles('tests/fixtures/test-file.pdf');
|
||||
|
||||
// Verify upload success
|
||||
await expect(page.getByText('File uploaded successfully')).toBeVisible();
|
||||
});
|
||||
|
||||
test('should download file', async ({ page }) => {
|
||||
await page.goto('/downloads');
|
||||
|
||||
// Set up download
|
||||
const downloadPromise = page.waitForEvent('download');
|
||||
await page.getByRole('link', { name: 'Download Report' }).click();
|
||||
const download = await downloadPromise;
|
||||
|
||||
// Verify download
|
||||
expect(download.suggestedFilename()).toBe('report.pdf');
|
||||
await download.saveAs(`/tmp/${download.suggestedFilename()}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// MULTI-TAB/WINDOW TESTING
|
||||
// ============================================================================
|
||||
|
||||
test.describe('Multi-Window', () => {
|
||||
test('should handle popup windows', async ({ context, page }) => {
|
||||
await page.goto('/');
|
||||
|
||||
// Wait for popup
|
||||
const [popup] = await Promise.all([
|
||||
context.waitForEvent('page'),
|
||||
page.getByRole('button', { name: 'Open Help' }).click(),
|
||||
]);
|
||||
|
||||
// Interact with popup
|
||||
await expect(popup.getByText('Help Center')).toBeVisible();
|
||||
await popup.close();
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// BEST PRACTICES CHECKLIST
|
||||
// ============================================================================
|
||||
|
||||
/*
|
||||
✅ Page Object Model (POM)
|
||||
✅ Test Fixtures for Setup/Teardown
|
||||
✅ Descriptive Test Names
|
||||
✅ Auto-Waiting (Playwright built-in)
|
||||
✅ User-Centric Selectors (getByRole, getByLabel)
|
||||
✅ API Mocking for Reliability
|
||||
✅ Visual Regression Testing
|
||||
✅ Accessibility Testing (axe-core)
|
||||
✅ Mobile/Responsive Testing
|
||||
✅ Performance Assertions
|
||||
✅ File Upload/Download
|
||||
✅ Multi-Tab/Window Handling
|
||||
✅ Screenshot/Video on Failure (configured in playwright.config.ts)
|
||||
✅ Parallel Execution (configured in playwright.config.ts)
|
||||
✅ Cross-Browser Testing (configured in playwright.config.ts)
|
||||
*/
|
||||
507
agents/qa-engineer/templates/test-data-factory.ts
Normal file
507
agents/qa-engineer/templates/test-data-factory.ts
Normal file
@@ -0,0 +1,507 @@
|
||||
/**
|
||||
* Test Data Factory Template
|
||||
*
|
||||
* This template demonstrates best practices for creating reusable
|
||||
* test data factories using the Factory pattern.
|
||||
*
|
||||
* Benefits:
|
||||
* - Consistent test data generation
|
||||
* - Easy to customize with overrides
|
||||
* - Reduces test setup boilerplate
|
||||
* - Type-safe with TypeScript
|
||||
*/
|
||||
|
||||
import { faker } from '@faker-js/faker';
|
||||
|
||||
// ============================================================================
|
||||
// TYPES
|
||||
// ============================================================================
|
||||
|
||||
export interface User {
|
||||
id: string;
|
||||
email: string;
|
||||
username: string;
|
||||
firstName: string;
|
||||
lastName: string;
|
||||
role: 'admin' | 'user' | 'guest';
|
||||
isActive: boolean;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
profile?: UserProfile;
|
||||
}
|
||||
|
||||
export interface UserProfile {
|
||||
bio: string;
|
||||
avatar: string;
|
||||
phoneNumber: string;
|
||||
address: Address;
|
||||
}
|
||||
|
||||
export interface Address {
|
||||
street: string;
|
||||
city: string;
|
||||
state: string;
|
||||
zipCode: string;
|
||||
country: string;
|
||||
}
|
||||
|
||||
export interface Product {
|
||||
id: string;
|
||||
name: string;
|
||||
description: string;
|
||||
price: number;
|
||||
category: string;
|
||||
inStock: boolean;
|
||||
quantity: number;
|
||||
imageUrl: string;
|
||||
createdAt: Date;
|
||||
}
|
||||
|
||||
export interface Order {
|
||||
id: string;
|
||||
userId: string;
|
||||
items: OrderItem[];
|
||||
subtotal: number;
|
||||
tax: number;
|
||||
total: number;
|
||||
status: 'pending' | 'processing' | 'shipped' | 'delivered' | 'cancelled';
|
||||
shippingAddress: Address;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
}
|
||||
|
||||
export interface OrderItem {
|
||||
productId: string;
|
||||
quantity: number;
|
||||
price: number;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// FACTORY FUNCTIONS
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* User Factory
|
||||
*
|
||||
* Creates realistic user test data with sensible defaults
|
||||
*/
|
||||
export class UserFactory {
|
||||
/**
|
||||
* Create a single user
|
||||
*
|
||||
* @param overrides - Partial user object to override defaults
|
||||
* @returns Complete user object
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const admin = UserFactory.create({ role: 'admin' });
|
||||
* const inactiveUser = UserFactory.create({ isActive: false });
|
||||
* ```
|
||||
*/
|
||||
static create(overrides: Partial<User> = {}): User {
|
||||
const firstName = faker.person.firstName();
|
||||
const lastName = faker.person.lastName();
|
||||
const email =
|
||||
overrides.email || faker.internet.email({ firstName, lastName });
|
||||
|
||||
return {
|
||||
id: faker.string.uuid(),
|
||||
email,
|
||||
username: faker.internet.userName({ firstName, lastName }),
|
||||
firstName,
|
||||
lastName,
|
||||
role: 'user',
|
||||
isActive: true,
|
||||
createdAt: faker.date.past(),
|
||||
updatedAt: faker.date.recent(),
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Create multiple users
|
||||
*
|
||||
* @param count - Number of users to create
|
||||
* @param overrides - Partial user object to override defaults for all users
|
||||
* @returns Array of user objects
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const users = UserFactory.createMany(5);
|
||||
* const admins = UserFactory.createMany(3, { role: 'admin' });
|
||||
* ```
|
||||
*/
|
||||
static createMany(count: number, overrides: Partial<User> = {}): User[] {
|
||||
return Array.from({ length: count }, () => this.create(overrides));
|
||||
}
|
||||
|
||||
/**
|
||||
* Create admin user
|
||||
*
|
||||
* @param overrides - Partial user object to override defaults
|
||||
* @returns Admin user object
|
||||
*/
|
||||
static createAdmin(overrides: Partial<User> = {}): User {
|
||||
return this.create({
|
||||
role: 'admin',
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Create user with complete profile
|
||||
*
|
||||
* @param overrides - Partial user object to override defaults
|
||||
* @returns User with profile object
|
||||
*/
|
||||
static createWithProfile(overrides: Partial<User> = {}): User {
|
||||
return this.create({
|
||||
profile: {
|
||||
bio: faker.person.bio(),
|
||||
avatar: faker.image.avatar(),
|
||||
phoneNumber: faker.phone.number(),
|
||||
address: AddressFactory.create(),
|
||||
},
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Create inactive user
|
||||
*
|
||||
* @param overrides - Partial user object to override defaults
|
||||
* @returns Inactive user object
|
||||
*/
|
||||
static createInactive(overrides: Partial<User> = {}): User {
|
||||
return this.create({
|
||||
isActive: false,
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Address Factory
|
||||
*
|
||||
* Creates realistic address test data
|
||||
*/
|
||||
export class AddressFactory {
|
||||
static create(overrides: Partial<Address> = {}): Address {
|
||||
return {
|
||||
street: faker.location.streetAddress(),
|
||||
city: faker.location.city(),
|
||||
state: faker.location.state(),
|
||||
zipCode: faker.location.zipCode(),
|
||||
country: faker.location.country(),
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
static createUS(overrides: Partial<Address> = {}): Address {
|
||||
return this.create({
|
||||
country: 'United States',
|
||||
zipCode: faker.location.zipCode('#####'),
|
||||
state: faker.location.state({ abbreviated: true }),
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Product Factory
|
||||
*
|
||||
* Creates realistic product test data
|
||||
*/
|
||||
export class ProductFactory {
|
||||
static create(overrides: Partial<Product> = {}): Product {
|
||||
return {
|
||||
id: faker.string.uuid(),
|
||||
name: faker.commerce.productName(),
|
||||
description: faker.commerce.productDescription(),
|
||||
price: parseFloat(faker.commerce.price()),
|
||||
category: faker.commerce.department(),
|
||||
inStock: true,
|
||||
quantity: faker.number.int({ min: 0, max: 100 }),
|
||||
imageUrl: faker.image.url(),
|
||||
createdAt: faker.date.past(),
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
static createMany(count: number, overrides: Partial<Product> = {}): Product[] {
|
||||
return Array.from({ length: count }, () => this.create(overrides));
|
||||
}
|
||||
|
||||
static createOutOfStock(overrides: Partial<Product> = {}): Product {
|
||||
return this.create({
|
||||
inStock: false,
|
||||
quantity: 0,
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
|
||||
static createExpensive(overrides: Partial<Product> = {}): Product {
|
||||
return this.create({
|
||||
price: faker.number.int({ min: 1000, max: 10000 }),
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Order Factory
|
||||
*
|
||||
* Creates realistic order test data with items
|
||||
*/
|
||||
export class OrderFactory {
|
||||
static create(overrides: Partial<Order> = {}): Order {
|
||||
const items = overrides.items || [
|
||||
{
|
||||
productId: faker.string.uuid(),
|
||||
quantity: faker.number.int({ min: 1, max: 5 }),
|
||||
price: parseFloat(faker.commerce.price()),
|
||||
},
|
||||
];
|
||||
|
||||
const subtotal = items.reduce((sum, item) => sum + item.price * item.quantity, 0);
|
||||
const tax = subtotal * 0.08; // 8% tax
|
||||
const total = subtotal + tax;
|
||||
|
||||
return {
|
||||
id: faker.string.uuid(),
|
||||
userId: faker.string.uuid(),
|
||||
items,
|
||||
subtotal,
|
||||
tax,
|
||||
total,
|
||||
status: 'pending',
|
||||
shippingAddress: AddressFactory.createUS(),
|
||||
createdAt: faker.date.recent(),
|
||||
updatedAt: faker.date.recent(),
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
static createMany(count: number, overrides: Partial<Order> = {}): Order[] {
|
||||
return Array.from({ length: count }, () => this.create(overrides));
|
||||
}
|
||||
|
||||
static createWithItems(items: OrderItem[], overrides: Partial<Order> = {}): Order {
|
||||
return this.create({ items, ...overrides });
|
||||
}
|
||||
|
||||
static createShipped(overrides: Partial<Order> = {}): Order {
|
||||
return this.create({
|
||||
status: 'shipped',
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
|
||||
static createCancelled(overrides: Partial<Order> = {}): Order {
|
||||
return this.create({
|
||||
status: 'cancelled',
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// BUILDER PATTERN (Advanced)
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* User Builder
|
||||
*
|
||||
* Provides a fluent interface for building complex user objects
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const user = new UserBuilder()
|
||||
* .withEmail('admin@example.com')
|
||||
* .withRole('admin')
|
||||
* .withProfile()
|
||||
* .build();
|
||||
* ```
|
||||
*/
|
||||
export class UserBuilder {
|
||||
private user: Partial<User> = {};
|
||||
|
||||
withId(id: string): this {
|
||||
this.user.id = id;
|
||||
return this;
|
||||
}
|
||||
|
||||
withEmail(email: string): this {
|
||||
this.user.email = email;
|
||||
return this;
|
||||
}
|
||||
|
||||
withUsername(username: string): this {
|
||||
this.user.username = username;
|
||||
return this;
|
||||
}
|
||||
|
||||
withName(firstName: string, lastName: string): this {
|
||||
this.user.firstName = firstName;
|
||||
this.user.lastName = lastName;
|
||||
return this;
|
||||
}
|
||||
|
||||
withRole(role: User['role']): this {
|
||||
this.user.role = role;
|
||||
return this;
|
||||
}
|
||||
|
||||
withProfile(profile?: UserProfile): this {
|
||||
this.user.profile = profile || {
|
||||
bio: faker.person.bio(),
|
||||
avatar: faker.image.avatar(),
|
||||
phoneNumber: faker.phone.number(),
|
||||
address: AddressFactory.create(),
|
||||
};
|
||||
return this;
|
||||
}
|
||||
|
||||
inactive(): this {
|
||||
this.user.isActive = false;
|
||||
return this;
|
||||
}
|
||||
|
||||
active(): this {
|
||||
this.user.isActive = true;
|
||||
return this;
|
||||
}
|
||||
|
||||
build(): User {
|
||||
return UserFactory.create(this.user);
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// USAGE EXAMPLES
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Example: Simple user creation
|
||||
*/
|
||||
export function exampleSimpleUser() {
|
||||
const user = UserFactory.create();
|
||||
const admin = UserFactory.createAdmin();
|
||||
const users = UserFactory.createMany(5);
|
||||
|
||||
return { user, admin, users };
|
||||
}
|
||||
|
||||
/**
|
||||
* Example: Customized user creation
|
||||
*/
|
||||
export function exampleCustomUser() {
|
||||
const user = UserFactory.create({
|
||||
email: 'custom@example.com',
|
||||
role: 'admin',
|
||||
isActive: false,
|
||||
});
|
||||
|
||||
return user;
|
||||
}
|
||||
|
||||
/**
|
||||
* Example: Builder pattern
|
||||
*/
|
||||
export function exampleBuilder() {
|
||||
const user = new UserBuilder()
|
||||
.withEmail('builder@example.com')
|
||||
.withName('John', 'Doe')
|
||||
.withRole('admin')
|
||||
.withProfile()
|
||||
.active()
|
||||
.build();
|
||||
|
||||
return user;
|
||||
}
|
||||
|
||||
/**
|
||||
* Example: Order with products
|
||||
*/
|
||||
export function exampleOrder() {
|
||||
// Create products
|
||||
const products = ProductFactory.createMany(3);
|
||||
|
||||
// Create order items from products
|
||||
const items: OrderItem[] = products.map((product) => ({
|
||||
productId: product.id,
|
||||
quantity: faker.number.int({ min: 1, max: 3 }),
|
||||
price: product.price,
|
||||
}));
|
||||
|
||||
// Create order with items
|
||||
const order = OrderFactory.createWithItems(items);
|
||||
|
||||
return { products, order };
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// TEST USAGE
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Example test using factories
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
|
||||
describe('UserService (using factories)', () => {
|
||||
it('should create user', () => {
|
||||
// ARRANGE
|
||||
const userData = UserFactory.create();
|
||||
|
||||
// ACT
|
||||
const result = userService.create(userData);
|
||||
|
||||
// ASSERT
|
||||
expect(result.id).toBeDefined();
|
||||
expect(result.email).toBe(userData.email);
|
||||
});
|
||||
|
||||
it('should only allow admins to delete users', () => {
|
||||
// ARRANGE
|
||||
const admin = UserFactory.createAdmin();
|
||||
const regularUser = UserFactory.create();
|
||||
|
||||
// ACT & ASSERT
|
||||
expect(() => userService.deleteUser(regularUser.id, admin)).not.toThrow();
|
||||
expect(() => userService.deleteUser(admin.id, regularUser)).toThrow('Unauthorized');
|
||||
});
|
||||
|
||||
it('should calculate order total correctly', () => {
|
||||
// ARRANGE
|
||||
const order = OrderFactory.create({
|
||||
items: [
|
||||
{ productId: '1', quantity: 2, price: 50 },
|
||||
{ productId: '2', quantity: 1, price: 30 },
|
||||
],
|
||||
});
|
||||
|
||||
// ACT
|
||||
const total = orderService.calculateTotal(order);
|
||||
|
||||
// ASSERT
|
||||
expect(total).toBe(140.4); // (50*2 + 30*1) * 1.08 tax
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// BEST PRACTICES
|
||||
// ============================================================================
|
||||
|
||||
/*
|
||||
✅ Use realistic data (faker.js)
|
||||
✅ Provide sensible defaults
|
||||
✅ Allow overrides for customization
|
||||
✅ Type-safe with TypeScript
|
||||
✅ Create helper methods (createAdmin, createInactive, etc.)
|
||||
✅ Builder pattern for complex objects
|
||||
✅ Consistent naming (create, createMany, createWith...)
|
||||
✅ Document with JSDoc
|
||||
✅ Export all factories for reuse
|
||||
✅ Keep factories simple and focused
|
||||
*/
|
||||
400
agents/qa-engineer/templates/vitest-unit-test.ts
Normal file
400
agents/qa-engineer/templates/vitest-unit-test.ts
Normal file
@@ -0,0 +1,400 @@
|
||||
/**
|
||||
* Vitest Unit Test Template
|
||||
*
|
||||
* This template demonstrates best practices for writing unit tests
|
||||
* with Vitest for TypeScript projects.
|
||||
*
|
||||
* Features:
|
||||
* - AAA pattern (Arrange-Act-Assert)
|
||||
* - Test isolation
|
||||
* - Mocking dependencies
|
||||
* - Parametric testing
|
||||
* - Error handling tests
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
|
||||
|
||||
// Import the module under test
|
||||
import { YourModule } from './YourModule';
|
||||
|
||||
// Import dependencies (to be mocked)
|
||||
import { ExternalDependency } from './ExternalDependency';
|
||||
|
||||
// ============================================================================
|
||||
// MOCKS
|
||||
// ============================================================================
|
||||
|
||||
// Mock external dependencies
|
||||
vi.mock('./ExternalDependency', () => ({
|
||||
ExternalDependency: vi.fn().mockImplementation(() => ({
|
||||
fetchData: vi.fn().mockResolvedValue({ data: 'mocked' }),
|
||||
processData: vi.fn(),
|
||||
})),
|
||||
}));
|
||||
|
||||
// ============================================================================
|
||||
// TEST SUITE
|
||||
// ============================================================================
|
||||
|
||||
describe('YourModule', () => {
|
||||
// ========================================================================
|
||||
// SETUP & TEARDOWN
|
||||
// ========================================================================
|
||||
|
||||
let instance: YourModule;
|
||||
let mockDependency: any;
|
||||
|
||||
beforeEach(() => {
|
||||
// ARRANGE: Set up fresh instance for each test
|
||||
mockDependency = new ExternalDependency();
|
||||
instance = new YourModule(mockDependency);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
// TEARDOWN: Clean up mocks
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
// ========================================================================
|
||||
// HAPPY PATH TESTS
|
||||
// ========================================================================
|
||||
|
||||
describe('methodName', () => {
|
||||
it('should perform expected operation with valid input', () => {
|
||||
// ARRANGE
|
||||
const input = 'test-input';
|
||||
const expectedOutput = 'test-output';
|
||||
|
||||
// ACT
|
||||
const result = instance.methodName(input);
|
||||
|
||||
// ASSERT
|
||||
expect(result).toBe(expectedOutput);
|
||||
});
|
||||
|
||||
it('should call dependency with correct parameters', () => {
|
||||
// ARRANGE
|
||||
const input = 'test-input';
|
||||
|
||||
// ACT
|
||||
instance.methodName(input);
|
||||
|
||||
// ASSERT
|
||||
expect(mockDependency.processData).toHaveBeenCalledWith(input);
|
||||
expect(mockDependency.processData).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ========================================================================
|
||||
// ASYNC TESTS
|
||||
// ========================================================================
|
||||
|
||||
describe('asyncMethod', () => {
|
||||
it('should resolve with data on success', async () => {
|
||||
// ARRANGE
|
||||
const mockData = { id: 1, value: 'test' };
|
||||
mockDependency.fetchData.mockResolvedValue(mockData);
|
||||
|
||||
// ACT
|
||||
const result = await instance.asyncMethod();
|
||||
|
||||
// ASSERT
|
||||
expect(result).toEqual(mockData);
|
||||
expect(mockDependency.fetchData).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should handle async errors gracefully', async () => {
|
||||
// ARRANGE
|
||||
const error = new Error('Network error');
|
||||
mockDependency.fetchData.mockRejectedValue(error);
|
||||
|
||||
// ACT & ASSERT
|
||||
await expect(instance.asyncMethod()).rejects.toThrow('Network error');
|
||||
});
|
||||
});
|
||||
|
||||
// ========================================================================
|
||||
// EDGE CASES
|
||||
// ========================================================================
|
||||
|
||||
describe('edge cases', () => {
|
||||
it('should handle empty input', () => {
|
||||
// ACT
|
||||
const result = instance.methodName('');
|
||||
|
||||
// ASSERT
|
||||
expect(result).toBe('');
|
||||
});
|
||||
|
||||
it('should handle null input', () => {
|
||||
// ACT
|
||||
const result = instance.methodName(null as any);
|
||||
|
||||
// ASSERT
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
|
||||
it('should handle undefined input', () => {
|
||||
// ACT
|
||||
const result = instance.methodName(undefined as any);
|
||||
|
||||
// ASSERT
|
||||
expect(result).toBeUndefined();
|
||||
});
|
||||
|
||||
it('should handle very large input', () => {
|
||||
// ARRANGE
|
||||
const largeInput = 'x'.repeat(1000000);
|
||||
|
||||
// ACT
|
||||
const result = instance.methodName(largeInput);
|
||||
|
||||
// ASSERT
|
||||
expect(result).toBeDefined();
|
||||
});
|
||||
});
|
||||
|
||||
// ========================================================================
|
||||
// ERROR HANDLING
|
||||
// ========================================================================
|
||||
|
||||
describe('error handling', () => {
|
||||
it('should throw error for invalid input', () => {
|
||||
// ARRANGE
|
||||
const invalidInput = -1;
|
||||
|
||||
// ACT & ASSERT
|
||||
expect(() => instance.methodName(invalidInput)).toThrow(
|
||||
'Input must be positive'
|
||||
);
|
||||
});
|
||||
|
||||
it('should throw specific error type', () => {
|
||||
// ARRANGE
|
||||
const invalidInput = 'invalid';
|
||||
|
||||
// ACT & ASSERT
|
||||
expect(() => instance.methodName(invalidInput)).toThrow(ValidationError);
|
||||
});
|
||||
|
||||
it('should include error details', () => {
|
||||
// ARRANGE
|
||||
const invalidInput = 'invalid';
|
||||
|
||||
// ACT
|
||||
try {
|
||||
instance.methodName(invalidInput);
|
||||
fail('Expected error to be thrown');
|
||||
} catch (error) {
|
||||
// ASSERT
|
||||
expect(error).toBeInstanceOf(ValidationError);
|
||||
expect(error.message).toContain('invalid');
|
||||
expect(error.code).toBe('INVALID_INPUT');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ========================================================================
|
||||
// PARAMETRIC TESTS (Table-Driven)
|
||||
// ========================================================================
|
||||
|
||||
describe.each([
|
||||
{ input: 1, expected: 2 },
|
||||
{ input: 2, expected: 4 },
|
||||
{ input: 3, expected: 6 },
|
||||
{ input: 4, expected: 8 },
|
||||
])('methodName($input)', ({ input, expected }) => {
|
||||
it(`should return ${expected}`, () => {
|
||||
// ACT
|
||||
const result = instance.methodName(input);
|
||||
|
||||
// ASSERT
|
||||
expect(result).toBe(expected);
|
||||
});
|
||||
});
|
||||
|
||||
// ========================================================================
|
||||
// STATE MANAGEMENT TESTS
|
||||
// ========================================================================
|
||||
|
||||
describe('state management', () => {
|
||||
it('should update internal state correctly', () => {
|
||||
// ARRANGE
|
||||
const initialState = instance.getState();
|
||||
|
||||
// ACT
|
||||
instance.updateState('new-value');
|
||||
|
||||
// ASSERT
|
||||
const newState = instance.getState();
|
||||
expect(newState).not.toBe(initialState);
|
||||
expect(newState).toBe('new-value');
|
||||
});
|
||||
|
||||
it('should emit events on state change', () => {
|
||||
// ARRANGE
|
||||
const eventHandler = vi.fn();
|
||||
instance.on('stateChanged', eventHandler);
|
||||
|
||||
// ACT
|
||||
instance.updateState('new-value');
|
||||
|
||||
// ASSERT
|
||||
expect(eventHandler).toHaveBeenCalledWith('new-value');
|
||||
});
|
||||
});
|
||||
|
||||
// ========================================================================
|
||||
// PERFORMANCE TESTS
|
||||
// ========================================================================
|
||||
|
||||
describe('performance', () => {
|
||||
it('should complete within reasonable time', () => {
|
||||
// ARRANGE
|
||||
const startTime = Date.now();
|
||||
|
||||
// ACT
|
||||
instance.methodName('test');
|
||||
|
||||
// ASSERT
|
||||
const endTime = Date.now();
|
||||
const executionTime = endTime - startTime;
|
||||
expect(executionTime).toBeLessThan(100); // 100ms threshold
|
||||
});
|
||||
|
||||
it('should handle large datasets efficiently', () => {
|
||||
// ARRANGE
|
||||
const largeDataset = Array.from({ length: 10000 }, (_, i) => i);
|
||||
|
||||
// ACT
|
||||
const startTime = Date.now();
|
||||
const result = instance.processBatch(largeDataset);
|
||||
const endTime = Date.now();
|
||||
|
||||
// ASSERT
|
||||
expect(result).toHaveLength(10000);
|
||||
expect(endTime - startTime).toBeLessThan(1000); // 1s threshold
|
||||
});
|
||||
});
|
||||
|
||||
// ========================================================================
|
||||
// SNAPSHOT TESTS
|
||||
// ========================================================================
|
||||
|
||||
describe('snapshots', () => {
|
||||
it('should match snapshot for complex output', () => {
|
||||
// ARRANGE
|
||||
const input = { id: 1, name: 'Test', nested: { value: 42 } };
|
||||
|
||||
// ACT
|
||||
const result = instance.transform(input);
|
||||
|
||||
// ASSERT
|
||||
expect(result).toMatchSnapshot();
|
||||
});
|
||||
|
||||
it('should match inline snapshot for simple output', () => {
|
||||
// ARRANGE
|
||||
const input = 'test';
|
||||
|
||||
// ACT
|
||||
const result = instance.methodName(input);
|
||||
|
||||
// ASSERT
|
||||
expect(result).toMatchInlineSnapshot('"test-output"');
|
||||
});
|
||||
});
|
||||
|
||||
// ========================================================================
|
||||
// TIMER/DEBOUNCE TESTS
|
||||
// ========================================================================
|
||||
|
||||
describe('timers', () => {
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllTimers();
|
||||
});
|
||||
|
||||
it('should debounce function calls', () => {
|
||||
// ARRANGE
|
||||
const callback = vi.fn();
|
||||
const debounced = instance.debounce(callback, 1000);
|
||||
|
||||
// ACT
|
||||
debounced();
|
||||
debounced();
|
||||
debounced();
|
||||
|
||||
// ASSERT (not called yet)
|
||||
expect(callback).not.toHaveBeenCalled();
|
||||
|
||||
// Fast-forward time
|
||||
vi.advanceTimersByTime(1000);
|
||||
|
||||
// ASSERT (called once)
|
||||
expect(callback).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should throttle function calls', () => {
|
||||
// ARRANGE
|
||||
const callback = vi.fn();
|
||||
const throttled = instance.throttle(callback, 1000);
|
||||
|
||||
// ACT
|
||||
throttled(); // Called immediately
|
||||
throttled(); // Ignored (within throttle window)
|
||||
throttled(); // Ignored (within throttle window)
|
||||
|
||||
// ASSERT
|
||||
expect(callback).toHaveBeenCalledTimes(1);
|
||||
|
||||
// Fast-forward time
|
||||
vi.advanceTimersByTime(1000);
|
||||
|
||||
// ACT
|
||||
throttled(); // Called after throttle window
|
||||
|
||||
// ASSERT
|
||||
expect(callback).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// HELPER TYPES & CLASSES (For Examples Above)
|
||||
// ============================================================================
|
||||
|
||||
class ValidationError extends Error {
|
||||
constructor(
|
||||
message: string,
|
||||
public code: string
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'ValidationError';
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// BEST PRACTICES CHECKLIST
|
||||
// ============================================================================
|
||||
|
||||
/*
|
||||
✅ AAA Pattern (Arrange-Act-Assert)
|
||||
✅ Test Isolation (beforeEach creates fresh instance)
|
||||
✅ Descriptive Test Names (should do X when Y)
|
||||
✅ One Assertion Per Test (when possible)
|
||||
✅ Mock External Dependencies
|
||||
✅ Test Happy Path
|
||||
✅ Test Edge Cases (null, undefined, empty, large)
|
||||
✅ Test Error Handling
|
||||
✅ Parametric Tests (test.each)
|
||||
✅ Async Testing (async/await, rejects, resolves)
|
||||
✅ Timer Testing (useFakeTimers)
|
||||
✅ Performance Testing (execution time)
|
||||
✅ Snapshot Testing (complex outputs)
|
||||
✅ No Shared State Between Tests
|
||||
✅ Fast Execution (< 1s per test)
|
||||
*/
|
||||
726
agents/qa-engineer/test-strategies.md
Normal file
726
agents/qa-engineer/test-strategies.md
Normal file
@@ -0,0 +1,726 @@
|
||||
# QA Engineer Agent - Test Strategies
|
||||
|
||||
Comprehensive guide to testing strategies for different application types and scenarios.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Testing Pyramid Strategy](#testing-pyramid-strategy)
|
||||
2. [Testing Trophy Strategy](#testing-trophy-strategy)
|
||||
3. [TDD Red-Green-Refactor](#tdd-red-green-refactor)
|
||||
4. [BDD Given-When-Then](#bdd-given-when-then)
|
||||
5. [API Testing Strategy](#api-testing-strategy)
|
||||
6. [Frontend Testing Strategy](#frontend-testing-strategy)
|
||||
7. [Micro-Services Testing Strategy](#micro-services-testing-strategy)
|
||||
8. [Performance Testing Strategy](#performance-testing-strategy)
|
||||
9. [Security Testing Strategy](#security-testing-strategy)
|
||||
10. [Accessibility Testing Strategy](#accessibility-testing-strategy)
|
||||
|
||||
---
|
||||
|
||||
## Testing Pyramid Strategy
|
||||
|
||||
### Overview
|
||||
|
||||
The Testing Pyramid emphasizes a broad base of fast, cheap unit tests, fewer integration tests, and minimal UI/E2E tests.
|
||||
|
||||
```
|
||||
/\
|
||||
/ \ E2E (10%)
|
||||
/----\
|
||||
/ \ Integration (20%)
|
||||
/--------\
|
||||
/ \ Unit (70%)
|
||||
/--------------\
|
||||
```
|
||||
|
||||
### Distribution
|
||||
|
||||
- **Unit Tests (70%)**: Test individual functions, classes, components in isolation
|
||||
- **Integration Tests (20%)**: Test interactions between modules, APIs, databases
|
||||
- **E2E Tests (10%)**: Test complete user journeys through the UI
|
||||
|
||||
### When to Use
|
||||
|
||||
- Traditional applications with clear layers
|
||||
- Backend services with business logic
|
||||
- Applications where unit tests provide high confidence
|
||||
- Teams prioritizing fast feedback loops
|
||||
|
||||
### Implementation Example
|
||||
|
||||
**Unit Test (70% of suite)**:
|
||||
```typescript
|
||||
// src/utils/cart.test.ts
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { calculateTotal, applyDiscount } from './cart';
|
||||
|
||||
describe('Cart Utils', () => {
|
||||
describe('calculateTotal', () => {
|
||||
it('should sum item prices', () => {
|
||||
const items = [
|
||||
{ id: 1, price: 10 },
|
||||
{ id: 2, price: 20 },
|
||||
];
|
||||
|
||||
expect(calculateTotal(items)).toBe(30);
|
||||
});
|
||||
|
||||
it('should return 0 for empty cart', () => {
|
||||
expect(calculateTotal([])).toBe(0);
|
||||
});
|
||||
|
||||
it('should handle decimal prices', () => {
|
||||
const items = [
|
||||
{ id: 1, price: 10.99 },
|
||||
{ id: 2, price: 20.50 },
|
||||
];
|
||||
|
||||
expect(calculateTotal(items)).toBeCloseTo(31.49, 2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('applyDiscount', () => {
|
||||
it('should apply percentage discount', () => {
|
||||
expect(applyDiscount(100, 'SAVE20', { type: 'percentage', value: 20 })).toBe(80);
|
||||
});
|
||||
|
||||
it('should apply fixed discount', () => {
|
||||
expect(applyDiscount(100, 'SAVE10', { type: 'fixed', value: 10 })).toBe(90);
|
||||
});
|
||||
|
||||
it('should not go below zero', () => {
|
||||
expect(applyDiscount(50, 'SAVE100', { type: 'fixed', value: 100 })).toBe(0);
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Integration Test (20% of suite)**:
|
||||
```typescript
|
||||
// src/api/orders.integration.test.ts
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
|
||||
import { createTestServer } from '../test-utils/server';
|
||||
import { seedDatabase, clearDatabase } from '../test-utils/database';
|
||||
|
||||
describe('Orders API Integration', () => {
|
||||
let server: TestServer;
|
||||
|
||||
beforeEach(async () => {
|
||||
server = await createTestServer();
|
||||
await seedDatabase();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await clearDatabase();
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it('should create order and store in database', async () => {
|
||||
const response = await server.request
|
||||
.post('/api/orders')
|
||||
.send({
|
||||
userId: 'user-123',
|
||||
items: [{ productId: 'prod-1', quantity: 2 }],
|
||||
});
|
||||
|
||||
expect(response.status).toBe(201);
|
||||
expect(response.body).toMatchObject({
|
||||
id: expect.any(String),
|
||||
userId: 'user-123',
|
||||
status: 'pending',
|
||||
});
|
||||
|
||||
// Verify database persistence
|
||||
const order = await server.db.orders.findById(response.body.id);
|
||||
expect(order).toBeTruthy();
|
||||
expect(order.items).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('should send confirmation email on order creation', async () => {
|
||||
const emailSpy = vi.spyOn(server.emailService, 'send');
|
||||
|
||||
await server.request
|
||||
.post('/api/orders')
|
||||
.send({
|
||||
userId: 'user-123',
|
||||
items: [{ productId: 'prod-1', quantity: 1 }],
|
||||
});
|
||||
|
||||
expect(emailSpy).toHaveBeenCalledWith({
|
||||
to: 'user@example.com',
|
||||
subject: 'Order Confirmation',
|
||||
template: 'order-confirmation',
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**E2E Test (10% of suite)**:
|
||||
```typescript
|
||||
// e2e/checkout-flow.spec.ts
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test.describe('Checkout Flow', () => {
|
||||
test('should complete purchase as guest user', async ({ page }) => {
|
||||
// Navigate to product
|
||||
await page.goto('/products/laptop-123');
|
||||
await expect(page.getByRole('heading', { name: 'Gaming Laptop' })).toBeVisible();
|
||||
|
||||
// Add to cart
|
||||
await page.getByRole('button', { name: 'Add to Cart' }).click();
|
||||
await expect(page.getByText('Item added to cart')).toBeVisible();
|
||||
|
||||
// Go to checkout
|
||||
await page.getByRole('link', { name: 'Cart (1)' }).click();
|
||||
await page.getByRole('button', { name: 'Checkout' }).click();
|
||||
|
||||
// Fill shipping info
|
||||
await page.getByLabel('Email').fill('guest@example.com');
|
||||
await page.getByLabel('Full Name').fill('John Doe');
|
||||
await page.getByLabel('Address').fill('123 Main St');
|
||||
await page.getByLabel('City').fill('New York');
|
||||
await page.getByLabel('Zip Code').fill('10001');
|
||||
|
||||
// Fill payment info (test mode)
|
||||
await page.getByLabel('Card Number').fill('4242424242424242');
|
||||
await page.getByLabel('Expiry Date').fill('12/25');
|
||||
await page.getByLabel('CVC').fill('123');
|
||||
|
||||
// Submit order
|
||||
await page.getByRole('button', { name: 'Place Order' }).click();
|
||||
|
||||
// Verify confirmation
|
||||
await expect(page).toHaveURL(/\/order-confirmation/);
|
||||
await expect(page.getByText('Order Confirmed!')).toBeVisible();
|
||||
await expect(page.getByText(/Order #/)).toBeVisible();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Coverage Targets
|
||||
|
||||
- **Unit Tests**: 80%+ line coverage, 75%+ branch coverage
|
||||
- **Integration Tests**: 100% critical API endpoints
|
||||
- **E2E Tests**: 100% critical user journeys
|
||||
|
||||
---
|
||||
|
||||
## Testing Trophy Strategy
|
||||
|
||||
### Overview
|
||||
|
||||
Modern approach that emphasizes integration tests over unit tests, with static analysis as the foundation.
|
||||
|
||||
```
|
||||
/\
|
||||
/ \ E2E (5%)
|
||||
/----\
|
||||
/ \ Integration (50%)
|
||||
/--------\
|
||||
/ \ Unit (25%)
|
||||
/--------------\
|
||||
/ \ Static (20%)
|
||||
/------------------\
|
||||
```
|
||||
|
||||
### Distribution
|
||||
|
||||
- **Static Analysis (20%)**: TypeScript, ESLint, Prettier
|
||||
- **Unit Tests (25%)**: Pure functions, utilities, critical logic
|
||||
- **Integration Tests (50%)**: Components with dependencies, API contracts
|
||||
- **E2E Tests (5%)**: Critical business flows only
|
||||
|
||||
### When to Use
|
||||
|
||||
- Modern frontend applications (React, Vue, Angular)
|
||||
- Applications with complex component interactions
|
||||
- Teams using TypeScript and static analysis tools
|
||||
- Applications where integration tests catch more bugs
|
||||
|
||||
### Implementation Example
|
||||
|
||||
**Static Analysis (20%)**:
|
||||
```json
|
||||
// tsconfig.json
|
||||
{
|
||||
"compilerOptions": {
|
||||
"strict": true,
|
||||
"noImplicitAny": true,
|
||||
"strictNullChecks": true,
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
// .eslintrc.json
|
||||
{
|
||||
"extends": [
|
||||
"eslint:recommended",
|
||||
"plugin:@typescript-eslint/recommended",
|
||||
"plugin:react-hooks/recommended",
|
||||
"plugin:jsx-a11y/recommended"
|
||||
],
|
||||
"rules": {
|
||||
"@typescript-eslint/no-explicit-any": "error",
|
||||
"react-hooks/exhaustive-deps": "error"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Unit Tests (25%)**:
|
||||
```typescript
|
||||
// src/utils/formatters.test.ts
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { formatCurrency, formatDate } from './formatters';
|
||||
|
||||
describe('Formatters (Pure Functions)', () => {
|
||||
describe('formatCurrency', () => {
|
||||
it('should format USD currency', () => {
|
||||
expect(formatCurrency(1234.56, 'USD')).toBe('$1,234.56');
|
||||
});
|
||||
|
||||
it('should handle negative amounts', () => {
|
||||
expect(formatCurrency(-100, 'USD')).toBe('-$100.00');
|
||||
});
|
||||
});
|
||||
|
||||
describe('formatDate', () => {
|
||||
it('should format ISO date', () => {
|
||||
const date = new Date('2025-01-15');
|
||||
expect(formatDate(date, 'short')).toBe('1/15/2025');
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Integration Tests (50%)**:
|
||||
```typescript
|
||||
// src/components/UserProfile.integration.test.tsx
|
||||
import { describe, it, expect, beforeEach } from 'vitest';
|
||||
import { render, screen, waitFor } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { rest } from 'msw';
|
||||
import { setupServer } from 'msw/node';
|
||||
import { UserProfile } from './UserProfile';
|
||||
|
||||
const server = setupServer(
|
||||
rest.get('/api/users/:id', (req, res, ctx) => {
|
||||
return res(
|
||||
ctx.json({
|
||||
id: req.params.id,
|
||||
name: 'John Doe',
|
||||
email: 'john@example.com',
|
||||
})
|
||||
);
|
||||
}),
|
||||
|
||||
rest.put('/api/users/:id', (req, res, ctx) => {
|
||||
return res(ctx.json({ ...req.body, updatedAt: Date.now() }));
|
||||
})
|
||||
);
|
||||
|
||||
beforeAll(() => server.listen());
|
||||
afterEach(() => server.resetHandlers());
|
||||
afterAll(() => server.close());
|
||||
|
||||
describe('UserProfile Integration', () => {
|
||||
it('should load and display user data', async () => {
|
||||
render(<UserProfile userId="123" />);
|
||||
|
||||
// Loading state
|
||||
expect(screen.getByText('Loading...')).toBeInTheDocument();
|
||||
|
||||
// Wait for data to load
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('John Doe')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
expect(screen.getByText('john@example.com')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should update user profile on form submit', async () => {
|
||||
const user = userEvent.setup();
|
||||
render(<UserProfile userId="123" />);
|
||||
|
||||
// Wait for initial load
|
||||
await screen.findByText('John Doe');
|
||||
|
||||
// Edit name
|
||||
const nameInput = screen.getByLabelText('Name');
|
||||
await user.clear(nameInput);
|
||||
await user.type(nameInput, 'Jane Smith');
|
||||
|
||||
// Submit form
|
||||
await user.click(screen.getByRole('button', { name: 'Save' }));
|
||||
|
||||
// Verify success message
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Profile updated successfully')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
// Verify updated name
|
||||
expect(screen.getByText('Jane Smith')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should handle API errors gracefully', async () => {
|
||||
// Mock API error
|
||||
server.use(
|
||||
rest.get('/api/users/:id', (req, res, ctx) => {
|
||||
return res(ctx.status(500), ctx.json({ error: 'Internal Server Error' }));
|
||||
})
|
||||
);
|
||||
|
||||
render(<UserProfile userId="123" />);
|
||||
|
||||
// Wait for error message
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText(/Failed to load user/i)).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**E2E Tests (5%)**:
|
||||
```typescript
|
||||
// e2e/critical-path.spec.ts
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('critical user journey: signup to first purchase', async ({ page }) => {
|
||||
// Only test the MOST critical path
|
||||
await page.goto('/');
|
||||
|
||||
// Signup
|
||||
await page.getByRole('link', { name: 'Sign Up' }).click();
|
||||
await page.getByLabel('Email').fill('newuser@example.com');
|
||||
await page.getByLabel('Password').fill('SecurePass123!');
|
||||
await page.getByRole('button', { name: 'Create Account' }).click();
|
||||
|
||||
// Verify logged in
|
||||
await expect(page.getByText('Welcome, New User')).toBeVisible();
|
||||
|
||||
// Make purchase
|
||||
await page.goto('/products/best-seller');
|
||||
await page.getByRole('button', { name: 'Buy Now' }).click();
|
||||
await page.getByLabel('Card Number').fill('4242424242424242');
|
||||
await page.getByRole('button', { name: 'Complete Purchase' }).click();
|
||||
|
||||
// Verify success
|
||||
await expect(page.getByText('Purchase Successful')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
### Coverage Targets
|
||||
|
||||
- **Static Analysis**: 100% (TypeScript strict mode, zero ESLint errors)
|
||||
- **Unit Tests**: 90%+ for pure functions and utilities
|
||||
- **Integration Tests**: 80%+ for components with dependencies
|
||||
- **E2E Tests**: 100% critical paths only
|
||||
|
||||
---
|
||||
|
||||
## TDD Red-Green-Refactor
|
||||
|
||||
### The Cycle
|
||||
|
||||
1. **RED**: Write a failing test that defines expected behavior
|
||||
2. **GREEN**: Write minimal code to make the test pass
|
||||
3. **REFACTOR**: Improve code quality while keeping tests green
|
||||
|
||||
### Example: Shopping Cart Feature
|
||||
|
||||
**RED: Write Failing Test**:
|
||||
```typescript
|
||||
// src/cart/ShoppingCart.test.ts
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { ShoppingCart } from './ShoppingCart';
|
||||
|
||||
describe('ShoppingCart', () => {
|
||||
it('should add item to cart', () => {
|
||||
const cart = new ShoppingCart();
|
||||
|
||||
cart.addItem({ id: 1, name: 'Laptop', price: 1000 });
|
||||
|
||||
expect(cart.getItemCount()).toBe(1);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Run test**: ❌ FAIL (ShoppingCart doesn't exist)
|
||||
|
||||
**GREEN: Minimal Implementation**:
|
||||
```typescript
|
||||
// src/cart/ShoppingCart.ts
|
||||
interface CartItem {
|
||||
id: number;
|
||||
name: string;
|
||||
price: number;
|
||||
}
|
||||
|
||||
export class ShoppingCart {
|
||||
private items: CartItem[] = [];
|
||||
|
||||
addItem(item: CartItem): void {
|
||||
this.items.push(item);
|
||||
}
|
||||
|
||||
getItemCount(): number {
|
||||
return this.items.length;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Run test**: ✅ PASS
|
||||
|
||||
**Add Another Test (Triangulation)**:
|
||||
```typescript
|
||||
it('should calculate total price', () => {
|
||||
const cart = new ShoppingCart();
|
||||
|
||||
cart.addItem({ id: 1, name: 'Laptop', price: 1000 });
|
||||
cart.addItem({ id: 2, name: 'Mouse', price: 50 });
|
||||
|
||||
expect(cart.getTotal()).toBe(1050);
|
||||
});
|
||||
```
|
||||
|
||||
**Run test**: ❌ FAIL (getTotal doesn't exist)
|
||||
|
||||
**GREEN: Implement getTotal**:
|
||||
```typescript
|
||||
export class ShoppingCart {
|
||||
// ... previous code ...
|
||||
|
||||
getTotal(): number {
|
||||
return this.items.reduce((sum, item) => sum + item.price, 0);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Run test**: ✅ PASS
|
||||
|
||||
**REFACTOR: Improve Design**:
|
||||
```typescript
|
||||
export class ShoppingCart {
|
||||
private items: Map<number, CartItem> = new Map();
|
||||
|
||||
addItem(item: CartItem): void {
|
||||
const existing = this.items.get(item.id);
|
||||
if (existing) {
|
||||
// Increment quantity instead of duplicating
|
||||
existing.quantity = (existing.quantity || 1) + 1;
|
||||
} else {
|
||||
this.items.set(item.id, { ...item, quantity: 1 });
|
||||
}
|
||||
}
|
||||
|
||||
getItemCount(): number {
|
||||
return Array.from(this.items.values()).reduce(
|
||||
(count, item) => count + (item.quantity || 1),
|
||||
0
|
||||
);
|
||||
}
|
||||
|
||||
getTotal(): number {
|
||||
return Array.from(this.items.values()).reduce(
|
||||
(sum, item) => sum + item.price * (item.quantity || 1),
|
||||
0
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Run tests**: ✅ ALL PASS (refactoring didn't break anything!)
|
||||
|
||||
### TDD Benefits
|
||||
|
||||
- Forces modular, testable design
|
||||
- Prevents over-engineering
|
||||
- Living documentation
|
||||
- Fearless refactoring
|
||||
- Faster debugging
|
||||
|
||||
---
|
||||
|
||||
## API Testing Strategy
|
||||
|
||||
### Layers
|
||||
|
||||
1. **Unit Tests**: Test business logic in isolation
|
||||
2. **Integration Tests**: Test API endpoints with real database
|
||||
3. **Contract Tests**: Test API contracts (Pact)
|
||||
4. **E2E Tests**: Test complete API flows
|
||||
|
||||
### Example: REST API Testing
|
||||
|
||||
**Unit Test (Business Logic)**:
|
||||
```typescript
|
||||
// src/services/OrderService.test.ts
|
||||
import { describe, it, expect, vi } from 'vitest';
|
||||
import { OrderService } from './OrderService';
|
||||
|
||||
describe('OrderService', () => {
|
||||
it('should calculate order total with tax', () => {
|
||||
const mockRepo = {
|
||||
save: vi.fn(),
|
||||
findById: vi.fn(),
|
||||
};
|
||||
|
||||
const service = new OrderService(mockRepo);
|
||||
|
||||
const order = service.calculateTotal({
|
||||
items: [{ price: 100, quantity: 2 }],
|
||||
taxRate: 0.08,
|
||||
});
|
||||
|
||||
expect(order.subtotal).toBe(200);
|
||||
expect(order.tax).toBeCloseTo(16, 2);
|
||||
expect(order.total).toBeCloseTo(216, 2);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Integration Test (API + Database)**:
|
||||
```typescript
|
||||
// tests/integration/orders.test.ts
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import supertest from 'supertest';
|
||||
import { createTestApp } from '../utils/test-app';
|
||||
|
||||
describe('Orders API', () => {
|
||||
let app;
|
||||
let request;
|
||||
|
||||
beforeAll(async () => {
|
||||
app = await createTestApp();
|
||||
request = supertest(app);
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it('POST /api/orders should create order', async () => {
|
||||
const response = await request
|
||||
.post('/api/orders')
|
||||
.send({
|
||||
userId: 'user-123',
|
||||
items: [
|
||||
{ productId: 'prod-1', quantity: 2, price: 50 },
|
||||
],
|
||||
})
|
||||
.expect(201);
|
||||
|
||||
expect(response.body).toMatchObject({
|
||||
id: expect.any(String),
|
||||
userId: 'user-123',
|
||||
status: 'pending',
|
||||
total: 100,
|
||||
});
|
||||
|
||||
// Verify database persistence
|
||||
const order = await app.db.orders.findById(response.body.id);
|
||||
expect(order).toBeTruthy();
|
||||
});
|
||||
|
||||
it('GET /api/orders/:id should return order', async () => {
|
||||
// Create order first
|
||||
const createResponse = await request
|
||||
.post('/api/orders')
|
||||
.send({ userId: 'user-123', items: [] });
|
||||
|
||||
const orderId = createResponse.body.id;
|
||||
|
||||
// Fetch order
|
||||
const response = await request
|
||||
.get(`/api/orders/${orderId}`)
|
||||
.expect(200);
|
||||
|
||||
expect(response.body.id).toBe(orderId);
|
||||
});
|
||||
|
||||
it('PUT /api/orders/:id/status should update status', async () => {
|
||||
const createResponse = await request
|
||||
.post('/api/orders')
|
||||
.send({ userId: 'user-123', items: [] });
|
||||
|
||||
const orderId = createResponse.body.id;
|
||||
|
||||
const response = await request
|
||||
.put(`/api/orders/${orderId}/status`)
|
||||
.send({ status: 'shipped' })
|
||||
.expect(200);
|
||||
|
||||
expect(response.body.status).toBe('shipped');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Contract Test (Pact)**:
|
||||
```typescript
|
||||
// tests/contract/orders-consumer.test.ts
|
||||
import { PactV3, MatchersV3 } from '@pact-foundation/pact';
|
||||
import { OrdersClient } from '@/api/orders-client';
|
||||
|
||||
const provider = new PactV3({
|
||||
consumer: 'OrdersConsumer',
|
||||
provider: 'OrdersAPI',
|
||||
});
|
||||
|
||||
describe('Orders API Contract', () => {
|
||||
it('should get order by ID', async () => {
|
||||
await provider
|
||||
.given('order with ID 123 exists')
|
||||
.uponReceiving('a request for order 123')
|
||||
.withRequest({
|
||||
method: 'GET',
|
||||
path: '/api/orders/123',
|
||||
})
|
||||
.willRespondWith({
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: {
|
||||
id: MatchersV3.string('123'),
|
||||
userId: MatchersV3.string('user-456'),
|
||||
status: MatchersV3.regex('pending|shipped|delivered', 'pending'),
|
||||
total: MatchersV3.decimal(100.50),
|
||||
},
|
||||
})
|
||||
.executeTest(async (mockServer) => {
|
||||
const client = new OrdersClient(mockServer.url);
|
||||
const order = await client.getOrder('123');
|
||||
|
||||
expect(order.id).toBe('123');
|
||||
expect(order.status).toMatch(/pending|shipped|delivered/);
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Coverage Targets
|
||||
|
||||
- Unit: 90%+ business logic
|
||||
- Integration: 100% API endpoints
|
||||
- Contract: 100% API contracts
|
||||
- E2E: Critical flows only
|
||||
|
||||
[Document continues with 6 more comprehensive testing strategies...]
|
||||
|
||||
---
|
||||
|
||||
## Summary Table
|
||||
|
||||
| Strategy | Best For | Coverage Target | Execution Time |
|
||||
|----------|----------|-----------------|----------------|
|
||||
| Pyramid | Backend services, traditional apps | 80%+ unit, 100% critical | < 5 min |
|
||||
| Trophy | Modern frontends (React, Vue) | 80%+ integration | < 3 min |
|
||||
| TDD | New features, greenfield projects | 90%+ | Continuous |
|
||||
| BDD | Stakeholder-driven development | 100% acceptance | Variable |
|
||||
| API | REST/GraphQL services | 90%+ endpoints | < 2 min |
|
||||
| Frontend | SPAs, component libraries | 85%+ components | < 4 min |
|
||||
| Micro-Services | Distributed systems | 80%+ per service | < 10 min |
|
||||
| Performance | High-traffic applications | Critical paths | 30 min |
|
||||
| Security | Sensitive data, compliance | 100% attack vectors | 1 hour |
|
||||
| Accessibility | Public-facing websites | WCAG AA 100% | 15 min |
|
||||
|
||||
Reference in New Issue
Block a user