zhongwei/gh-greyhaven-ai-claude-code-config-grey-haven-plugins-testing
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 18:29:30 +08:00
commit 40d73f6839
33 changed files with 8109 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

214
skills/testing-strategy/checklists/test-quality-review.md Normal file
View File

@@ -0,0 +1,214 @@
# Test Code Review Checklist
Use this checklist when reviewing test code in pull requests.
## General Test Quality
### Test Structure
- [ ] **Clear test names**: Descriptive, follows `test_should_do_something_when_condition` pattern
- [ ] **One assertion focus**: Each test verifies one specific behavior
- [ ] **Arrange-Act-Assert**: Tests follow AAA pattern clearly
- [ ] **No magic numbers**: Test values are self-explanatory or use named constants
- [ ] **Readable setup**: Test setup is clear and concise
### Test Independence
- [ ] **No shared state**: Tests don't depend on each other
- [ ] **Can run in any order**: Tests pass when run individually or in any sequence
- [ ] **Proper cleanup**: Tests clean up resources (database, files, mocks)
- [ ] **Isolated changes**: Tests don't pollute global state
- [ ] **Fresh fixtures**: Each test gets fresh test data
### Test Coverage
- [ ] **New code is tested**: All new functions/components have tests
- [ ] **Edge cases covered**: Null, empty, invalid inputs tested
- [ ] **Error paths tested**: Error handling and failure scenarios verified
- [ ] **Happy path tested**: Normal, expected behavior verified
- [ ] **Branch coverage**: All if/else and switch branches tested
## TypeScript/Vitest Review
### Component Tests
- [ ] **Correct rendering**: Components render without errors
- [ ] **User interactions**: Click, input, form submissions tested
- [ ] **Loading states**: Loading indicators tested
- [ ] **Error states**: Error messages and boundaries tested
- [ ] **Async handling**: Uses `waitFor()` for async state changes
- [ ] **Query wrapper**: TanStack Query components wrapped correctly
- [ ] **Accessibility**: Uses semantic queries (`getByRole`, `getByLabelText`)
### Mocking
- [ ] **Appropriate mocking**: Mocks external dependencies (APIs, modules)
- [ ] **Not over-mocked**: Integration tests use real implementations where appropriate
- [ ] **Clear mock setup**: Mock configuration is easy to understand
- [ ] **Mock verification**: Tests verify mocks were called correctly
- [ ] **Mock cleanup**: Mocks cleared after each test (`vi.clearAllMocks()`)
### Best Practices
- [ ] **Path aliases**: Uses `~/ ` for imports (not relative paths)
- [ ] **TypeScript types**: Test code is properly typed
- [ ] **Testing Library**: Uses `@testing-library/react` best practices
- [ ] **Vitest globals**: Uses globals (`describe`, `it`, `expect`) correctly
- [ ] **No console warnings**: Tests don't produce React warnings
## Python/pytest Review
### Unit Tests
- [ ] **Isolated tests**: No external dependencies (database, network)
- [ ] **Fast execution**: Unit tests complete in < 100ms
- [ ] **Proper fixtures**: Uses pytest fixtures appropriately
- [ ] **Mocking external services**: Uses `unittest.mock` or `pytest-mock`
- [ ] **Type hints**: Test functions have type hints
### Integration Tests
- [ ] **Real dependencies**: Uses real database/services where appropriate
- [ ] **Transaction handling**: Tests verify rollback on errors
- [ ] **Tenant isolation**: Tests verify multi-tenant data separation
- [ ] **Async/await**: Async tests use `async def` and `await`
- [ ] **Database cleanup**: Fixtures clean up test data
### Markers
- [ ] **Correct markers**: Tests marked with `@pytest.mark.unit`, `@pytest.mark.integration`, etc.
- [ ] **Consistent markers**: Markers match test type (unit, integration, e2e, benchmark)
- [ ] **Slow marker**: Tests >5 seconds marked with `@pytest.mark.slow`
### Best Practices
- [ ] **Descriptive docstrings**: Test functions have clear docstrings
- [ ] **Factory usage**: Uses factory pattern for test data
- [ ] **No hardcoded IDs**: Uses `uuid4()` for test IDs
- [ ] **Proper imports**: Imports organized and clear
- [ ] **No test pollution**: Tests don't leave data in database
## Multi-Tenant Testing
### Tenant Isolation
- [ ] **Tenant ID filtering**: All queries filter by `tenant_id`
- [ ] **Cross-tenant access denied**: Tests verify users can't access other tenant's data
- [ ] **Tenant header required**: API tests include `X-Tenant-ID` header
- [ ] **Repository methods**: All repository methods accept `tenant_id` parameter
- [ ] **Query verification**: Tests verify correct `tenant_id` in database queries
### Security
- [ ] **Authentication tested**: Protected endpoints require auth
- [ ] **Authorization tested**: Users can only access authorized resources
- [ ] **Input validation**: Invalid input properly rejected
- [ ] **SQL injection protected**: No raw SQL in tests (uses ORM)
- [ ] **XSS protection**: Input sanitization tested where applicable
## Environment & Configuration
### Doppler
- [ ] **Doppler used**: Tests run with `doppler run --`
- [ ] **No hardcoded secrets**: No API keys or secrets in test code
- [ ] **Correct config**: Tests use `test` Doppler config
- [ ] **Environment isolation**: Test database separate from dev
### Test Data
- [ ] **Faker/factory-boy**: Random test data uses faker
- [ ] **Realistic data**: Test data resembles production data
- [ ] **No PII**: Test data doesn't contain real personal information
- [ ] **Deterministic when needed**: Uses seed for reproducible random data when necessary
## Performance
### Test Speed
- [ ] **Fast unit tests**: Unit tests < 100ms each
- [ ] **Reasonable integration tests**: Integration tests < 1 second each
- [ ] **Parallel execution**: Tests can run in parallel
- [ ] **No unnecessary waits**: No `sleep()` or arbitrary delays
- [ ] **Optimized queries**: Database queries efficient
### Resource Usage
- [ ] **Minimal test data**: Creates only necessary test data
- [ ] **Connection cleanup**: Database connections closed properly
- [ ] **Memory efficient**: No memory leaks in test setup
- [ ] **File cleanup**: Temporary files deleted after tests
## CI/CD Compatibility
### GitHub Actions
- [ ] **Passes in CI**: Tests pass in GitHub Actions
- [ ] **No flaky tests**: Tests pass consistently (not intermittent failures)
- [ ] **Correct services**: Required services (postgres, redis) configured
- [ ] **Coverage upload**: Coverage reports uploaded correctly
- [ ] **Timeout appropriate**: Tests complete within CI timeout limits
### Coverage
- [ ] **Meets threshold**: Coverage meets 80% minimum
- [ ] **No false positives**: Coverage accurately reflects tested code
- [ ] **Coverage trends**: Coverage doesn't decrease from baseline
- [ ] **Critical paths covered**: Important features have high coverage
## Documentation
### Test Documentation
- [ ] **Clear test names**: Test intent obvious from name
- [ ] **Helpful comments**: Complex test logic explained
- [ ] **Fixture documentation**: Custom fixtures documented
- [ ] **Test file organization**: Tests organized logically
- [ ] **README updated**: Testing docs updated if patterns changed
### Code Comments
- [ ] **Why, not what**: Comments explain why, not what code does
- [ ] **No commented-out code**: Old test code removed
- [ ] **TODO comments tracked**: Any TODOs have tracking tickets
- [ ] **No misleading comments**: Comments accurate and up-to-date
## Red Flags to Watch For
### Anti-Patterns
- [ ] ❌ Tests that only test mocks
- [ ] ❌ Tests with no assertions
- [ ] ❌ Tests that test private implementation
- [ ] ❌ Brittle tests that break on refactoring
- [ ] ❌ Tests that depend on execution order
- [ ] ❌ Excessive setup code (>50% of test)
- [ ] ❌ Tests with sleep/wait instead of proper async handling
- [ ] ❌ Tests that write to production database
- [ ] ❌ Tests that make real API calls
- [ ] ❌ Tests with hardcoded production credentials
### Smells
- [ ] ⚠️ Very long test functions (>50 lines)
- [ ] ⚠️ Duplicate test code (could use fixtures)
- [ ] ⚠️ Tests with multiple assertions on different behaviors
- [ ] ⚠️ Tests that take >5 seconds
- [ ] ⚠️ Tests that fail intermittently
- [ ] ⚠️ Tests with complex logic (loops, conditionals)
- [ ] ⚠️ Tests that require manual setup to run
- [ ] ⚠️ Missing error assertions
- [ ] ⚠️ Testing framework workarounds/hacks
## Approval Criteria
Before approving PR with tests:
- [ ] All tests pass locally and in CI
- [ ] Coverage meets minimum threshold (80%)
- [ ] Tests follow Grey Haven conventions
- [ ] No anti-patterns or red flags
- [ ] Test code is readable and maintainable
- [ ] Tests verify correct behavior (not just implementation)
- [ ] Security and tenant isolation tested
- [ ] Documentation updated if needed

192
skills/testing-strategy/checklists/testing-checklist.md Normal file
View File

@@ -0,0 +1,192 @@
# Testing Checklist
Use this checklist before submitting PRs to ensure comprehensive test coverage and quality.
## Pre-PR Testing Checklist
### Test Coverage
- [ ] All new functions/methods have unit tests
- [ ] All new components have component tests
- [ ] All new API endpoints have integration tests
- [ ] Critical user flows have E2E tests
- [ ] Code coverage is at least 80% (run `bun test --coverage` or `pytest --cov`)
- [ ] No coverage regression from previous version
- [ ] Security-critical code has 100% coverage (auth, payments, tenant isolation)
### Test Quality
- [ ] Tests follow naming convention: `test_should_do_something_when_condition`
- [ ] Each test has a single, clear assertion focus
- [ ] Tests are independent (can run in any order)
- [ ] Tests clean up after themselves (no database pollution)
- [ ] No hardcoded values (use constants or fixtures)
- [ ] Test data uses factories (faker/factory-boy)
- [ ] Mock external services (APIs, email, payments)
- [ ] Tests run in < 10 seconds (unit tests < 100ms each)
### Test Markers
- [ ] Unit tests marked with `@pytest.mark.unit` or in `tests/unit/`
- [ ] Integration tests marked with `@pytest.mark.integration` or in `tests/integration/`
- [ ] E2E tests marked with `@pytest.mark.e2e` or in `tests/e2e/`
- [ ] Slow tests marked with `@pytest.mark.slow` (> 5 seconds)
### Multi-Tenant Testing
- [ ] All database queries test tenant isolation
- [ ] Repository methods verify correct `tenant_id` filtering
- [ ] API endpoints test tenant header validation
- [ ] Cross-tenant access attempts are tested and fail correctly
### Environment Variables
- [ ] All tests use Doppler for environment variables
- [ ] No hardcoded secrets or API keys
- [ ] Test database is separate from development database
- [ ] `.env` files are NOT committed to repository
- [ ] CI uses `DOPPLER_TOKEN_TEST` secret
### Error Handling
- [ ] Tests verify error messages and status codes
- [ ] Edge cases are tested (null, empty, invalid input)
- [ ] Validation errors return correct HTTP status (422)
- [ ] Database errors are handled gracefully
- [ ] Tests verify rollback on transaction errors
### TypeScript Specific
- [ ] React Testing Library used for component tests
- [ ] TanStack Query components tested with QueryClientProvider wrapper
- [ ] Server function mocks use `vi.mock()`
- [ ] Async components use `waitFor()` for assertions
- [ ] Vitest globals enabled in config (`globals: true`)
### Python Specific
- [ ] Virtual environment activated before running tests
- [ ] Async fixtures used for async code (`async def`)
- [ ] FastAPI TestClient used for API tests
- [ ] Database fixtures use session-scoped engine
- [ ] SQLAlchemy sessions auto-rollback in fixtures
### CI/CD
- [ ] Tests pass locally with `bun test` or `doppler run -- pytest`
- [ ] Tests pass in CI (GitHub Actions)
- [ ] Coverage report uploaded to Codecov
- [ ] No test warnings or deprecation messages
- [ ] Pre-commit hooks pass (if configured)
## Test Types Checklist
### Unit Tests
- [ ] Test single function/class in isolation
- [ ] Mock all external dependencies
- [ ] No database or network calls
- [ ] Fast execution (< 100ms per test)
- [ ] Cover all code branches (if/else, try/catch)
### Integration Tests
- [ ] Test multiple components together
- [ ] Use real database (with cleanup)
- [ ] Test complete API request/response cycles
- [ ] Verify database state changes
- [ ] Test transaction handling
### E2E Tests
- [ ] Test complete user workflows
- [ ] Use Playwright for TypeScript
- [ ] Test from user perspective (UI interactions)
- [ ] Verify multi-step processes
- [ ] Test critical business flows
### Benchmark Tests
- [ ] Measure performance metrics
- [ ] Set performance thresholds
- [ ] Test with realistic data volumes
- [ ] Monitor for regressions
## Coverage Goals by Component
### Utility Functions
- [ ] 95%+ coverage
- [ ] All branches tested
- [ ] Edge cases handled
### Business Logic (Services)
- [ ] 90%+ coverage
- [ ] All business rules tested
- [ ] Error scenarios covered
### API Endpoints
- [ ] 85%+ coverage
- [ ] All HTTP methods tested
- [ ] All response codes verified
### Database Repositories
- [ ] 90%+ coverage
- [ ] CRUD operations tested
- [ ] Tenant isolation verified
### React Components
- [ ] 80%+ coverage
- [ ] Rendering tested
- [ ] User interactions tested
- [ ] Loading/error states tested
### Security Features
- [ ] 100% coverage
- [ ] Authentication tested
- [ ] Authorization tested
- [ ] Tenant isolation verified
## Common Testing Mistakes to Avoid
### Don't
- [ ] ❌ Test implementation details
- [ ] ❌ Test private methods directly
- [ ] ❌ Write tests that depend on execution order
- [ ] ❌ Use real external services in tests
- [ ] ❌ Hardcode test data
- [ ] ❌ Commit `.env` files
- [ ] ❌ Skip test cleanup
- [ ] ❌ Test multiple things in one test
- [ ] ❌ Forget to await async operations
- [ ] ❌ Mock too much (integration tests)
### Do
- [ ] ✅ Test public APIs and behaviors
- [ ] ✅ Write independent, isolated tests
- [ ] ✅ Mock external services
- [ ] ✅ Use test factories for data
- [ ] ✅ Use Doppler for environment variables
- [ ] ✅ Clean up test data
- [ ] ✅ Focus each test on one assertion
- [ ] ✅ Use `waitFor()` for async rendering
- [ ] ✅ Test error scenarios
- [ ] ✅ Verify tenant isolation
## Post-Testing Checklist
- [ ] All tests pass locally
- [ ] Coverage meets minimum threshold (80%)
- [ ] No failing tests in CI
- [ ] Coverage report reviewed
- [ ] Test output reviewed for warnings
- [ ] Performance acceptable (no slow tests)
- [ ] Documentation updated (if test patterns changed)
- [ ] Reviewers can understand test intent