Initial commit
This commit is contained in:
706
commands/story-qa.md
Normal file
706
commands/story-qa.md
Normal file
@@ -0,0 +1,706 @@
|
||||
# /sdd:story-qa
|
||||
|
||||
## Meta
|
||||
- Version: 2.0
|
||||
- Category: quality-gates
|
||||
- Complexity: high
|
||||
- Purpose: Move story to QA stage and execute comprehensive test validation pipeline
|
||||
|
||||
## Definition
|
||||
**Purpose**: Execute comprehensive automated QA test suite including unit, integration, browser, and performance tests before final validation.
|
||||
|
||||
**Syntax**: `/sdd:story-qa [story_id]`
|
||||
|
||||
## Parameters
|
||||
| Parameter | Type | Required | Default | Description | Validation |
|
||||
|-----------|------|----------|---------|-------------|------------|
|
||||
| story_id | string | No | current branch | Story ID (STORY-YYYY-NNN) | Must match format STORY-YYYY-NNN |
|
||||
|
||||
## INSTRUCTION: Execute Story QA
|
||||
|
||||
### INPUTS
|
||||
- story_id: Story identifier (auto-detected from branch if not provided)
|
||||
- Project context from `/docs/project-context/` directory
|
||||
- Story file from `/docs/stories/review/[story-id].md`
|
||||
- Complete test suite (unit, feature, browser)
|
||||
- Performance benchmarks (if defined)
|
||||
|
||||
### PROCESS
|
||||
|
||||
#### Phase 1: Project Context Loading
|
||||
1. **CHECK** if `/docs/project-context/` directory exists
|
||||
2. IF missing:
|
||||
- SUGGEST running `/sdd:project-init` first
|
||||
- EXIT with initialization guidance
|
||||
3. **LOAD** project-specific QA requirements from:
|
||||
- `/docs/project-context/technical-stack.md` - Testing tools and frameworks
|
||||
- `/docs/project-context/coding-standards.md` - QA standards and thresholds
|
||||
- `/docs/project-context/development-process.md` - QA stage requirements
|
||||
|
||||
#### Phase 2: Story Identification & Validation
|
||||
1. IF story_id NOT provided:
|
||||
- **DETECT** current git branch
|
||||
- **EXTRACT** story ID from branch name
|
||||
- EXAMPLE: Branch `feature/STORY-2025-001-auth` → ID `STORY-2025-001`
|
||||
|
||||
2. **VALIDATE** story exists:
|
||||
- CHECK `/docs/stories/review/[story-id].md` exists
|
||||
- IF NOT found in review:
|
||||
- CHECK if already in `/docs/stories/qa/`
|
||||
- INFORM user and ask to proceed with re-QA
|
||||
- IF in development:
|
||||
- ERROR: "Story must pass review first"
|
||||
- SUGGEST: `/sdd:story-review [story-id]`
|
||||
- IF NOT found anywhere:
|
||||
- ERROR: "Story [story-id] not found"
|
||||
- EXIT with guidance
|
||||
|
||||
3. **READ** story file for:
|
||||
- Current status
|
||||
- Success criteria (will map to browser tests)
|
||||
- Implementation checklist state
|
||||
- QA checklist requirements
|
||||
|
||||
#### Phase 3: Directory Preparation
|
||||
1. **ENSURE** `/docs/stories/qa/` directory exists
|
||||
- CREATE directory if missing
|
||||
- ADD `.gitkeep` file if directory was created
|
||||
|
||||
2. **MOVE** story file:
|
||||
- FROM: `/docs/stories/review/[story-id].md`
|
||||
- TO: `/docs/stories/qa/[story-id].md`
|
||||
- PRESERVE all content and formatting
|
||||
|
||||
3. **UPDATE** story metadata:
|
||||
- Change status from "review" to "qa"
|
||||
- KEEP existing dates and branch information
|
||||
- ADD QA start timestamp to progress log
|
||||
|
||||
#### Phase 4: Test Suite Execution
|
||||
|
||||
##### 4.1 Unit Tests (Discovered Framework)
|
||||
1. **IDENTIFY** unit test framework from technical-stack.md:
|
||||
- PHP/Laravel: Pest, PHPUnit
|
||||
- Node.js: Jest, Vitest, Mocha
|
||||
- Python: pytest, unittest
|
||||
- Go: go test
|
||||
- Java: JUnit, TestNG
|
||||
- .NET: xUnit, NUnit, MSTest
|
||||
|
||||
2. **RUN** unit tests with coverage:
|
||||
```bash
|
||||
# Example for Laravel Pest:
|
||||
vendor/bin/pest --filter=Unit --coverage --min=80
|
||||
|
||||
# Example for Node.js Jest:
|
||||
npm test -- --coverage --testPathPattern=unit
|
||||
|
||||
# Example for Python pytest:
|
||||
pytest tests/unit/ --cov --cov-report=term-missing
|
||||
```
|
||||
|
||||
3. **CAPTURE** results:
|
||||
- PASS/FAIL count
|
||||
- Execution time
|
||||
- Coverage percentage (overall, per file)
|
||||
- Failed test details with stack traces
|
||||
- Slowest tests (performance indicators)
|
||||
|
||||
##### 4.2 Feature/Integration Tests (Discovered Patterns)
|
||||
1. **IDENTIFY** integration test patterns from technical-stack.md:
|
||||
- Laravel: Feature tests with database interactions
|
||||
- Node.js: Integration tests with API calls
|
||||
- Python: Integration tests with service layer
|
||||
- Java: Integration tests with Spring context
|
||||
|
||||
2. **RUN** integration tests:
|
||||
```bash
|
||||
# Example for Laravel Pest:
|
||||
vendor/bin/pest --filter=Feature --parallel
|
||||
|
||||
# Example for Node.js:
|
||||
npm test -- --testPathPattern=integration
|
||||
|
||||
# Example for Python:
|
||||
pytest tests/integration/ -v
|
||||
```
|
||||
|
||||
3. **VALIDATE** integrations:
|
||||
- API endpoints returning correct responses
|
||||
- Database operations (CRUD, transactions)
|
||||
- Service-to-service communication
|
||||
- External API integrations (with mocks/stubs)
|
||||
- Queue/job processing
|
||||
- Cache operations
|
||||
|
||||
##### 4.3 Browser/E2E Tests (Discovered Browser Testing Tools)
|
||||
1. **IDENTIFY** browser testing framework from technical-stack.md:
|
||||
- Laravel: Laravel Dusk, Pest Browser
|
||||
- Node.js: Playwright, Cypress, Puppeteer
|
||||
- Python: Playwright, Selenium
|
||||
- Java: Selenium, Playwright
|
||||
|
||||
2. **LOCATE** browser test files:
|
||||
- Laravel: `tests/Browser/[StoryId]Test.php`
|
||||
- Node.js: `tests/e2e/[story-id].spec.js`
|
||||
- Python: `tests/browser/test_[story_id].py`
|
||||
- Java: `src/test/java/**/[StoryId]Test.java`
|
||||
|
||||
3. **RUN** browser tests:
|
||||
```bash
|
||||
# Example for Laravel Pest Browser:
|
||||
vendor/bin/pest --filter=Browser
|
||||
|
||||
# Example for Playwright (Node.js):
|
||||
npx playwright test tests/e2e/sdd:story-2025-001
|
||||
|
||||
# Example for Python Playwright:
|
||||
pytest tests/browser/test_story_2025_001.py --headed
|
||||
```
|
||||
|
||||
4. **VALIDATE** against Success Criteria:
|
||||
- MAP each acceptance criterion to browser test
|
||||
- VERIFY each criterion has passing test
|
||||
- CAPTURE screenshots of test execution
|
||||
- RECORD video of test runs (if tool supports)
|
||||
- VALIDATE all user workflows end-to-end
|
||||
|
||||
5. **TEST** across environments (if specified in standards):
|
||||
- Different browsers (Chrome, Firefox, Safari)
|
||||
- Different devices (desktop, tablet, mobile)
|
||||
- Different viewports (responsive design)
|
||||
- Light/dark mode (if applicable)
|
||||
|
||||
##### 4.4 Performance Tests (Discovered Performance Tools)
|
||||
1. **CHECK** if performance requirements defined in story
|
||||
2. IF performance criteria exist:
|
||||
- **IDENTIFY** performance tools from technical-stack.md:
|
||||
* Laravel: Laravel Debugbar, Telescope, Blackfire
|
||||
* Node.js: Artillery, k6, Apache Bench
|
||||
* Python: Locust, pytest-benchmark
|
||||
* Java: JMeter, Gatling
|
||||
|
||||
3. **RUN** performance benchmarks:
|
||||
```bash
|
||||
# Example for Laravel:
|
||||
php artisan serve &
|
||||
ab -n 1000 -c 10 http://localhost:8000/api/endpoint
|
||||
|
||||
# Example for Node.js k6:
|
||||
k6 run performance/sdd:story-2025-001.js
|
||||
```
|
||||
|
||||
4. **VALIDATE** performance targets:
|
||||
- Response time < target (e.g., 200ms)
|
||||
- Throughput > target (e.g., 100 req/sec)
|
||||
- Memory usage < target
|
||||
- No memory leaks
|
||||
- Database query count optimal
|
||||
|
||||
##### 4.5 Security Testing (Discovered Security Tools)
|
||||
1. **RUN** security validation:
|
||||
```bash
|
||||
# Example for Laravel:
|
||||
composer audit
|
||||
|
||||
# Example for Node.js:
|
||||
npm audit --production
|
||||
```
|
||||
|
||||
2. **VALIDATE** security requirements:
|
||||
- No HIGH/CRITICAL vulnerabilities
|
||||
- Authentication/Authorization working
|
||||
- CSRF protection enabled
|
||||
- XSS prevention implemented
|
||||
- SQL injection prevention verified
|
||||
- Rate limiting functional (if applicable)
|
||||
|
||||
#### Phase 5: Quality Gate Validation
|
||||
1. **APPLY** quality gates from coding-standards.md:
|
||||
- BLOCK progression if ANY critical test fails
|
||||
- BLOCK progression if coverage below threshold
|
||||
- BLOCK progression if performance targets not met
|
||||
- BLOCK progression if security vulnerabilities found
|
||||
|
||||
2. **CAPTURE** test artifacts:
|
||||
- Test reports (XML, JSON, HTML)
|
||||
- Coverage reports
|
||||
- Screenshots from browser tests
|
||||
- Videos from browser tests (if available)
|
||||
- Performance benchmark results
|
||||
- Log files from test runs
|
||||
|
||||
#### Phase 6: QA Report Generation
|
||||
1. **COMPILE** all test results
|
||||
2. **GENERATE** automated QA report:
|
||||
|
||||
```
|
||||
✅ AUTOMATED QA RESULTS
|
||||
════════════════════════════════════════════════
|
||||
Story: [STORY-ID] - [Title]
|
||||
Stack: [Discovered Framework/Language/Tools]
|
||||
QA Executed: [Timestamp]
|
||||
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
📋 FUNCTIONAL TESTING
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
✅ Acceptance Criteria: 5/5 verified by browser tests
|
||||
✓ User can toggle dark mode → tests/Browser/DarkModeTest.php::line 45
|
||||
✓ Theme persists on refresh → tests/Browser/DarkModeTest.php::line 67
|
||||
✓ All components support dark mode → tests/Browser/DarkModeTest.php::line 89
|
||||
✓ Keyboard shortcut works → tests/Browser/DarkModeTest.php::line 112
|
||||
✓ Preference syncs across tabs → tests/Browser/DarkModeTest.php::line 134
|
||||
|
||||
📸 Screenshots: /storage/screenshots/sdd:story-2025-003/
|
||||
🎥 Videos: /storage/videos/sdd:story-2025-003/ (if applicable)
|
||||
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
🧪 UNIT TESTING
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
✅ Tests Passed: 45/45 (100%)
|
||||
✅ Coverage: 87% (target: 80%)
|
||||
⏱️ Execution Time: 2.34s
|
||||
|
||||
Top Coverage Files:
|
||||
✓ DarkModeService.php: 95%
|
||||
✓ ThemeController.php: 92%
|
||||
✓ UserPreferenceRepository.php: 88%
|
||||
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
🔗 INTEGRATION TESTING
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
✅ Feature Tests: 23/23 passed
|
||||
✓ API endpoints: 8/8 passed
|
||||
✓ Database operations: 10/10 passed
|
||||
✓ Service integrations: 5/5 passed
|
||||
|
||||
Operations Tested:
|
||||
✓ GET /api/user/theme → 200 OK (45ms)
|
||||
✓ POST /api/user/theme → 200 OK (67ms)
|
||||
✓ Theme preference persisted to database
|
||||
✓ Cache invalidation on theme change
|
||||
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
🌐 COMPATIBILITY TESTING
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
✅ Browsers: Chrome, Firefox, Safari (all passed)
|
||||
✅ Devices: Desktop (1920x1080), Tablet (768x1024), Mobile (375x667)
|
||||
✅ Viewports: All responsive breakpoints validated
|
||||
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
⚡ PERFORMANCE TESTING
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
✅ Response Times: Within targets
|
||||
✓ Theme toggle: 45ms (target: <100ms)
|
||||
✓ Initial page load: 234ms (target: <500ms)
|
||||
✅ Throughput: 250 req/sec (target: >100 req/sec)
|
||||
✅ Memory: Stable (no leaks detected)
|
||||
✅ Database Queries: Optimized (N+1 prevented)
|
||||
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
🔒 SECURITY TESTING
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
✅ Vulnerability Scan: No issues (composer audit)
|
||||
✅ Authentication: All protected routes secure
|
||||
✅ CSRF Protection: Enabled and functional
|
||||
✅ XSS Prevention: Input sanitization verified
|
||||
✅ Rate Limiting: 60 requests/minute enforced
|
||||
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
📊 SUMMARY
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
✅ ALL QUALITY GATES PASSED
|
||||
|
||||
Total Tests: 76/76 passed
|
||||
Total Coverage: 87%
|
||||
Total Execution Time: 45.67s
|
||||
Browser Test Coverage: 100% of acceptance criteria
|
||||
Performance: All targets met
|
||||
Security: No vulnerabilities
|
||||
```
|
||||
|
||||
#### Phase 7: Story File Updates
|
||||
1. **UPDATE** QA Checklist based on test results:
|
||||
|
||||
**FUNCTIONAL TESTING:**
|
||||
- `[x]` Each acceptance criterion validated by browser tests
|
||||
- `[x]` All user workflows tested end-to-end
|
||||
- `[x]` Error scenarios handled gracefully
|
||||
- `[x]` Edge cases covered
|
||||
|
||||
**INTEGRATION TESTING:**
|
||||
- `[x]` API endpoints return correct responses
|
||||
- `[x]` Database operations work correctly
|
||||
- `[x]` Service integrations functional
|
||||
- `[x]` External APIs integrated properly
|
||||
|
||||
**COMPATIBILITY TESTING:**
|
||||
- `[x]` Works across target browsers
|
||||
- `[x]` Responsive on all devices
|
||||
- `[x]` Accessible via keyboard
|
||||
- `[x]` Screen reader compatible
|
||||
|
||||
**PERFORMANCE TESTING:**
|
||||
- `[x]` Response times within targets
|
||||
- `[x]` No memory leaks
|
||||
- `[x]` Optimized database queries
|
||||
- `[x]` Bundle size acceptable
|
||||
|
||||
**REGRESSION TESTING:**
|
||||
- `[x]` Existing features still work
|
||||
- `[x]` No unintended side effects
|
||||
|
||||
**SECURITY TESTING:**
|
||||
- `[x]` No vulnerabilities introduced
|
||||
- `[x]` Authentication/Authorization working
|
||||
- `[x]` Input validation functional
|
||||
|
||||
2. **UPDATE** Implementation Checklist remaining items:
|
||||
- `[x]` Browser tests (if now at 100% acceptance criteria coverage)
|
||||
- `[x]` Documentation (if QA revealed complete docs)
|
||||
|
||||
3. **ADD** to Progress Log:
|
||||
```markdown
|
||||
- [Today]: Moved to QA stage
|
||||
- [Today]: Executed comprehensive test suite
|
||||
* Unit tests: 45/45 passed (87% coverage)
|
||||
* Feature tests: 23/23 passed
|
||||
* Browser tests: 8/8 passed (100% criteria coverage)
|
||||
* Performance: All targets met
|
||||
* Security: No vulnerabilities
|
||||
- [Today]: All quality gates PASSED
|
||||
```
|
||||
|
||||
4. **RECORD** test artifacts:
|
||||
- Test report locations
|
||||
- Screenshot/video paths
|
||||
- Coverage report path
|
||||
- Performance benchmark results
|
||||
|
||||
#### Phase 8: Next Steps
|
||||
1. **DETERMINE** QA outcome:
|
||||
- IF all tests PASS → Ready for `/sdd:story-validate`
|
||||
- IF any critical failures → Requires `/sdd:story-refactor`
|
||||
- IF performance issues → Optimize and re-run QA
|
||||
|
||||
2. **DISPLAY** next actions:
|
||||
```
|
||||
💡 NEXT STEPS:
|
||||
════════════════════════════════════════════════
|
||||
|
||||
[IF ALL PASSED:]
|
||||
✅ All QA tests passed - Ready for validation
|
||||
1. /sdd:story-validate [story-id] # Final validation before ship
|
||||
2. /sdd:story-ship [story-id] # Deploy to production (after validation)
|
||||
|
||||
[IF FAILURES:]
|
||||
⚠️ X test(s) failed
|
||||
1. /sdd:story-refactor [story-id] # Return to development
|
||||
2. Fix failing tests:
|
||||
- [Test 1 that failed]
|
||||
- [Test 2 that failed]
|
||||
3. /sdd:story-review [story-id] # Re-run review
|
||||
4. /sdd:story-qa [story-id] # Re-run QA after fixes
|
||||
|
||||
[ARTIFACT LOCATIONS:]
|
||||
📸 Screenshots: /storage/screenshots/[story-id]/
|
||||
🎥 Videos: /storage/videos/[story-id]/
|
||||
📊 Coverage: /storage/coverage/[story-id]/
|
||||
📈 Performance: /storage/benchmarks/[story-id]/
|
||||
```
|
||||
|
||||
3. **SHOW** debugging commands for discovered stack:
|
||||
```bash
|
||||
# Laravel:
|
||||
vendor/bin/pest --filter=Browser --parallel=false # Run browser tests sequentially
|
||||
vendor/bin/pest --filter=Feature::testName # Run specific test
|
||||
php artisan telescope:prune # Clear performance logs
|
||||
|
||||
# Node.js Playwright:
|
||||
npx playwright test --debug # Debug mode
|
||||
npx playwright show-report # View HTML report
|
||||
npx playwright codegen # Generate new test code
|
||||
|
||||
# Python Pytest:
|
||||
pytest tests/browser/ -v -s # Verbose with print output
|
||||
pytest tests/browser/ --headed --slowmo=1000 # Visual debugging
|
||||
```
|
||||
|
||||
### OUTPUTS
|
||||
- `/docs/stories/qa/[story-id].md` - Updated story file with QA results
|
||||
- Automated QA report (displayed to user)
|
||||
- Test artifacts (screenshots, videos, reports)
|
||||
- Updated QA Checklist with validation status
|
||||
- Progress log entry with QA timestamp
|
||||
|
||||
### RULES
|
||||
- MUST load project context before running any tests
|
||||
- MUST use discovered testing tools from technical-stack.md
|
||||
- MUST validate against coding-standards.md thresholds
|
||||
- MUST move story file from review to qa
|
||||
- MUST run ALL test types (unit, feature, browser)
|
||||
- MUST validate 100% of acceptance criteria via browser tests
|
||||
- SHOULD capture test artifacts for documentation
|
||||
- SHOULD test across browsers/devices if specified
|
||||
- NEVER skip browser tests (critical for acceptance criteria)
|
||||
- NEVER allow QA to pass if critical tests fail
|
||||
- ALWAYS block progression if quality gates fail
|
||||
- ALWAYS update checklist items accurately
|
||||
- ALWAYS record test artifact locations
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: QA with All Tests Passing
|
||||
```bash
|
||||
INPUT:
|
||||
/sdd:story-qa STORY-2025-003
|
||||
|
||||
OUTPUT:
|
||||
→ Loading project context...
|
||||
✓ technical-stack.md: Laravel + Pest + Playwright
|
||||
✓ coding-standards.md: 80% coverage, all browsers
|
||||
✓ development-process.md: QA requirements
|
||||
|
||||
→ Moving story to QA...
|
||||
✓ Moved STORY-2025-003.md to /docs/stories/qa/
|
||||
|
||||
→ Executing test suite...
|
||||
[Unit Tests] Running Pest...
|
||||
✓ 45/45 passed (87% coverage) - 2.34s
|
||||
|
||||
[Feature Tests] Running Pest...
|
||||
✓ 23/23 passed - 5.67s
|
||||
|
||||
[Browser Tests] Running Pest Browser...
|
||||
✓ 8/8 passed - 34.21s
|
||||
✓ All 5 acceptance criteria validated
|
||||
📸 Screenshots saved to /storage/screenshots/STORY-2025-003/
|
||||
|
||||
[Performance] Running benchmarks...
|
||||
✓ Theme toggle: 45ms (target: <100ms)
|
||||
✓ Page load: 234ms (target: <500ms)
|
||||
|
||||
[Security] Running Composer audit...
|
||||
✓ No vulnerabilities
|
||||
|
||||
✅ AUTOMATED QA RESULTS
|
||||
════════════════════════════════════════════════
|
||||
Story: STORY-2025-003 - Dark Mode Toggle
|
||||
Stack: Laravel 12, Livewire 3, Pest 4, Playwright
|
||||
QA Executed: 2025-10-01 16:45:23
|
||||
|
||||
✅ ALL QUALITY GATES PASSED
|
||||
|
||||
Total Tests: 76/76 passed
|
||||
Total Coverage: 87%
|
||||
Browser Test Coverage: 100% of acceptance criteria
|
||||
Performance: All targets met
|
||||
Security: No vulnerabilities
|
||||
|
||||
💡 NEXT STEPS:
|
||||
════════════════════════════════════════════════
|
||||
✅ All QA tests passed - Ready for validation
|
||||
1. /sdd:story-validate STORY-2025-003 # Final validation before ship
|
||||
2. /sdd:story-ship STORY-2025-003 # Deploy to production (after validation)
|
||||
```
|
||||
|
||||
### Example 2: QA with Browser Test Failures
|
||||
```bash
|
||||
INPUT:
|
||||
/sdd:story-qa
|
||||
|
||||
OUTPUT:
|
||||
→ Detecting story from current branch...
|
||||
✓ Current branch: feature/STORY-2025-004-notifications
|
||||
✓ Story ID: STORY-2025-004
|
||||
|
||||
→ Loading project context...
|
||||
✓ technical-stack.md loaded
|
||||
|
||||
→ Moving story to QA...
|
||||
✓ Moved STORY-2025-004.md to /docs/stories/qa/
|
||||
|
||||
→ Executing test suite...
|
||||
[Unit Tests] Running Pest...
|
||||
✓ 32/32 passed (92% coverage) - 1.87s
|
||||
|
||||
[Feature Tests] Running Pest...
|
||||
✓ 18/18 passed - 4.23s
|
||||
|
||||
[Browser Tests] Running Pest Browser...
|
||||
✗ 2/5 passed - 3 failed
|
||||
✗ Failed tests:
|
||||
- NotificationTest::test_notification_appears
|
||||
- NotificationTest::test_notification_dismisses
|
||||
- NotificationTest::test_notification_persistence
|
||||
|
||||
✅ AUTOMATED QA RESULTS
|
||||
════════════════════════════════════════════════
|
||||
Story: STORY-2025-004 - Notification System
|
||||
Stack: Laravel 12, Pest 4, Playwright
|
||||
QA Executed: 2025-10-01 17:12:45
|
||||
|
||||
❌ QUALITY GATES FAILED
|
||||
|
||||
Unit Tests: ✅ 32/32 passed
|
||||
Feature Tests: ✅ 18/18 passed
|
||||
Browser Tests: ❌ 2/5 passed (3 failed)
|
||||
Acceptance Criteria Coverage: 40% (2/5 criteria validated)
|
||||
|
||||
❌ BLOCKING ISSUES:
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
Browser Tests:
|
||||
✗ NotificationTest::test_notification_appears
|
||||
Element not found: [wire:id="notification-toast"]
|
||||
|
||||
✗ NotificationTest::test_notification_dismisses
|
||||
Click failed: Element not visible
|
||||
|
||||
✗ NotificationTest::test_notification_persistence
|
||||
Timeout waiting for element
|
||||
|
||||
💡 NEXT STEPS:
|
||||
════════════════════════════════════════════════
|
||||
⚠️ 3 browser tests failed
|
||||
1. /sdd:story-refactor STORY-2025-004 # Return to development
|
||||
2. Fix failing browser tests:
|
||||
- Verify notification component is rendered
|
||||
- Check Livewire component IDs are correct
|
||||
- Ensure dismiss button is visible
|
||||
3. /sdd:story-review STORY-2025-004 # Re-run review
|
||||
4. /sdd:story-qa STORY-2025-004 # Re-run QA after fixes
|
||||
|
||||
[DEBUGGING COMMANDS:]
|
||||
vendor/bin/pest --filter=Browser::NotificationTest # Run specific test
|
||||
vendor/bin/pest --filter=Browser --parallel=false # Run sequentially
|
||||
php artisan serve # Start server for manual testing
|
||||
```
|
||||
|
||||
### Example 3: QA with Performance Issues
|
||||
```bash
|
||||
INPUT:
|
||||
/sdd:story-qa STORY-2025-005
|
||||
|
||||
OUTPUT:
|
||||
→ Loading project context...
|
||||
✓ technical-stack.md: Laravel + Pest + k6
|
||||
|
||||
→ Moving story to QA...
|
||||
✓ Moved STORY-2025-005.md to /docs/stories/qa/
|
||||
|
||||
→ Executing test suite...
|
||||
[Unit Tests] ✓ 28/28 passed (85% coverage)
|
||||
[Feature Tests] ✓ 15/15 passed
|
||||
[Browser Tests] ✓ 6/6 passed
|
||||
|
||||
[Performance] Running k6 benchmarks...
|
||||
⚠️ Search endpoint: 450ms (target: <200ms)
|
||||
⚠️ Database queries: 15 queries (N+1 detected)
|
||||
|
||||
✅ AUTOMATED QA RESULTS
|
||||
════════════════════════════════════════════════
|
||||
Story: STORY-2025-005 - Advanced Search
|
||||
Stack: Laravel 12, Pest 4, k6
|
||||
QA Executed: 2025-10-01 18:34:12
|
||||
|
||||
⚠️ PERFORMANCE ISSUES DETECTED
|
||||
|
||||
All tests: ✅ PASSED
|
||||
Performance: ⚠️ Below targets
|
||||
|
||||
❌ BLOCKING ISSUES:
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
Performance:
|
||||
✗ Search response time: 450ms (target: <200ms)
|
||||
✗ N+1 query problem detected in SearchController
|
||||
|
||||
🔧 SUGGESTED OPTIMIZATIONS:
|
||||
→ Add eager loading: ->with(['category', 'user'])
|
||||
→ Add database index on search_terms column
|
||||
→ Implement search results caching
|
||||
|
||||
💡 NEXT STEPS:
|
||||
════════════════════════════════════════════════
|
||||
1. /sdd:story-refactor STORY-2025-005 # Optimize performance
|
||||
2. Add eager loading and indexes
|
||||
3. /sdd:story-qa STORY-2025-005 # Re-run QA with optimizations
|
||||
```
|
||||
|
||||
## Edge Cases
|
||||
|
||||
### No Project Context
|
||||
- DETECT missing `/docs/project-context/` directory
|
||||
- SUGGEST running `/sdd:project-init`
|
||||
- ERROR: Cannot determine testing tools without context
|
||||
- EXIT with guidance
|
||||
|
||||
### Story Not in Review
|
||||
- CHECK if story in `/docs/stories/development/`
|
||||
- IF found: ERROR "Story must pass review first"
|
||||
- SUGGEST: `/sdd:story-review [story-id]` first
|
||||
- IF in `/docs/stories/qa/`: ASK if user wants to re-run QA
|
||||
|
||||
### No Browser Tests Found
|
||||
- DETECT missing browser test files
|
||||
- ERROR: "Browser tests required for acceptance criteria validation"
|
||||
- PROVIDE test file path examples for stack
|
||||
- SUGGEST: Create browser tests before QA
|
||||
|
||||
### Browser Test Coverage < 100% Criteria
|
||||
- COUNT acceptance criteria in story
|
||||
- COUNT passing browser tests
|
||||
- CALCULATE coverage gap
|
||||
- BLOCK QA progression
|
||||
- LIST uncovered criteria
|
||||
|
||||
### Performance Benchmarks Not Defined
|
||||
- CHECK if performance targets in story
|
||||
- IF missing: SKIP performance testing
|
||||
- WARN: "No performance targets defined"
|
||||
- CONTINUE with other tests
|
||||
|
||||
### Test Framework Not Installed
|
||||
- DETECT missing testing tools
|
||||
- PROVIDE installation commands for stack
|
||||
- ERROR: Cannot proceed without test framework
|
||||
- EXIT with setup instructions
|
||||
|
||||
### Flaky Browser Tests
|
||||
- DETECT intermittent failures
|
||||
- RETRY failed tests (up to 3 times)
|
||||
- IF still failing: MARK as FAILED
|
||||
- SUGGEST: Investigate timing/race conditions
|
||||
|
||||
## Error Handling
|
||||
- **Missing /docs/project-context/**: Suggest `/sdd:project-init`, exit gracefully
|
||||
- **Story not in review**: Provide clear workflow guidance
|
||||
- **Test framework errors**: Capture full error, suggest fixes
|
||||
- **Browser test timeouts**: Increase timeout, suggest element inspection
|
||||
- **Performance test failures**: Provide optimization suggestions
|
||||
- **Security vulnerabilities**: Block with specific CVE details
|
||||
|
||||
## Performance Considerations
|
||||
- Run unit, feature, and browser tests in parallel when possible
|
||||
- Stream test output in real-time (don't wait for all tests)
|
||||
- Cache test database between feature tests
|
||||
- Reuse browser instances in browser tests
|
||||
- Limit performance benchmarks to changed endpoints
|
||||
|
||||
## Related Commands
|
||||
- `/sdd:story-review [id]` - Must pass before QA
|
||||
- `/sdd:story-validate [id]` - Run after QA passes
|
||||
- `/sdd:story-refactor [id]` - Return to development if QA fails
|
||||
- `/sdd:story-ship [id]` - Deploy after validation
|
||||
- `/sdd:story-status [id]` - Check current state
|
||||
|
||||
## Constraints
|
||||
- ✅ MUST load project context before tests
|
||||
- ✅ MUST move story file to qa directory
|
||||
- ✅ MUST run ALL test types (unit, feature, browser)
|
||||
- ✅ MUST validate 100% acceptance criteria via browser tests
|
||||
- ⚠️ NEVER skip browser tests
|
||||
- ⚠️ NEVER allow QA to pass with critical failures
|
||||
- 📋 MUST capture test artifacts (screenshots, videos)
|
||||
- 🔧 SHOULD test across browsers/devices if specified
|
||||
- 💾 MUST update progress log with test results
|
||||
- 🚫 BLOCK validation if quality gates fail
|
||||
Reference in New Issue
Block a user