Initial commit
This commit is contained in:
@@ -0,0 +1,444 @@
|
||||
# Manual Testing Results Comment Template
|
||||
|
||||
## Purpose
|
||||
|
||||
This template defines the standardized format for Linear comments created by ln-343-manual-tester (invoked by ln-340-story-quality-gate Pass 1). The structured format ensures reliable parsing by ln-350-story-test-planner for E2E-first test design.
|
||||
|
||||
## Format Version
|
||||
|
||||
**Current Version:** 1.0
|
||||
**Last Updated:** 2025-10-31
|
||||
|
||||
## Template Structure
|
||||
|
||||
```markdown
|
||||
## 🧪 Manual Testing Results
|
||||
|
||||
**Format Version:** 1.0
|
||||
**Story ID:** [Story identifier, e.g., US042]
|
||||
**Tested By:** ln-343-manual-tester
|
||||
**Date:** [YYYY-MM-DD]
|
||||
**Status:** [✅ PASSED (X/Y AC) | ❌ FAILED (X/Y AC)]
|
||||
|
||||
---
|
||||
|
||||
### Acceptance Criteria (from Story)
|
||||
|
||||
**AC1:** [AC title/description]
|
||||
- **Given:** [Precondition]
|
||||
- **When:** [Action]
|
||||
- **Then:** [Expected outcome]
|
||||
|
||||
**AC2:** [AC title/description]
|
||||
- **Given:** [Precondition]
|
||||
- **When:** [Action]
|
||||
- **Then:** [Expected outcome]
|
||||
|
||||
[Repeat for each AC in Story]
|
||||
|
||||
---
|
||||
|
||||
### Test Results by AC
|
||||
|
||||
**AC1: [AC title]**
|
||||
- [✅ PASS | ❌ FAIL] **Status:** [PASS|FAIL]
|
||||
- **Method:** [Full curl command OR puppeteer code]
|
||||
- **Result:** [Actual HTTP status, response body, or UI state]
|
||||
- **Notes:** [Any relevant observations]
|
||||
|
||||
**AC2: [AC title]**
|
||||
- [✅ PASS | ❌ FAIL] **Status:** [PASS|FAIL]
|
||||
- **Method:** [Full curl command OR puppeteer code]
|
||||
- **Result:** [Actual response or behavior]
|
||||
- **Notes:** [Any relevant observations]
|
||||
|
||||
[Repeat for each AC]
|
||||
|
||||
---
|
||||
|
||||
### Edge Cases Discovered
|
||||
|
||||
1. **[Edge case description]**
|
||||
- **Input:** [Specific input that triggers edge case]
|
||||
- **Expected:** [Expected behavior]
|
||||
- **Actual:** [Actual behavior observed]
|
||||
- [✅ PASS | ❌ FAIL] **Status:** [PASS|FAIL]
|
||||
|
||||
2. **[Edge case description]**
|
||||
- **Input:** [Specific input]
|
||||
- **Expected:** [Expected behavior]
|
||||
- **Actual:** [Actual behavior]
|
||||
- [✅ PASS | ❌ FAIL] **Status:** [PASS|FAIL]
|
||||
|
||||
[Continue numbering for all discovered edge cases]
|
||||
|
||||
---
|
||||
|
||||
### Error Handling Verified
|
||||
|
||||
| HTTP Code | Scenario | Error Message | Verified |
|
||||
|-----------|----------|---------------|----------|
|
||||
| [Code] | [What triggers this error] | [Exact error message returned] | [✅ | ❌ | ⚠️ Not tested] |
|
||||
| [Code] | [Scenario] | [Error message] | [✅ | ❌] |
|
||||
|
||||
[Add all HTTP error codes tested: 400, 401, 403, 404, 429, 500, etc.]
|
||||
|
||||
---
|
||||
|
||||
### Integration Testing
|
||||
|
||||
**[Component A] → [Component B] → [Component C] Flow:**
|
||||
- [✅ | ❌] [Description of integration point 1]
|
||||
- [✅ | ❌] [Description of integration point 2]
|
||||
- [✅ | ❌] [Description of integration point 3]
|
||||
|
||||
**Transaction Handling:**
|
||||
- [✅ | ❌] [Transaction behavior description]
|
||||
- [✅ | ❌] [Rollback behavior if applicable]
|
||||
|
||||
**Performance/Concurrency (if applicable):**
|
||||
- [✅ | ❌] [Any performance observations]
|
||||
|
||||
---
|
||||
|
||||
### Summary
|
||||
|
||||
**Overall Result:** [✅ ALL ACCEPTANCE CRITERIA PASSED | ❌ X/Y ACCEPTANCE CRITERIA FAILED]
|
||||
|
||||
**Coverage:**
|
||||
- [X/Y] AC verified [✅ | ❌]
|
||||
- [X] edge cases tested [✅]
|
||||
- [X/Y] error scenarios verified [✅ | ⚠️]
|
||||
- Integration flow validated [✅ | ❌]
|
||||
|
||||
**Recommendation:** [Proceed to test task creation via ln-350-story-test-planner | Create refactoring task for issues found]
|
||||
|
||||
---
|
||||
|
||||
### Risk Assessment for Test Planning
|
||||
|
||||
**Purpose:** Provide Priority scores for ln-350-story-test-planner to select tests based on business risk
|
||||
|
||||
| Scenario | Type | Business Impact (1-5) | Probability (1-5) | Priority | Reason |
|
||||
|----------|------|----------------------|-------------------|----------|--------|
|
||||
| [AC1: AC title] | AC | [1-5] | [1-5] | [Result] | [Why this impact/probability] |
|
||||
| [AC2: AC title] | AC | [1-5] | [1-5] | [Result] | [Why this impact/probability] |
|
||||
| [Edge Case 1: description] | Edge Case | [1-5] | [1-5] | [Result] | [Why this impact/probability] |
|
||||
| [Edge Case 2: description] | Edge Case | [1-5] | [1-5] | [Result] | [Why this impact/probability] |
|
||||
| [Error: HTTP 400 scenario] | Error Handling | [1-5] | [1-5] | [Result] | [Why this impact/probability] |
|
||||
| [Error: HTTP 401 scenario] | Error Handling | [1-5] | [1-5] | [Result] | [Why this impact/probability] |
|
||||
|
||||
**Priority Calculation:** Priority = Business Impact (1-5) × Probability (1-5)
|
||||
|
||||
**Decision Criteria:**
|
||||
- Priority ≥15 → MUST test (ln-350-story-test-planner will create automated tests)
|
||||
- Priority 9-14 → SHOULD test if not already covered
|
||||
- Priority ≤8 → SKIP (manual testing sufficient)
|
||||
|
||||
**Reference:** See `ln-350-story-test-planner/references/risk_based_testing_guide.md` for complete Business Impact/Probability scoring tables and methodology.
|
||||
|
||||
**Total Scenarios:** [X scenarios], **Priority ≥15:** [Y scenarios] (will be tested)
|
||||
```
|
||||
|
||||
## Usage Instructions
|
||||
|
||||
### For ln-343-manual-tester (Phase 5 Step 1)
|
||||
|
||||
1. **Copy template structure** (do NOT include this instruction section)
|
||||
2. **Fill required fields:**
|
||||
- Story ID from Linear
|
||||
- Current date in YYYY-MM-DD format
|
||||
- Status calculated from AC pass/fail count
|
||||
3. **Extract AC from Story description:**
|
||||
- Copy Given-When-Then exactly as written in Story
|
||||
- Maintain numbering (AC1, AC2, AC3...)
|
||||
4. **Document test results for EACH AC:**
|
||||
- Include full curl command or puppeteer code used
|
||||
- Copy exact HTTP status codes and response bodies
|
||||
- Note any deviations from expected behavior
|
||||
5. **List ALL edge cases discovered** during testing:
|
||||
- Enumerate sequentially (1, 2, 3...)
|
||||
- Provide concrete input/expected/actual values
|
||||
6. **Create error handling table:**
|
||||
- Test all error codes mentioned in Story Technical Notes
|
||||
- Include 400, 401, 404, 500 at minimum
|
||||
- Mark ⚠️ for codes not testable without setup
|
||||
7. **Verify integration flow:**
|
||||
- Trace request through all architectural layers
|
||||
- Note any transaction/rollback behavior
|
||||
8. **Write summary:**
|
||||
- Count passed AC vs total AC
|
||||
- Recommend next action (ln-350-story-test-planner or refactoring task)
|
||||
|
||||
### For ln-350-story-test-planner (Phase 2 Step 1)
|
||||
|
||||
**Parsing strategy:**
|
||||
|
||||
1. **Find comment with marker:**
|
||||
- Search for `## 🧪 Manual Testing Results`
|
||||
- Verify `**Format Version:** 1.0` present
|
||||
|
||||
2. **Extract sections using regex:**
|
||||
- `^### Acceptance Criteria` → parse AC with Given-When-Then
|
||||
- `^### Test Results by AC` → extract status, method, results per AC
|
||||
- `^### Edge Cases Discovered` → parse numbered list items
|
||||
- `^### Error Handling Verified` → parse markdown table
|
||||
- `^### Integration Testing` → extract component flows
|
||||
|
||||
3. **Map to test design:**
|
||||
- Each PASSED AC → 1 E2E test (copy method from "Method:" field)
|
||||
- Each edge case → Unit or Integration test
|
||||
- Each verified error code → Error handling test
|
||||
- Integration flow → Integration test suite
|
||||
|
||||
4. **Handle parsing errors:**
|
||||
- Missing Format Version → warn user, try legacy parsing
|
||||
- Missing required section → error with clear message
|
||||
- Cannot parse AC → request Story description fix
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: API Endpoint Testing
|
||||
|
||||
```markdown
|
||||
## 🧪 Manual Testing Results
|
||||
|
||||
**Format Version:** 1.0
|
||||
**Story ID:** US042
|
||||
**Tested By:** ln-343-manual-tester
|
||||
**Date:** 2025-10-31
|
||||
**Status:** ✅ PASSED (3/3 AC)
|
||||
|
||||
---
|
||||
|
||||
### Acceptance Criteria (from Story)
|
||||
|
||||
**AC1:** User can login with valid credentials
|
||||
- **Given:** Valid email and password
|
||||
- **When:** User submits login form
|
||||
- **Then:** Returns 200 OK with JWT token
|
||||
|
||||
**AC2:** Invalid credentials are rejected
|
||||
- **Given:** Invalid email or password
|
||||
- **When:** User submits login form
|
||||
- **Then:** Returns 401 Unauthorized with error message
|
||||
|
||||
**AC3:** Rate limiting prevents brute force
|
||||
- **Given:** More than 5 failed login attempts within 1 minute
|
||||
- **When:** User submits 6th attempt
|
||||
- **Then:** Returns 429 Too Many Requests
|
||||
|
||||
---
|
||||
|
||||
### Test Results by AC
|
||||
|
||||
**AC1: User can login with valid credentials**
|
||||
- ✅ **Status:** PASS
|
||||
- **Method:** `curl -X POST http://localhost:8000/api/auth/login -H "Content-Type: application/json" -d '{"email":"test@example.com","password":"SecurePass123"}'`
|
||||
- **Result:** 200 OK, JWT token received: `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...`
|
||||
- **Notes:** Token validated successfully, expires in 1 hour
|
||||
|
||||
**AC2: Invalid credentials are rejected**
|
||||
- ✅ **Status:** PASS
|
||||
- **Method:** `curl -X POST http://localhost:8000/api/auth/login -H "Content-Type: application/json" -d '{"email":"test@example.com","password":"WrongPassword"}'`
|
||||
- **Result:** 401 Unauthorized, `{"error":"Invalid credentials"}`
|
||||
- **Notes:** Error message does not reveal if email or password is wrong (good security practice)
|
||||
|
||||
**AC3: Rate limiting prevents brute force**
|
||||
- ✅ **Status:** PASS
|
||||
- **Method:** Bash loop: `for i in {1..6}; do curl -X POST http://localhost:8000/api/auth/login -d '{"email":"test@example.com","password":"wrong"}'; done`
|
||||
- **Result:** First 5 attempts → 401, 6th attempt → 429 with `{"error":"Too many requests, try again in 52 seconds"}`
|
||||
- **Notes:** Rate limit counter resets correctly after 1 minute
|
||||
|
||||
---
|
||||
|
||||
### Edge Cases Discovered
|
||||
|
||||
1. **Empty email field**
|
||||
- **Input:** `{"email":"","password":"test123"}`
|
||||
- **Expected:** 400 Bad Request
|
||||
- **Actual:** 400 Bad Request with `{"error":"Email is required"}`
|
||||
- ✅ **Status:** PASS
|
||||
|
||||
2. **SQL injection attempt**
|
||||
- **Input:** `{"email":"'; DROP TABLE users;--","password":"test"}`
|
||||
- **Expected:** Properly escaped, 401 Invalid credentials
|
||||
- **Actual:** 401 Invalid credentials, SQL not executed (verified in logs)
|
||||
- ✅ **Status:** PASS
|
||||
|
||||
3. **Unicode characters in password**
|
||||
- **Input:** Password: `Test🔒Pass123`
|
||||
- **Expected:** Works correctly
|
||||
- **Actual:** Login successful, password stored and validated with UTF-8 encoding
|
||||
- ✅ **Status:** PASS
|
||||
|
||||
4. **Very long password (1000 chars)**
|
||||
- **Input:** Password with 1000 'a' characters
|
||||
- **Expected:** 400 Bad Request (max length validation)
|
||||
- **Actual:** 400 Bad Request with `{"error":"Password too long (max 128 characters)"}`
|
||||
- ✅ **Status:** PASS
|
||||
|
||||
---
|
||||
|
||||
### Error Handling Verified
|
||||
|
||||
| HTTP Code | Scenario | Error Message | Verified |
|
||||
|-----------|----------|---------------|----------|
|
||||
| 400 | Missing email field | "Email is required" | ✅ |
|
||||
| 400 | Invalid email format | "Invalid email format" | ✅ |
|
||||
| 400 | Missing password field | "Password is required" | ✅ |
|
||||
| 400 | Password too long | "Password too long (max 128 characters)" | ✅ |
|
||||
| 401 | Wrong email | "Invalid credentials" | ✅ |
|
||||
| 401 | Wrong password | "Invalid credentials" | ✅ |
|
||||
| 429 | Rate limit exceeded | "Too many requests, try again in X seconds" | ✅ |
|
||||
| 500 | Database connection error | "Internal server error" | ⚠️ Not tested (requires DB failure simulation) |
|
||||
|
||||
---
|
||||
|
||||
### Integration Testing
|
||||
|
||||
**API → Service → Repository → Database Flow:**
|
||||
- ✅ API endpoint receives request and validates JSON schema
|
||||
- ✅ Service layer calls UserRepository.findByEmail()
|
||||
- ✅ Repository queries PostgreSQL users table
|
||||
- ✅ Password comparison using bcrypt.compare() works correctly
|
||||
- ✅ JWT token generated and signed with SECRET_KEY
|
||||
- ✅ Response formatted according to API spec
|
||||
|
||||
**Transaction Handling:**
|
||||
- ✅ Failed login attempt logged in audit_log table (INSERT)
|
||||
- ✅ Rate limit counter incremented in Redis
|
||||
- ✅ No database locks observed during concurrent login attempts
|
||||
|
||||
---
|
||||
|
||||
### Summary
|
||||
|
||||
**Overall Result:** ✅ **ALL ACCEPTANCE CRITERIA PASSED**
|
||||
|
||||
**Coverage:**
|
||||
- 3/3 AC verified ✅
|
||||
- 4 edge cases tested ✅
|
||||
- 7/8 error scenarios verified (1 requires failure injection) ✅
|
||||
- Integration flow validated ✅
|
||||
|
||||
**Recommendation:** Proceed to test task creation via ln-350-story-test-planner
|
||||
```
|
||||
|
||||
### Example 2: UI Testing with Puppeteer
|
||||
|
||||
```markdown
|
||||
## 🧪 Manual Testing Results
|
||||
|
||||
**Format Version:** 1.0
|
||||
**Story ID:** US045
|
||||
**Tested By:** ln-343-manual-tester
|
||||
**Date:** 2025-10-31
|
||||
**Status:** ✅ PASSED (2/2 AC)
|
||||
|
||||
---
|
||||
|
||||
### Acceptance Criteria (from Story)
|
||||
|
||||
**AC1:** User can see product list on homepage
|
||||
- **Given:** User navigates to homepage
|
||||
- **When:** Page loads
|
||||
- **Then:** Product grid displays with images, names, and prices
|
||||
|
||||
**AC2:** User can filter products by category
|
||||
- **Given:** User is on homepage with products displayed
|
||||
- **When:** User clicks category filter
|
||||
- **Then:** Only products from selected category are shown
|
||||
|
||||
---
|
||||
|
||||
### Test Results by AC
|
||||
|
||||
**AC1: User can see product list on homepage**
|
||||
- ✅ **Status:** PASS
|
||||
- **Method:**
|
||||
```javascript
|
||||
const page = await browser.newPage();
|
||||
await page.goto('http://localhost:3000');
|
||||
await page.waitForSelector('.product-grid');
|
||||
const products = await page.$$('.product-card');
|
||||
console.log(`Found ${products.length} products`);
|
||||
```
|
||||
- **Result:** 12 products displayed, all with images, names, and prices visible
|
||||
- **Notes:** Images load correctly, no broken thumbnails
|
||||
|
||||
**AC2: User can filter products by category**
|
||||
- ✅ **Status:** PASS
|
||||
- **Method:**
|
||||
```javascript
|
||||
await page.click('[data-category="electronics"]');
|
||||
await page.waitForTimeout(500); // Wait for filter animation
|
||||
const filteredProducts = await page.$$('.product-card[data-category="electronics"]');
|
||||
console.log(`Filtered to ${filteredProducts.length} electronics`);
|
||||
```
|
||||
- **Result:** Filter works, showing only 5 electronics products. Other categories hidden.
|
||||
- **Notes:** Filter animation smooth, no flickering
|
||||
|
||||
---
|
||||
|
||||
### Edge Cases Discovered
|
||||
|
||||
1. **Empty category returns "No products" message**
|
||||
- **Input:** Click category "Books" (which has 0 products)
|
||||
- **Expected:** Show "No products found" message
|
||||
- **Actual:** Message displayed correctly with suggestion to clear filters
|
||||
- ✅ **Status:** PASS
|
||||
|
||||
2. **Multiple rapid filter clicks**
|
||||
- **Input:** Click different category filters rapidly 5 times
|
||||
- **Expected:** UI remains stable, shows final selection
|
||||
- **Actual:** No race conditions, final filter applied correctly
|
||||
- ✅ **Status:** PASS
|
||||
|
||||
---
|
||||
|
||||
### Error Handling Verified
|
||||
|
||||
| HTTP Code | Scenario | Error Message | Verified |
|
||||
|-----------|----------|---------------|----------|
|
||||
| 404 | Navigate to /products/invalid-id | "Product not found" page | ✅ |
|
||||
| 500 | API returns error | "Failed to load products" toast | ⚠️ Not tested (requires API mock failure) |
|
||||
|
||||
---
|
||||
|
||||
### Integration Testing
|
||||
|
||||
**Frontend → API → Backend Flow:**
|
||||
- ✅ React component fetches from /api/products on mount
|
||||
- ✅ API returns JSON with product array
|
||||
- ✅ Product images loaded from CDN correctly
|
||||
- ✅ Category filter sends query param ?category=electronics
|
||||
- ✅ React state updates trigger re-render without full page reload
|
||||
|
||||
---
|
||||
|
||||
### Summary
|
||||
|
||||
**Overall Result:** ✅ **ALL ACCEPTANCE CRITERIA PASSED**
|
||||
|
||||
**Coverage:**
|
||||
- 2/2 AC verified ✅
|
||||
- 2 edge cases tested ✅
|
||||
- 1/2 error scenarios verified ✅
|
||||
- Integration flow validated ✅
|
||||
|
||||
**Recommendation:** Proceed to test task creation via ln-350-story-test-planner
|
||||
```
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 1.1 | 2025-10-31 | Added Risk Assessment section with Priority Matrix (Business Impact × Probability) for ln-350-story-test-planner |
|
||||
| 1.0 | 2025-10-31 | Initial structured format with AC, Test Results, Edge Cases, Errors, Integration |
|
||||
|
||||
## References
|
||||
|
||||
- ln-340-story-quality-gate SKILL.md Phase 5 Step 3
|
||||
- ln-350-story-test-planner SKILL.md Phase 2 Step 1
|
||||
- Story Template (story_template_universal.md) for AC format
|
||||
@@ -0,0 +1,394 @@
|
||||
# Story Review Checklist (14 Points)
|
||||
|
||||
Complete checklist for validating User Story after all tasks are Done.
|
||||
|
||||
## A. Goal Achievement (CRITICAL)
|
||||
|
||||
### 1. Story Statement Fulfilled
|
||||
|
||||
**Check:**
|
||||
- [ ] "As a [role]" - Does role get needed functionality?
|
||||
- [ ] "I want [capability]" - Is capability fully implemented?
|
||||
- [ ] "So that [value]" - Is business value delivered?
|
||||
|
||||
**Method:**
|
||||
- Code review of implemented functionality
|
||||
- Run E2E tests covering Story goal
|
||||
- Validate user can achieve stated goal
|
||||
|
||||
**Red Flags:**
|
||||
- Story goal partially implemented
|
||||
- Business value not delivered
|
||||
- Role cannot use functionality as intended
|
||||
|
||||
---
|
||||
|
||||
### 2. All Story AC Satisfied
|
||||
|
||||
**Check:**
|
||||
- [ ] Main Scenarios: All Given-When-Then scenarios work
|
||||
- [ ] Edge Cases: All boundary conditions handled
|
||||
- [ ] Error Handling: All error scenarios handled correctly
|
||||
|
||||
**Method:**
|
||||
- Compare Story AC (from Story description) with E2E tests
|
||||
- Verify each Given-When-Then has corresponding E2E test
|
||||
- Run all E2E tests - all must pass
|
||||
|
||||
**Red Flags:**
|
||||
- Story AC missing E2E test coverage
|
||||
- E2E tests fail
|
||||
- Edge cases not handled
|
||||
|
||||
---
|
||||
|
||||
## B. Integration Quality (CRITICAL)
|
||||
|
||||
### 3. Tasks Integrated Correctly
|
||||
|
||||
**Check:**
|
||||
- [ ] API → Service → Repository integration works
|
||||
- [ ] Contracts between layers honored (types, parameters match)
|
||||
- [ ] Data flow correct (no data loss, no corruption)
|
||||
- [ ] External dependencies integrated (APIs, databases)
|
||||
|
||||
**Method:**
|
||||
- Code review of integration points
|
||||
- Run Integration tests - all must pass
|
||||
- Check types match between layers
|
||||
|
||||
**Red Flags:**
|
||||
- Type mismatches (API sends `string`, Service expects `number`)
|
||||
- Missing integration (API doesn't call Service)
|
||||
- Integration tests fail
|
||||
|
||||
**Example Problem:**
|
||||
```
|
||||
Task 1 Done: API endpoint accepts user_id: string ✅
|
||||
Task 2 Done: Service expects userId: number ✅
|
||||
Problem: API calls Service with string → Runtime error ❌
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. No Gaps Between Tasks
|
||||
|
||||
**Check:**
|
||||
- [ ] All components are used (no orphaned methods)
|
||||
- [ ] All necessary calls present (no missing calls)
|
||||
- [ ] Component dependencies satisfied
|
||||
|
||||
**Method:**
|
||||
- Grep for unused functions/classes in Story files
|
||||
- Code review for missing calls
|
||||
- Check all Affected Components from tasks are integrated
|
||||
|
||||
**Red Flags:**
|
||||
- Unused methods (created but never called)
|
||||
- Missing calls (Service method exists but API doesn't call it)
|
||||
- Dead code
|
||||
|
||||
**Example Problem:**
|
||||
```
|
||||
Task 1 Done: sendEmail() function created ✅
|
||||
Task 2 Done: saveUser() function created ✅
|
||||
Problem: API endpoint calls saveUser() but NOT sendEmail() → Email not sent ❌
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## C. Overall Code Quality
|
||||
|
||||
### 5. Patterns Consistent
|
||||
|
||||
**Check:**
|
||||
- [ ] All tasks use same approaches (async/await vs callbacks)
|
||||
- [ ] No code duplication between tasks (DRY at Story level)
|
||||
- [ ] Error handling consistent across tasks
|
||||
- [ ] Architecture coherent
|
||||
- [ ] Configuration management consistent (no hardcoded values, no magic numbers)
|
||||
|
||||
**Method:**
|
||||
- Grep for similar code patterns across Story files
|
||||
- Code review for duplication
|
||||
- Check error handling patterns match
|
||||
|
||||
**Red Flags:**
|
||||
- Task 1 uses async/await, Task 2 uses callbacks
|
||||
- Same validation logic duplicated in 3 tasks
|
||||
- Inconsistent error handling (Task 1 throws, Task 2 returns null)
|
||||
- Magic numbers or hardcoded URLs in code (should be in config)
|
||||
|
||||
---
|
||||
|
||||
### 6. Following guides/
|
||||
|
||||
**Check:**
|
||||
- [ ] Story implemented per project patterns (from guides/)
|
||||
- [ ] Technical Notes from Story executed
|
||||
- [ ] Architectural patterns followed
|
||||
|
||||
**Method:**
|
||||
- Compare implementation with guides/ referenced in Story
|
||||
- Check Technical Notes section of Story
|
||||
- Verify patterns match guide examples
|
||||
|
||||
**Red Flags:**
|
||||
- Story says "Use Repository Pattern" but implementation uses direct DB calls
|
||||
- Technical Notes mention caching but no cache implemented
|
||||
|
||||
---
|
||||
|
||||
## D. Testing & Coverage (CRITICAL)
|
||||
|
||||
### 7. All Story Tests Pass
|
||||
|
||||
**Check:**
|
||||
- [ ] Unit tests (70%) pass
|
||||
- [ ] Integration tests (20%) pass
|
||||
- [ ] E2E tests (10%) pass
|
||||
- [ ] No flaky tests (all deterministic)
|
||||
- [ ] Tests focus on business logic (not frameworks/libraries/getters)
|
||||
- [ ] No test duplication - Each behavior tested once at pyramid level
|
||||
|
||||
**Method:**
|
||||
- Run all tests for Story files: `npm test` / `pytest` / `go test`
|
||||
- Check test output - 0 failures
|
||||
|
||||
**Red Flags:**
|
||||
- Any test fails
|
||||
- Flaky tests (pass sometimes, fail sometimes)
|
||||
- Tests skipped
|
||||
|
||||
---
|
||||
|
||||
### 8. Coverage Story ≥80%
|
||||
|
||||
**Check:**
|
||||
- [ ] Overall coverage across all Story files ≥80%
|
||||
- [ ] Integration layer covered (not just unit level)
|
||||
- [ ] Critical paths covered
|
||||
|
||||
**Method:**
|
||||
- Run coverage report for Story files
|
||||
- Check overall percentage
|
||||
- Review uncovered lines (should be non-critical)
|
||||
|
||||
**Red Flags:**
|
||||
- Coverage <80%
|
||||
- Critical business logic uncovered
|
||||
- Integration points uncovered
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Task 1 coverage: 90% ✅
|
||||
Task 2 coverage: 85% ✅
|
||||
Task 3 coverage: 80% ✅
|
||||
But: Story coverage: 65% ❌ (integration layer uncovered)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 9. Test Limits and Priority Scenarios
|
||||
|
||||
**Check:**
|
||||
- [ ] Test count within limits: 2-5 E2E, 3-8 Integration, 5-15 Unit (10-28 total)
|
||||
- [ ] All Priority ≥15 scenarios from manual testing tested
|
||||
- [ ] No duplicate test coverage (each test adds unique business value)
|
||||
- [ ] Trivial code skipped (simple CRUD, framework code, getters/setters)
|
||||
|
||||
**Method:**
|
||||
- Count tests by type for Story
|
||||
- Verify total tests 10-28
|
||||
- Check Risk Priority Matrix in test task - ensure all Priority ≥15 scenarios have tests
|
||||
- Verify no tests for frameworks, libraries, or trivial logic
|
||||
|
||||
**Red Flags:**
|
||||
- Total tests exceed 28 (maintenance burden)
|
||||
- Priority ≥15 scenarios not tested (critical paths uncovered)
|
||||
- Tests for framework code or trivial logic (waste of time)
|
||||
- Duplicate coverage (same scenario tested at multiple levels)
|
||||
|
||||
---
|
||||
|
||||
### 10. E2E Cover All Story AC
|
||||
|
||||
**Check:**
|
||||
- [ ] Each Main Scenario from Story AC has E2E test
|
||||
- [ ] Edge Cases from Story AC covered by E2E
|
||||
- [ ] Error Handling from Story AC covered by E2E
|
||||
|
||||
**Method:**
|
||||
- List all Story AC (Given-When-Then scenarios)
|
||||
- List all E2E tests
|
||||
- Match AC to E2E tests (1:1 mapping for main scenarios)
|
||||
|
||||
**Red Flags:**
|
||||
- Story AC without corresponding E2E test
|
||||
- E2E test doesn't match Story AC wording
|
||||
- Critical AC not covered
|
||||
|
||||
**Example Problem:**
|
||||
```
|
||||
Story AC: "Given user submits form, When submit, Then data saved AND email sent"
|
||||
E2E test: "test_user_can_save_data" ✅ (covers save)
|
||||
Missing: E2E test for email sent ❌
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## E. Guides & Infrastructure (NEW)
|
||||
|
||||
### 11. Guides Correct
|
||||
|
||||
**Check:**
|
||||
- [ ] All guides/ created in Story are correct
|
||||
- [ ] Guide template followed (from guide-creator)
|
||||
- [ ] Patterns documented accurately
|
||||
- [ ] Anti-patterns documented
|
||||
- [ ] Examples correct
|
||||
|
||||
**Method:**
|
||||
- Identify guides/ created/modified in Story tasks
|
||||
- Review each guide structure matches guide_template.md
|
||||
- Verify patterns are actually used in codebase
|
||||
- Check examples are correct and runnable
|
||||
|
||||
**Red Flags:**
|
||||
- Guide template not followed
|
||||
- Pattern documented but not used in code
|
||||
- Incorrect examples
|
||||
- Anti-patterns missing
|
||||
|
||||
**Skipped if:** No guides created in Story
|
||||
|
||||
---
|
||||
|
||||
### 12. Infrastructure Updated
|
||||
|
||||
**Check when packages added:**
|
||||
|
||||
**Package managers:**
|
||||
- [ ] package.json updated (Node.js)
|
||||
- [ ] requirements.txt updated (Python)
|
||||
- [ ] go.mod updated (Go)
|
||||
- [ ] Cargo.toml updated (Rust)
|
||||
- [ ] Gemfile updated (Ruby)
|
||||
|
||||
**Docker:**
|
||||
- [ ] Dockerfile updated if package needs system dependencies
|
||||
- Example: `RUN apt-get install -y libpq-dev` for PostgreSQL Python driver
|
||||
- [ ] docker-compose.yml updated if package needs services
|
||||
- Example: Redis, Postgres, MongoDB services added
|
||||
|
||||
**Documentation:**
|
||||
- [ ] README.md updated with setup instructions
|
||||
- Installation steps
|
||||
- New environment variables
|
||||
- New service dependencies
|
||||
|
||||
**Method:**
|
||||
- Check if packages were added in Story tasks (search for "npm install", "pip install", etc.)
|
||||
- If yes, verify infrastructure files updated
|
||||
- Run docker build to verify Dockerfile still works
|
||||
- Run docker-compose up to verify services start
|
||||
|
||||
**Red Flags:**
|
||||
- Package added but package.json not committed
|
||||
- Package needs system library but Dockerfile not updated
|
||||
- Package needs Redis but docker-compose.yml not updated
|
||||
- README missing setup instructions for new package
|
||||
|
||||
**Example Problems:**
|
||||
```
|
||||
Problem 1:
|
||||
- Task adds `pg` package (PostgreSQL driver)
|
||||
- package.json updated ✅
|
||||
- Dockerfile NOT updated ❌ (missing: RUN apt-get install -y libpq-dev)
|
||||
- Result: Docker build fails
|
||||
|
||||
Problem 2:
|
||||
- Task adds Redis caching
|
||||
- package.json updated ✅
|
||||
- docker-compose.yml NOT updated ❌ (missing Redis service)
|
||||
- Result: App can't connect to Redis
|
||||
|
||||
Problem 3:
|
||||
- Task adds complex package
|
||||
- package.json updated ✅
|
||||
- README NOT updated ❌ (missing setup instructions)
|
||||
- Result: Other developers can't run project
|
||||
```
|
||||
|
||||
**Skipped if:** No packages added in Story
|
||||
|
||||
---
|
||||
|
||||
## F. Completeness
|
||||
|
||||
### 13. All Tasks Done
|
||||
|
||||
**Check:**
|
||||
- [ ] No tasks in Todo
|
||||
- [ ] No tasks in In Progress
|
||||
- [ ] No tasks in To Review
|
||||
- [ ] No tasks in To Rework
|
||||
- [ ] All tasks in Done status
|
||||
|
||||
**Method:**
|
||||
- Linear query: `parentId=Story ID, status≠Done`
|
||||
- Result should be empty
|
||||
|
||||
**Red Flags:**
|
||||
- Any task not Done
|
||||
- Tasks forgotten
|
||||
|
||||
---
|
||||
|
||||
### 14. Documentation Complete
|
||||
|
||||
**Check:**
|
||||
- [ ] STRUCTURE.md updated (all new components documented)
|
||||
- [ ] ARCHITECTURE.md updated (all architectural changes documented)
|
||||
- [ ] guides/ current (if created in Story)
|
||||
- [ ] tests/README.md current (if test approach changed)
|
||||
- [ ] README.md updated (if setup changed)
|
||||
|
||||
**Method:**
|
||||
- Check files modified in Story tasks
|
||||
- Verify documentation sections updated
|
||||
- Cross-reference with Affected Components from tasks
|
||||
|
||||
**Red Flags:**
|
||||
- New component not in STRUCTURE.md
|
||||
- Architectural change not in ARCHITECTURE.md
|
||||
- Outdated documentation
|
||||
|
||||
---
|
||||
|
||||
## Summary Checklist
|
||||
|
||||
Quick checklist for ln-340-story-quality-gate Pass 1:
|
||||
|
||||
- [ ] 1. Story statement fulfilled
|
||||
- [ ] 2. All Story AC satisfied
|
||||
- [ ] 3. Tasks integrated correctly
|
||||
- [ ] 4. No gaps between tasks
|
||||
- [ ] 5. Patterns consistent
|
||||
- [ ] 6. Following guides/
|
||||
- [ ] 7. All tests pass
|
||||
- [ ] 8. Priority ≥15 scenarios tested
|
||||
- [ ] 9. Test limits (10-28 total)
|
||||
- [ ] 10. E2E cover all AC
|
||||
- [ ] 11. Guides correct
|
||||
- [ ] 12. Infrastructure updated
|
||||
- [ ] 13. All tasks Done
|
||||
- [ ] 14. Documentation complete
|
||||
|
||||
**Pass:** All 14 checks ✓ → Story Done
|
||||
**Fail:** Any check ✗ → Create fix tasks
|
||||
|
||||
---
|
||||
|
||||
**Version:** 1.3.0 (Configuration management check)
|
||||
**Last Updated:** 2025-11-07
|
||||
Reference in New Issue
Block a user