Initial commit
This commit is contained in:
Zhongwei Li
190
commands/ultimate_validate_command.md
Normal file
190
commands/ultimate_validate_command.md
Normal file
@@ -0,0 +1,190 @@
|
||||
---
|
||||
description: Generate comprehensive validation command for backend or frontend
|
||||
---
|
||||
|
||||
# Generate Ultimate Validation Command
|
||||
|
||||
Analyze a codebase deeply and create `.claude/commands/validate.md` that comprehensively validates everything.
|
||||
|
||||
**Usage:**
|
||||
```
|
||||
/ultimate_validate_command [backend|frontend]
|
||||
```
|
||||
|
||||
## Step 0: Discover Real User Workflows
|
||||
|
||||
**Before analyzing tooling, understand what users ACTUALLY do:**
|
||||
|
||||
1. Read workflow documentation:
|
||||
- README.md - Look for "Usage", "Quickstart", "Examples" sections
|
||||
- CLAUDE.md - Look for workflow patterns
|
||||
- docs/ folder - User guides, tutorials, feature documentation
|
||||
|
||||
2. Identify external integrations:
|
||||
- What CLIs does the app use? (Check Dockerfile, requirements.txt, package.json)
|
||||
- What external APIs does it call? (Google Calendar, Gmail, Telegram, LiveKit, etc.)
|
||||
- What services does it interact with? (AWS, Supabase, databases, etc.)
|
||||
|
||||
3. Extract complete user journeys from docs:
|
||||
- Find examples like "User asks voice agent → processes calendar → sends email"
|
||||
- Each workflow becomes an E2E test scenario
|
||||
|
||||
**Critical: Your E2E tests should mirror actual workflows from docs, not just test internal APIs.**
|
||||
|
||||
## Step 1: Deep Codebase Analysis
|
||||
|
||||
Navigate to the appropriate repo:
|
||||
```bash
|
||||
cd backend/ # or frontend/
|
||||
```
|
||||
|
||||
Explore the codebase to understand:
|
||||
|
||||
**What validation tools already exist:**
|
||||
- **Linting config:** `.eslintrc*`, `.pylintrc`, `ruff.toml`, `.flake8`, etc.
|
||||
- **Type checking:** `tsconfig.json`, `mypy.ini`, etc.
|
||||
- **Style/formatting:** `.prettierrc*`, `black`, `.editorconfig`
|
||||
- **Unit tests:** `jest.config.*`, `pytest.ini`, test directories
|
||||
- **Package manager scripts:** `package.json` scripts, `Makefile`, `pyproject.toml` tools
|
||||
|
||||
**What the application does:**
|
||||
- **Frontend:** Routes, pages, components, user flows (Next.js app)
|
||||
- **Backend:** API endpoints, voice agent, Google services integration, authentication
|
||||
- **Database:** Schema, migrations, models
|
||||
- **Infrastructure:** Docker services, dependencies, AWS resources
|
||||
|
||||
**How things are currently tested:**
|
||||
- Existing test files and patterns
|
||||
- CI/CD workflows (`.github/workflows/`, etc.)
|
||||
- Test commands in package.json or pyproject.toml
|
||||
|
||||
## Step 2: Generate validate.md
|
||||
|
||||
Create `.claude/commands/validate.md` in the **appropriate repo** (backend/ or frontend/) with these phases:
|
||||
|
||||
**ONLY include phases that exist in the codebase.**
|
||||
|
||||
### Phase 1: Linting
|
||||
Run the actual linter commands found in the project:
|
||||
- Python: `ruff check .`, `flake8`, `pylint`
|
||||
- JavaScript/TypeScript: `npm run lint`, `eslint`
|
||||
|
||||
### Phase 2: Type Checking
|
||||
Run the actual type checker commands found:
|
||||
- TypeScript: `npx tsc --noEmit`
|
||||
- Python: `mypy src/`, `mypy .`
|
||||
|
||||
### Phase 3: Style Checking
|
||||
Run the actual formatter check commands found:
|
||||
- JavaScript/TypeScript: `npm run format:check`, `prettier --check`
|
||||
- Python: `black --check .`
|
||||
|
||||
### Phase 4: Unit Testing
|
||||
Run the actual test commands found:
|
||||
- Python: `pytest`, `pytest tests/unit`
|
||||
- JavaScript/TypeScript: `npm test`, `jest`
|
||||
|
||||
### Phase 5: End-to-End Testing (BE CREATIVE AND COMPREHENSIVE)
|
||||
|
||||
Test COMPLETE user workflows from documentation, not just internal APIs.
|
||||
|
||||
**The Three Levels of E2E Testing:**
|
||||
|
||||
1. **Internal APIs** (what you might naturally test):
|
||||
- Test adapter endpoints work
|
||||
- Database queries succeed
|
||||
- Commands execute
|
||||
|
||||
2. **External Integrations** (what you MUST test):
|
||||
- CLI operations (AWS CLI, gh, gcloud, etc.)
|
||||
- Platform APIs (LiveKit, Google Calendar, Gmail, Telegram)
|
||||
- Any external services the app depends on
|
||||
|
||||
3. **Complete User Journeys** (what gives 100% confidence):
|
||||
- Follow workflows from docs start-to-finish
|
||||
- **Backend Example:** "Voice agent receives request → processes calendar event → sends email → confirms to user"
|
||||
- **Frontend Example:** "User logs in → views dashboard → schedules event → receives confirmation"
|
||||
- Test like a user would actually use the application in production
|
||||
|
||||
**Examples of good vs. bad E2E tests:**
|
||||
|
||||
**Backend:**
|
||||
- ❌ Bad: Tests that Google Calendar adapter returns data
|
||||
- ✅ Good: Voice agent receives calendar request → fetches events → formats response → returns to user
|
||||
- ✅ Great: Voice interaction → Calendar query → Gmail notification → LiveKit response confirmation
|
||||
|
||||
**Frontend:**
|
||||
- ❌ Bad: Tests that login form submits data
|
||||
- ✅ Good: User submits login → Gets redirected → Dashboard loads with data
|
||||
- ✅ Great: Full auth flow → Create calendar event via UI → Verify in Google Calendar → See update in dashboard
|
||||
|
||||
**Approach:**
|
||||
- Use Docker for isolated, reproducible testing
|
||||
- Create test data/accounts as needed
|
||||
- Verify outcomes in external systems (Google Calendar, database, LiveKit rooms)
|
||||
- Clean up after tests
|
||||
- Use environment variables for test credentials
|
||||
|
||||
## Critical: Don't Stop Until Everything is Validated
|
||||
|
||||
**Your job is to create a validation command that leaves NO STONE UNTURNED.**
|
||||
|
||||
- Every user workflow from docs should be tested end-to-end
|
||||
- Every external integration should be exercised (LiveKit, Google services, AWS, etc.)
|
||||
- Every API endpoint should be hit
|
||||
- Every error case should be verified
|
||||
- Database integrity should be confirmed
|
||||
- The validation should be so thorough that manual testing is completely unnecessary
|
||||
|
||||
If `/validate` passes, the user should have 100% confidence their application works correctly in production. Don't settle for partial coverage - make it comprehensive, creative, and complete.
|
||||
|
||||
## Mygentic Personal Assistant Specific Workflows
|
||||
|
||||
**Backend workflows to test:**
|
||||
1. **Voice Agent Interaction:**
|
||||
- LiveKit room connection
|
||||
- Speech-to-text processing
|
||||
- Intent recognition
|
||||
- Response generation
|
||||
- Text-to-speech output
|
||||
|
||||
2. **Google Services Integration:**
|
||||
- OAuth authentication flow
|
||||
- Calendar event creation/retrieval
|
||||
- Gmail message sending
|
||||
- Multiple account support
|
||||
|
||||
3. **Telegram Bot:**
|
||||
- Message receiving
|
||||
- Command processing
|
||||
- Response sending
|
||||
|
||||
**Frontend workflows to test:**
|
||||
1. **Authentication:**
|
||||
- User registration
|
||||
- Email verification
|
||||
- Login/logout
|
||||
- Session management
|
||||
|
||||
2. **Dashboard:**
|
||||
- Calendar view
|
||||
- Event management
|
||||
- Settings configuration
|
||||
|
||||
3. **Voice Agent UI:**
|
||||
- Connection to LiveKit
|
||||
- Audio streaming
|
||||
- Real-time transcription display
|
||||
- Response visualization
|
||||
|
||||
## Output
|
||||
|
||||
Write the generated validation command to:
|
||||
- `backend/.claude/commands/validate.md` for backend
|
||||
- `frontend/.claude/commands/validate.md` for frontend
|
||||
|
||||
The command should be executable, practical, and give complete confidence in the codebase.
|
||||
|
||||
## Example Structure
|
||||
|
||||
See the example validation command for reference, but adapt it to the specific tools, frameworks, and workflows found in this codebase. Don't copy-paste - generate based on actual codebase analysis.
|
||||
Reference in New Issue
Block a user