commit 34a2423d78de64caeaf6c7db34c52a0de9430764 Author: Zhongwei Li Date: Sat Nov 29 18:02:42 2025 +0800 Initial commit diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json new file mode 100644 index 0000000..46baa47 --- /dev/null +++ b/.claude-plugin/plugin.json @@ -0,0 +1,17 @@ +{ + "name": "go-linter-driven-development", + "description": "Linter-driven development workflow for Go with six specialized skills: design, testing, refactoring, review, and documentation", + "version": "1.0.0", + "author": { + "name": "Dan Mordechay" + }, + "skills": [ + "./skills" + ], + "agents": [ + "./agents" + ], + "commands": [ + "./commands" + ] +} \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 0000000..58e1e65 --- /dev/null +++ b/README.md @@ -0,0 +1,3 @@ +# go-linter-driven-development + +Linter-driven development workflow for Go with six specialized skills: design, testing, refactoring, review, and documentation diff --git a/agents/go-code-reviewer.md b/agents/go-code-reviewer.md new file mode 100644 index 0000000..d0b5fe8 --- /dev/null +++ b/agents/go-code-reviewer.md @@ -0,0 +1,288 @@ +--- +name: go-code-reviewer +description: Reviews Go code for design debt, primitive obsession, mixed abstractions, and architectural issues. Returns structured report without making changes. Used by linter-driven-development orchestrator for parallel analysis. +--- + +You are a Go Code Design Reviewer specializing in detecting design patterns and architectural issues that linters cannot catch. You are invoked as a **read-only subagent** during the parallel analysis phase of the linter-driven development workflow. + +## Your Role + +**IMPORTANT: You are READ-ONLY. Do not make changes, invoke other skills, or provide fixes. Only analyze and report findings.** + +You will be provided: +- **Files to review**: List of .go files (changed since last commit) +- **Review mode**: `full` (first run) or `incremental` (subsequent runs after fixes) +- **Previous findings** (optional, for incremental mode) + +Your job: Analyze the code and return a **structured report** that the orchestrator can parse and combine with linter output. + +## Analysis Process + +### Step 1: Load Pre-Commit Review Skill + +Automatically use the @pre-commit-review skill to guide your analysis. This skill contains: +- Detection checklist for 8 design issue categories +- Juiciness scoring algorithm for primitive obsession +- Examples of good vs bad patterns +- Effort estimation guidelines + +### Step 2: Read and Analyze Files + +For each file in the review scope: +1. Use Read tool to examine code +2. Use Grep tool to find usage patterns across codebase +3. Apply design principles from @pre-commit-review skill + +### Step 3: Detect Design Issues + +Focus on issues **linters cannot detect**: + +**🐛 Bugs** (will cause runtime failures): +- Nil dereferences (returning nil without checking) +- Ignored errors (err != nil but not handled) +- Resource leaks (missing Close() calls) +- Race conditions (shared state without locks) + +**🔴 Design Debt** (will cause pain when extending): +- **Primitive obsession**: String/int used where custom type would add safety + - Apply juiciness scoring (see @pre-commit-review) + - Example: `func GetUser(id string)` → Should be `UserID` type +- **Non-self-validating types**: Validation in methods instead of constructor + - Example: Methods check `if u.Email == ""` → Should validate in `NewUser()` +- **Missing domain concepts**: Implicit types that should be explicit + - Example: Magic number 1024 appearing 5 times → Should be `const maxBufferSize` +- **Wrong architecture**: Horizontal slicing instead of vertical + - Example: `domain/user`, `services/user` → Should be `user/` package + +**🟡 Readability Debt** (makes code harder to understand): +- **Mixed abstraction levels** (not "storified"): + - Example: Function mixes high-level steps with low-level string manipulation + - Top-level functions should read like a story +- **Functions too long or complex**: + - Even if linter passes, flag if hard to understand +- **Poor naming**: Generic names like `data`, `process`, `handler` +- **Comment quality**: Explaining WHAT instead of WHY + +**🟢 Polish** (minor improvements): +- Non-idiomatic naming (e.g., `SaveUser` → `Save` when receiver provides context) +- Missing godoc examples +- Minor refactoring opportunities + +### Step 4: Generate Structured Report + +**CRITICAL: Output must follow exact format for orchestrator parsing.** + +## Output Format + +### For Full Review Mode + +``` +📊 CODE REVIEW REPORT +Scope: [list of files reviewed] +Mode: FULL + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +SUMMARY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Total findings: [N] +🐛 Bugs: [N] (fix immediately) +🔴 Design Debt: [N] (fix before commit) +🟡 Readability Debt: [N] (improves maintainability) +🟢 Polish: [N] (nice to have) + +Estimated total effort: [X.Y hours] + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +DETAILED FINDINGS +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +🐛 BUGS +──────────────────────────────────────────────── +[For each bug finding:] +pkg/file.go:123 | [Issue description] | [Why it matters] | [Fix strategy] | Effort: [Trivial/Moderate/Significant] + +🔴 DESIGN DEBT +──────────────────────────────────────────────── +[For each design debt finding:] +pkg/file.go:45 | [Issue description] | [Why it matters] | [Fix strategy] | Effort: [Trivial/Moderate/Significant] + +🟡 READABILITY DEBT +──────────────────────────────────────────────── +[For each readability finding:] +pkg/file.go:78 | [Issue description] | [Why it matters] | [Fix strategy] | Effort: [Trivial/Moderate/Significant] + +🟢 POLISH +──────────────────────────────────────────────── +[For each polish opportunity:] +pkg/file.go:34 | [Issue description] | [Why it matters] | [Fix strategy] | Effort: [Trivial/Moderate/Significant] +``` + +### For Incremental Review Mode + +``` +📊 CODE REVIEW DELTA REPORT +Scope: [changed files only] +Mode: INCREMENTAL +Previous run: [timestamp or iteration number] + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +SUMMARY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +✅ Fixed: [N] (resolved from previous run) +⚠️ Remaining: [N] (still need attention) +🆕 New: [N] (introduced by recent changes) + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +DELTA FINDINGS +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +✅ FIXED (from previous run) +──────────────────────────────────────────────── +pkg/file.go:45 | [What was fixed] | [How it was resolved] + +⚠️ REMAINING (still need attention) +──────────────────────────────────────────────── +pkg/file.go:78 | [Issue] | [Why still present] | [Fix strategy] | Effort: [X] + +🆕 NEW (introduced by recent changes) +──────────────────────────────────────────────── +pkg/file.go:123 | [New issue] | [Why it matters] | [Fix strategy] | Effort: [X] +``` + +## Format Requirements + +**file:line Format**: Must be exact for correlation with linter errors +- ✅ Correct: `pkg/parser.go:45` +- ❌ Wrong: `parser.go line 45`, `pkg/parser.go (line 45)`, `parser.go:45` + +**Effort Estimates**: +- **Trivial**: <5 minutes + - Examples: Extract constant, rename variable, fix comment +- **Moderate**: 5-20 minutes + - Examples: Extract function, storifying, create simple self-validating type +- **Significant**: >20 minutes + - Examples: Architectural refactoring, complex type extraction, package restructuring + +**Fix Strategy**: Be specific and actionable +- ✅ Good: "Apply STORIFYING: Extract parseRawInput(), validateFields(), buildResult() functions" +- ❌ Bad: "Refactor this function" + +## Incremental Mode Instructions + +When review mode is `incremental`: + +1. **Compare against previous findings**: + - Read previous report (provided in prompt) + - Check each previous issue against current code state + +2. **Categorize outcomes**: + - ✅ **Fixed**: Issue no longer present (code changed to resolve it) + - ⚠️ **Remaining**: Issue still exists in same location + - 🆕 **New**: Issue not in previous report, introduced by recent changes + +3. **Focus on changed files**: + - Only analyze files modified since last review + - Use `git diff` to identify changed sections + - Don't re-analyze unchanged files + +4. **Detect regressions**: + - Watch for new issues introduced by fixes + - Example: Fix complexity but introduce primitive obsession + +## Juiciness Scoring Algorithm + +For primitive obsession findings, calculate juiciness score (1-10): + +**Factors to consider**: +- **Validation complexity** (0-4 points): + - Trivial (empty check): 1 point + - Format check (regex, length): 2 points + - Business rules (range, state): 3 points + - Complex validation (multiple rules, cross-field): 4 points + +- **Usage frequency** (0-3 points): + - 1-2 places: 1 point + - 3-5 places: 2 points + - 6+ places: 3 points + +- **Methods needed** (0-3 points): + - Just constructor: 1 point + - Constructor + 1-2 methods: 2 points + - Constructor + 3+ methods: 3 points + +**Interpretation**: +- **1-3**: Not worth extracting (trivial validation, used once) +- **4-6**: Consider extracting (moderate complexity or usage) +- **7-10**: Definitely extract (complex validation, widely used) + +**Example**: +``` +UserID string validation: +- Validation: Non-empty + UUID format (3 points) +- Usage: 7 places in codebase (3 points) +- Methods: NewUserID(), String(), Equals() (2 points) += Juiciness: 8/10 → Extract UserID type +``` + +## Performance Targets + +- **Full review**: Complete within 30-45 seconds for typical feature (5-10 files) +- **Incremental review**: Complete within 15-20 seconds (2-3 changed files) +- **Parallel execution**: Your runtime should not block linter or tests + +## What You Must NOT Do + +❌ **Do NOT invoke other skills** (@refactoring, @code-designing, @testing) +❌ **Do NOT make code changes** (you are read-only) +❌ **Do NOT run linter** (orchestrator handles this) +❌ **Do NOT run tests** (orchestrator handles this) +❌ **Do NOT make decisions for user** (just report findings) +❌ **Do NOT iterate** (run once and return report) + +## Integration with Orchestrator + +You are invoked by the @linter-driven-development orchestrator during: + +**Phase 2: Parallel Quality Analysis** +- Runs simultaneously with tests and linter +- Receives list of changed .go files +- Returns structured report for combined analysis + +**Phase 4: Iterative Fix Loop** +- Re-invoked after each fix application +- Runs in `incremental` mode +- Only analyzes changed files +- Tracks fix progress (✅ Fixed | ⚠️ Remaining | 🆕 New) + +**Your report enables intelligent combined analysis**: +- Orchestrator merges your findings with linter errors +- Identifies overlapping issues (same file:line) +- Generates unified fix strategies +- Prioritizes by impact and effort + +## Example Invocation + +``` +Review these Go files: +- pkg/parser.go +- pkg/validator.go +- pkg/types.go + +Mode: full +``` + +That's it! The agent's own instructions handle everything else: +- Automatically loads @pre-commit-review skill +- Detects design issues in 8 categories +- Returns structured report with effort estimates +- Operates in read-only mode + +## Remember + +- You are a **reporter**, not a **fixer** +- Your output is **parsed by orchestrator**, format must be exact +- Your findings are **combined with linter errors** for smart analysis +- You enable **intelligent root cause analysis** and **unified fix strategies** +- You run **in parallel** with tests and linter for 40-50% speedup diff --git a/agents/quality-analyzer.md b/agents/quality-analyzer.md new file mode 100644 index 0000000..496c099 --- /dev/null +++ b/agents/quality-analyzer.md @@ -0,0 +1,451 @@ +--- +name: quality-analyzer +description: Executes parallel quality analysis (tests, linter, code review), normalizes outputs, identifies overlapping issues, and returns intelligent combined reports with root cause analysis. Supports both full and incremental modes. +--- + +You are a Quality Analyzer Agent that orchestrates parallel quality analysis for Go projects. You are invoked as a **read-only subagent** that runs quality gates in parallel, combines their results intelligently, and returns structured reports. + +## Your Role + +**IMPORTANT: You are READ-ONLY. Do not make changes, apply fixes, or invoke refactoring skills. Only analyze and report findings.** + +You will be provided: +- **Mode**: `full` (initial comprehensive analysis) or `incremental` (delta analysis after fixes) +- **Files to analyze**: List of .go files (typically changed files from git) +- **Project commands** (optional): Test and lint commands discovered by orchestrator +- **Previous findings** (optional, for incremental mode) + +Your job: Execute parallel quality analysis and return a **structured report** with intelligent combined findings. + +## Core Responsibilities + +1. **Execute parallel analysis**: Launch 3 tools simultaneously: + - Bash([PROJECT_TEST_COMMAND]) + - Bash([PROJECT_LINT_COMMAND]) + - Task(subagent_type: "go-code-reviewer") + +2. **Normalize outputs**: Parse different output formats into common structure + +3. **Find overlaps**: Identify issues at same file:line from multiple sources + +4. **Root cause analysis**: Use LLM reasoning to understand underlying problems + +5. **Generate reports**: Output full analysis or delta report based on mode + +## Workflow + +### Phase A: Pre-Flight Check (Command Discovery) + +**If `project_commands` parameter is provided:** +- ✅ Use them directly (fast path - orchestrator already discovered) +- Skip to Phase B + +**If `project_commands` parameter is NOT provided:** +- 🔍 Discover commands autonomously: + 1. Search project docs: `README.md`, `CLAUDE.md`, `Makefile`, `Taskfile.yaml` + 2. Extract test command (look for `go test`, `make test`, `task test`) + 3. Extract lint command (look for `golangci-lint run --fix`, `make lint`, `task lintwithfix`) + 4. Fallback to defaults: + - Tests: `go test ./...` + - Linter: `golangci-lint run --fix` + 5. Verify fallbacks work: + - Check: `which go` + - Check: `which golangci-lint` + 6. If fallbacks fail: + - Return **TOOLS_UNAVAILABLE** status with details + +### Phase B: Parallel Execution + +**Step 1: Launch all quality gates simultaneously** + +Execute in a **single message with 3 tool calls**: + +``` +Tool Call 1: Bash + command: [PROJECT_TEST_COMMAND] + description: "Run project tests" + +Tool Call 2: Bash + command: [PROJECT_LINT_COMMAND] + description: "Run linter with autofix" + +Tool Call 3: Task + subagent_type: "go-code-reviewer" + prompt: "Review these Go files: [FILES]\nMode: [full|incremental]\n[Previous findings if incremental]" +``` + +**Step 2: Wait for all results** +- Collect test output (pass/fail, coverage) +- Collect linter output (errors with file:line) +- Collect review report (structured findings by category) + +**Step 3: Check test results FIRST** +- If tests failed → Return **TEST_FAILURE** immediately (skip Phases C-E) +- If tests passed → Continue to Phase C + +### Phase C: Normalize Results + +**Note:** Only execute when tests PASS. Tests are binary (pass/fail) and not normalized as "issues". + +Convert linter and reviewer outputs to common format: + +```yaml +normalized_issue: + source: "linter" | "review" + file: "pkg/parser.go" + line: 45 + category: "complexity" | "style" | "design" | "bug" + severity: "critical" | "high" | "medium" | "low" + message: "Cognitive complexity 18 (>15)" + raw_output: "..." +``` + +### Phase D: Find Overlapping Issues + +Group issues by location (file:line): + +```yaml +overlapping_group: + location: "pkg/parser.go:45" + issues: + - source: linter, category: complexity, message: "Cognitive complexity 18" + - source: linter, category: length, message: "Function length 58 statements" + - source: review, category: design, message: "Mixed abstraction levels" + - source: review, category: design, message: "Defensive null checking" +``` + +### Phase E: Root Cause Analysis (LLM Reasoning) + +For each overlapping group: +1. List all issues at this location +2. Ask: "What's the underlying problem causing ALL these issues?" +3. Describe the pattern (without prescribing the fix) +4. Score: Impact (issues resolved) + complexity + +**Example analysis:** +``` +Location: pkg/parser.go:45 (4 issues) + +Issues: +- Linter: Cognitive complexity 18 (>15) +- Linter: Function length 58 statements (>50) +- Review: Mixed abstraction levels +- Review: Defensive null checking + +Root Cause Analysis: + Pattern: Function handles multiple responsibilities at different + abstraction levels (parsing, validation, building) + Impact: HIGH (4 issues at same location) + Complexity: MODERATE (function boundaries clear) + + This is a classic case where multiple concerns are intertwined. +``` + +**Important:** No fix suggestions - just the analysis. The orchestrator passes this to @refactoring skill. + +## Output Format + +### Status Types + +Return one of four status types: + +**TOOLS_UNAVAILABLE**: One or more required tools can't be found or executed +**TEST_FAILURE**: Tests ran but failed (test cases failed) +**ISSUES_FOUND**: Tests passed, tools ran, but linter/reviewer found quality issues +**CLEAN_STATE**: Tests passed, linter clean, reviewer clean - all quality gates green + +### Full Mode Output (Initial Analysis) + +``` +═══════════════════════════════════════════════════════ +QUALITY ANALYSIS REPORT +Mode: FULL +Files analyzed: [N] +═══════════════════════════════════════════════════════ + +📊 SUMMARY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Tests: ✅ PASS (coverage: 87%) | ❌ FAIL (3 failures) +Linter: ✅ PASS (0 errors) | ❌ FAIL (5 errors) +Review: ✅ CLEAN (0 findings) | ⚠️ FINDINGS (8 issues: 0 bugs, 3 design, 4 readability, 1 polish) + +Total issues: [N] from [sources] + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +OVERLAPPING ISSUES ANALYSIS +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Found [N] locations with overlapping issues: + +┌─────────────────────────────────────────────────────┐ +│ pkg/parser.go:45 - function Parse │ +│ OVERLAPPING (4 issues): │ +│ │ +│ ⚠️ Linter: Cognitive complexity 18 (>15) │ +│ ⚠️ Linter: Function length 58 statements (>50) │ +│ 🔴 Review: Mixed abstraction levels │ +│ 🔴 Review: Defensive null checking │ +│ │ +│ 🎯 ROOT CAUSE: │ +│ Function handles multiple responsibilities at │ +│ different abstraction levels (parsing, validation, │ +│ building result). │ +│ │ +│ Impact: HIGH (4 issues) | Complexity: MODERATE │ +│ Priority: #1 CRITICAL │ +└─────────────────────────────────────────────────────┘ + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +ISOLATED ISSUES (No overlaps) +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +pkg/types.go:12 | Linter | Naming: exported type should have comment +pkg/handler.go:89 | Review | Polish | Non-idiomatic naming + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +PRIORITIZED FIX ORDER +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Priority #1: pkg/parser.go:45 (4 issues, HIGH impact) +Priority #2: pkg/validator.go:23 (3 issues, HIGH impact) +Priority #3: pkg/handler.go:67 (2 issues, MEDIUM impact) + +Isolated issues: [N] (fix individually) + +Total fix targets: [N] overlapping groups + [N] isolated = [N] fixes + +STATUS: [TOOLS_UNAVAILABLE | TEST_FAILURE | ISSUES_FOUND | CLEAN_STATE] +``` + +### Incremental Mode Output (After Fixes) + +``` +═══════════════════════════════════════════════════════ +QUALITY ANALYSIS DELTA REPORT +Mode: INCREMENTAL +Files re-analyzed: [N] (changed since last run) +═══════════════════════════════════════════════════════ + +📊 SUMMARY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Tests: ✅ PASS (coverage: 89% ↑) | ❌ FAIL (1 failure) +Linter: ✅ PASS (0 errors) | ❌ FAIL (2 errors) +Review: ✅ CLEAN (0 findings) | ⚠️ FINDINGS (1 issue) + +✅ Fixed: [N] issues from [locations] +⚠️ Remaining: [N] issues +🆕 New: [N] issues introduced + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +RESOLUTION DETAILS +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +✅ FIXED: + pkg/parser.go:45 | Linter | Cognitive complexity (was 18, now 8) + pkg/parser.go:45 | Linter | Function length (was 58, now 25) + pkg/parser.go:45 | Review | Mixed abstraction levels (resolved) + pkg/parser.go:45 | Review | Defensive null checking (resolved) + +⚠️ REMAINING: + pkg/types.go:12 | Linter | Naming: exported type should have comment + +🆕 NEW: + pkg/validator.go:89 | Review | Primitive obsession with string ID + +STATUS: [TEST_FAILURE | ISSUES_FOUND | CLEAN_STATE] +``` + +### TOOLS_UNAVAILABLE Report + +```yaml +status: TOOLS_UNAVAILABLE +timestamp: "2025-11-11T10:30:00Z" +unavailable_tools: + - name: "test" + command: "go test ./..." + error: "command not found: go" + suggestion: "Install Go toolchain" + - name: "lint" + command: "golangci-lint run --fix" + error: "executable not found in PATH" + suggestion: "Install golangci-lint: https://golangci-lint.run/usage/install/" + +message: "Cannot proceed: 2 tools unavailable. Fix tool issues and re-run." +``` + +### TEST_FAILURE Report + +```yaml +status: TEST_FAILURE +timestamp: "2025-11-11T10:30:00Z" +tests: + total: 45 + passed: 42 + failed: 3 + coverage: 87% + failures: + - test: "TestParser_Parse" + file: "pkg/parser_test.go:25" + error: "Expected 'foo', got 'bar'" + - test: "TestValidator_Validate" + file: "pkg/validator_test.go:42" + error: "Validation failed: missing required field" + raw_output: "... full test output ..." + +message: "Tests failed. Fix failing tests before proceeding to quality analysis." +``` + +## Error Handling + +### Partial Failure Handling + +When tools execute but fail mid-execution, continue with available data: + +**Linter crashes:** +```yaml +status: ISSUES_FOUND +tests: {passed: true, ...} +linter: + status: "error" + error: "Failed to parse linter output: unexpected format" + raw_output: "..." +reviewer: {status: "success", findings: [...]} + +# Continue with just reviewer data +message: "Tests passed. Linter failed (parse error). Showing reviewer findings only." +``` + +**Reviewer fails:** +```yaml +status: ISSUES_FOUND +tests: {passed: true, ...} +linter: {status: "success", issues: [...]} +reviewer: + status: "error" + error: "Agent timeout after 300s" + +# Continue with just linter data +message: "Tests passed. Code review failed (timeout). Showing linter findings only." +``` + +**Key Principle:** As long as tests pass, return ISSUES_FOUND/CLEAN_STATE and provide whatever quality data is available. + +## File Parameter Usage + +The `files` parameter is interpreted differently by each tool: + +- **Tests**: IGNORES `files` parameter - always runs full test suite + - Reason: Catch regressions across entire codebase + - Command: `go test ./...` (all packages) + +- **Linter**: IGNORES `files` parameter - runs configured command as-is + - Reason: Linters need package/project scope, not file-level scope + - Linters typically already target changes via flags like `--new-from-rev` + - Command examples: + - `golangci-lint run --config .golangci.yaml --new-from-rev=origin/dev --fix ./...` + - `golangci-lint run --fix` (if configured to use git diff internally) + +- **Reviewer**: USES `files` parameter - reviews specific files only + - Reason: Focused review on new/changed code + - Passes file list to go-code-reviewer agent + +## Performance Targets + +- **Full analysis**: Complete within 60-90 seconds for typical feature (5-10 files) +- **Incremental analysis**: Complete within 30-45 seconds (2-3 changed files) +- **Parallel execution**: All 3 tools run simultaneously for maximum efficiency + +## What You Must NOT Do + +❌ **Do NOT apply fixes** (that's @refactoring skill's job) +❌ **Do NOT make decisions for user** (just report findings) +❌ **Do NOT do code review yourself** (delegate to go-code-reviewer agent) +❌ **Do NOT run iterative loops** (orchestrator handles that) +❌ **Do NOT invoke other skills** beyond go-code-reviewer agent +❌ **Do NOT make code changes** (you are read-only) + +## Integration with Orchestrator + +You are invoked by the @linter-driven-development orchestrator during: + +**Phase 2: Quality Analysis (Agent is the Gate)** +- Orchestrator calls: `Task(subagent_type: "quality-analyzer", mode: "full", ...)` +- You execute parallel analysis and return combined report +- Orchestrator routes based on your status: + - TEST_FAILURE → Enter Test Focus Mode + - CLEAN_STATE → Skip to Documentation Phase + - ISSUES_FOUND → Proceed to Quality Fix Loop + +**Phase 3: Iterative Quality Fix Loop** +- Orchestrator calls: `Task(subagent_type: "quality-analyzer", mode: "incremental", ...)` +- You verify fix progress with delta analysis +- Orchestrator uses delta report to: + - Continue to next fix (if progress made) + - Enter Test Focus Mode (if tests failed) + - Complete loop (if clean state achieved) + +## Example Invocation + +### Full Mode (Initial Analysis) + +``` +Analyze code quality for this Go project. + +Mode: full + +Project commands: +- Test: go test ./... -v -cover +- Lint: golangci-lint run --fix + +Files to analyze: +- pkg/parser.go +- pkg/validator.go +- pkg/types.go +- pkg/handler.go + +Run all quality gates in parallel and return combined analysis. +``` + +### Incremental Mode (After Fix Applied) + +``` +Re-analyze code quality after refactoring. + +Mode: incremental + +Project commands: +- Test: go test ./... -v -cover +- Lint: golangci-lint run --fix + +Files to analyze (changed): +- pkg/parser.go + +Previous findings: +{ + "overlapping_groups": [ + { + "location": "pkg/parser.go:45", + "issues": [ + {"source": "linter", "message": "Cognitive complexity 18"}, + {"source": "linter", "message": "Function length 58"}, + {"source": "review", "message": "Mixed abstractions"}, + {"source": "review", "message": "Defensive checking"} + ] + } + ], + "isolated_issues": [...] +} + +Run quality gates and return delta report (what changed). +``` + +## Remember + +- You are a **quality gate orchestrator**, not a **fixer** +- Your output is **parsed by orchestrator**, format must be exact +- Your findings enable **intelligent root cause analysis** and **unified fix strategies** +- You run all tools **in parallel** for maximum efficiency +- You return one of 4 statuses: **TOOLS_UNAVAILABLE** | **TEST_FAILURE** | **ISSUES_FOUND** | **CLEAN_STATE** +- Tests always take priority - return **TEST_FAILURE** immediately if tests fail diff --git a/commands/go-ldd-analyze.md b/commands/go-ldd-analyze.md new file mode 100644 index 0000000..b6fabc3 --- /dev/null +++ b/commands/go-ldd-analyze.md @@ -0,0 +1,133 @@ +--- +name: go-ldd-analyze +description: Run quality analysis only - invoke quality-analyzer agent and display combined report without auto-fixing +--- + +Run comprehensive quality analysis with intelligent combining of test results, linter findings, and code review feedback. + +**This command does NOT auto-fix anything** - it provides read-only analysis with overlapping issue detection and root cause analysis. + +Execute these steps: + +## Step 1: Discover Project Commands + +Search project documentation to find test and lint commands: + +1. **Read project files** in order of preference: + - `CLAUDE.md` (project-specific instructions) + - `README.md` (project documentation) + - `Makefile` (look for `test:` and `lint:` targets) + - `Taskfile.yaml` (look for `test:` and `lintwithfix:` tasks) + - `.golangci.yaml` (linter configuration) + +2. **Extract commands**: + - **Test command**: Look for patterns like: + - `go test ./... -v -cover` + - `make test` + - `task test` + - **Lint command**: Look for patterns like: + - `golangci-lint run --fix` + - `golangci-lint run --config .golangci.yaml --new-from-rev=origin/dev --fix ./...` + - `make lint` + - `task lintwithfix` + +3. **Fallback to defaults** if not found: + - Test: `go test ./...` + - Lint: `golangci-lint run --fix` + +## Step 2: Identify Changed Files + +Use git to find files that have changed: + +```bash +git diff --name-only --diff-filter=ACMR HEAD | grep '\.go$' +``` + +If no git repository or no changes, analyze all `.go` files in the project (excluding vendor/, testdata/). + +## Step 3: Invoke Quality Analyzer Agent + +Call the quality-analyzer agent with discovered commands and files: + +``` +Task(subagent_type: "quality-analyzer") + +Prompt: +"Analyze code quality for this Go project. + +Mode: full + +Project commands: +- Test: [discovered test command] +- Lint: [discovered lint command] + +Files to analyze: +[list of changed .go files, one per line] + +Run all quality gates in parallel and return combined analysis." +``` + +## Step 4: Display Report + +The agent will return a structured report with one of four statuses: + +**TOOLS_UNAVAILABLE**: Display the report and suggest installing missing tools +**TEST_FAILURE**: Display test failures and suggest fixing them before quality analysis +**ISSUES_FOUND**: Display combined report with overlapping issues analysis and prioritized fix order +**CLEAN_STATE**: Display success message - all quality gates passed + +## Report Format + +The agent returns: +- ✅/❌ **Tests**: Pass/fail status with coverage +- ✅/❌ **Linter**: Clean/errors count +- ✅/⚠️ **Review**: Clean/findings (bugs, design debt, readability debt, polish) +- 🎯 **Overlapping Issues**: Multiple issues at same file:line with root cause analysis +- 📋 **Isolated Issues**: Single issues that don't overlap +- 🔢 **Prioritized Fix Order**: Which issues to tackle first based on impact + +## Example Usage + +```bash +/go-ldd-analyze +``` + +This will: +1. Discover test and lint commands from your project docs +2. Find changed Go files from git +3. Run tests, linter, and code review in parallel +4. Display intelligent combined analysis with overlapping issue detection + +## Use Cases + +- ✅ Quick quality check before committing +- ✅ Understand what issues exist without making changes +- ✅ Get intelligent combined view of tests + linter + review findings +- ✅ See overlapping issues with root cause analysis +- ✅ Identify high-impact fixes (multiple issues at same location) + +## Comparison with Other Commands + +| Command | Purpose | Auto-Fix | Agent | +|---------|---------|----------|-------| +| `/go-ldd-autopilot` | Complete workflow (Phase 1-6) | ✅ Yes | No | +| `/go-ldd-quickfix` | Quality gates loop with auto-fix | ✅ Yes | No | +| `/go-ldd-review` | Final verification, no auto-fix | ❌ No | No | +| `/go-ldd-analyze` | Quality analysis with intelligent combining | ❌ No | ✅ Yes | +| `/go-ldd-status` | Show workflow status | N/A | No | + +## Key Benefits + +1. **Parallel Execution**: Runs tests, linter, and code review simultaneously +2. **Intelligent Combining**: Identifies overlapping issues at same file:line +3. **Root Cause Analysis**: Explains why multiple issues occur at same location +4. **Prioritized Fixes**: Suggests fix order based on impact (issues resolved) +5. **Read-Only**: No auto-fix, just analysis and reporting +6. **Autonomous**: Discovers commands automatically from project docs + +## Notes + +- This command is equivalent to running the quality-analyzer agent standalone +- For auto-fix capability, use `/go-ldd-quickfix` instead +- For final commit-ready verification, use `/go-ldd-review` instead +- For complete workflow with implementation, use `/go-ldd-autopilot` instead diff --git a/commands/go-ldd-autopilot.md b/commands/go-ldd-autopilot.md new file mode 100644 index 0000000..8b50bfa --- /dev/null +++ b/commands/go-ldd-autopilot.md @@ -0,0 +1,16 @@ +--- +name: go-ldd-autopilot +description: Start complete linter-driven autopilot workflow (Phase 1-6) +--- + +Invoke the @linter-driven-development skill to run the complete autopilot workflow from design through commit-ready. + +The skill will automatically: +1. Run Pre-Flight Check (detect intent, find commands, verify Go project) +2. Execute all 6 phases with 2 quality gates +3. Use parallel analysis (tests + linter + go-code-reviewer agent) +4. Generate intelligent combined reports +5. Auto-fix all issues iteratively +6. Prepare commit-ready summary + +This is the full workflow - use for implementing features or fixes from start to finish. diff --git a/commands/go-ldd-quickfix.md b/commands/go-ldd-quickfix.md new file mode 100644 index 0000000..4e9e82f --- /dev/null +++ b/commands/go-ldd-quickfix.md @@ -0,0 +1,31 @@ +--- +name: go-ldd-quickfix +description: Run quality gates loop until all green (tests+linter+review → fix → repeat) +--- + +Execute the quality gates loop for already-implemented code that needs cleanup. + +Run these phases from @linter-driven-development skill: + +**Phase 2**: Parallel Analysis +- Discover project test/lint commands +- Launch 3 tools simultaneously: tests, linter, go-code-reviewer agent +- Wait for all results + +**Phase 3**: Intelligent Combined Report +- Merge findings from linter + review +- Identify overlapping issues at same file:line +- Analyze root causes +- Generate unified fix strategies +- Prioritize: Impact × Effort × Risk + +**Phase 4**: Iterative Fix Loop +- Apply fixes using @refactoring skill (auto, no confirmation) +- Re-verify with parallel analysis (incremental review mode) +- Repeat until all green + +**Loop until**: +✅ Tests pass | ✅ Linter clean | ✅ Review clean + +Use this when code is already written but needs to pass quality gates. +Skip the implementation phase (Phase 1) and go straight to fixing issues. diff --git a/commands/go-ldd-review.md b/commands/go-ldd-review.md new file mode 100644 index 0000000..4f040fa --- /dev/null +++ b/commands/go-ldd-review.md @@ -0,0 +1,25 @@ +--- +name: go-ldd-review +description: Check if code is commit-ready (final verification, no auto-fix) +--- + +Run final verification checks **without** the auto-fix loop. + +Execute these steps: + +1. **Discover commands** from project docs (README, CLAUDE.md, Makefile, etc.) +2. **Run in parallel**: + - Tests: Bash([PROJECT_TEST_COMMAND]) + - Linter: Bash([PROJECT_LINT_COMMAND]) + - Review: Task(subagent_type: "go-code-reviewer") with mode: full +3. **Generate commit readiness report**: + - ✅/❌ Tests: [pass/fail] + coverage + - ✅/❌ Linter: [clean/errors] + - ✅/❌ Review: [clean/findings] + - 📝 Files in scope: [list with +/- lines] + - 💡 Suggested commit message + +**Does NOT auto-fix anything** - just reports current state. + +Use when you want to verify code is ready without making changes. +Equivalent to Phase 2 (Parallel Analysis) + Gate 2 (Final Verification) only. diff --git a/commands/go-ldd-status.md b/commands/go-ldd-status.md new file mode 100644 index 0000000..43df563 --- /dev/null +++ b/commands/go-ldd-status.md @@ -0,0 +1,24 @@ +--- +name: go-ldd-status +description: Show current workflow status and progress +--- + +Display current implementation status: + +📍 Current Context: + - Active plan: [file path or "conversation"] + - Current step: [step number] + - Phase: [design/testing/linting/refactoring/review/docs/commit] + +📊 Last Results: + Tests: [status + coverage] + Linter: [status + error count] + Review: [status + finding count] + +📝 Files Modified: + [list with +/- lines] + +🎯 Next Action: + [What happens next in the workflow] + +Perfect for: "where are we?", "what's the status?", "what's next?" diff --git a/plugin.lock.json b/plugin.lock.json new file mode 100644 index 0000000..d063538 --- /dev/null +++ b/plugin.lock.json @@ -0,0 +1,161 @@ +{ + "$schema": "internal://schemas/plugin.lock.v1.json", + "pluginId": "gh:buzzdan/ai-coding-rules:go-linter-driven-development", + "normalized": { + "repo": null, + "ref": "refs/tags/v20251128.0", + "commit": "4ff99ed5f38fde8710ce7c36158dc798d6f18e22", + "treeHash": "62e5783b92aceda28c42ea4fb4c791cc719fa685b896ded43098d5776182231f", + "generatedAt": "2025-11-28T10:14:26.671666Z", + "toolVersion": "publish_plugins.py@0.2.0" + }, + "origin": { + "remote": "git@github.com:zhongweili/42plugin-data.git", + "branch": "master", + "commit": "aa1497ed0949fd50e99e70d6324a29c5b34f9390", + "repoRoot": "/Users/zhongweili/projects/openmind/42plugin-data" + }, + "manifest": { + "name": "go-linter-driven-development", + "description": "Linter-driven development workflow for Go with six specialized skills: design, testing, refactoring, review, and documentation", + "version": "1.0.0" + }, + "content": { + "files": [ + { + "path": "README.md", + "sha256": "a9394d443ed29109e294be03d1e62d4b110827ff828317d03f2a4b484c75f2cc" + }, + { + "path": "agents/quality-analyzer.md", + "sha256": "6cc12adac4943d798883dccf27e215c6b4292b714f03832a79cbc48d26a6fb57" + }, + { + "path": "agents/go-code-reviewer.md", + "sha256": "14e43c33b15e1f3d812a87e2da5233bbdad66f98cfbc4ee31048e821d635300a" + }, + { + "path": ".claude-plugin/plugin.json", + "sha256": "366f84401b60e9ee58a6e40ef13bfb7fe9bb3eb748ea8769d67329b7821445b0" + }, + { + "path": "commands/go-ldd-status.md", + "sha256": "28980b4e98cf8448ccda05dbe164259ad7b7c95e61d6fb771bf80edf92e77d02" + }, + { + "path": "commands/go-ldd-quickfix.md", + "sha256": "3aa19f4c298c5d3cecdad74bf7f4a1d75ba39a86bd1a4d3d50b3b58c0c67ab4c" + }, + { + "path": "commands/go-ldd-autopilot.md", + "sha256": "c069ce1448ecbea615d395387e0ddc7a93b890df55cfaf8f713313ab8072548e" + }, + { + "path": "commands/go-ldd-analyze.md", + "sha256": "476d1e24e88af7673a190fbb11e42072401b9fd0078038c0f9113a50db003568" + }, + { + "path": "commands/go-ldd-review.md", + "sha256": "f6f03906ba38fbaf90973c02ed4ebd358a24563d27b78b7a51a09257945ee2ab" + }, + { + "path": "skills/documentation/reference.md", + "sha256": "f88555911d15c6368574a5f306097b551ab99a33b980a60652bb2b83997c6e51" + }, + { + "path": "skills/documentation/SKILL.md", + "sha256": "fcd7995fc561fd7728414933ce7e7f725fc2082e8563acb83036d2c6c8d3b38b" + }, + { + "path": "skills/pre-commit-review/examples.md", + "sha256": "b444972c54a9a56a71976c5b96593c09bd3f1813596e9ba5987a411f50f17127" + }, + { + "path": "skills/pre-commit-review/reference.md", + "sha256": "67509db971d575127a1f27b91354e2f5ff33a14d5a13e35ecd09d488086a9b45" + }, + { + "path": "skills/pre-commit-review/SKILL.md", + "sha256": "8fac6b27aeeb899aaa40dfce2671c96598b02fad1d77a75d6bbc841e438a8fb5" + }, + { + "path": "skills/testing/reference.md", + "sha256": "87af459765a54bc4c16efb1655ecb908eded65340be4c74b98beeacc90b971e1" + }, + { + "path": "skills/testing/SKILL.md", + "sha256": "21a6cb2c55a01b2e6ef673f6ced8ced6d2919774d2ccb14c61e40c6ceaae5b53" + }, + { + "path": "skills/testing/examples/jsonrpc-mock.md", + "sha256": "27de2720e3c1eeccdfef47819ef4944b546cb07339093299dc60203dbb10ec05" + }, + { + "path": "skills/testing/examples/grpc-bufconn.md", + "sha256": "d74b30371d9f9dc92d39069fbca1847986899604c8c4d9bc8a75f07b8fc049b3" + }, + { + "path": "skills/testing/examples/system-patterns.md", + "sha256": "5d932252305b3246a953ebdf57dbfeef31a467ed05e43a515c4e41040c8ee608" + }, + { + "path": "skills/testing/examples/integration-patterns.md", + "sha256": "32c9725b6bec7349666959b76545bdbb2d9487ad0ca226d8884cdc4cd203e757" + }, + { + "path": "skills/testing/examples/nats-in-memory.md", + "sha256": "5971af8fcbde23208a401c80f4fab27369979fed52bd9693b25b2e266a74e3c2" + }, + { + "path": "skills/testing/examples/victoria-metrics.md", + "sha256": "39166610f54c583dd422384b4f87d83882d200e9351d754795fa22477908d1ca" + }, + { + "path": "skills/testing/examples/test-organization.md", + "sha256": "94672c890b71b45b8ce97dace08a928f0c44750f6c9cc82cba75bd6ffab9bd45" + }, + { + "path": "skills/testing/examples/httptest-dsl.md", + "sha256": "676044a837e158254d066d351053088b38e5c940b781fba0c396cfee355d818e" + }, + { + "path": "skills/refactoring/examples.md", + "sha256": "d62ec9524458b87cbc43c5f7cb64c5a4e1c66da6011608d5f5f708cb89a71164" + }, + { + "path": "skills/refactoring/reference.md", + "sha256": "1a3ed692db86eac381e76868da03086746f474f9acc0f195dfc998d48640000a" + }, + { + "path": "skills/refactoring/code-design-ref.md", + "sha256": "3391ddc56eb8d214ee0945a232c176f809932c25362b4d540a6c48ad16fcb3fd" + }, + { + "path": "skills/refactoring/SKILL.md", + "sha256": "7347830ea645b1a4aa51901cfabf723e157518f8aa1d0b3b165d2a9bd039cd2a" + }, + { + "path": "skills/linter-driven-development/reference.md", + "sha256": "758de7374b698c5d4c10d3237e7ef857ad57c571069e7dd3d2af050ba534538e" + }, + { + "path": "skills/linter-driven-development/SKILL.md", + "sha256": "406503f4056473f6e9513332fd6b7ad020dce62bf956edc15c6b359b6f979aab" + }, + { + "path": "skills/code-designing/reference.md", + "sha256": "91c93ad6ce027f2498ed4a977d88a090765feee24c2d94f177e36c3404aa3217" + }, + { + "path": "skills/code-designing/SKILL.md", + "sha256": "f3924c651ca89d4aec41f6b8a6f4498345d19adb1f22197a8c5be1fb9bcdf069" + } + ], + "dirSha256": "62e5783b92aceda28c42ea4fb4c791cc719fa685b896ded43098d5776182231f" + }, + "security": { + "scannedAt": null, + "scannerVersion": null, + "flags": [] + } +} \ No newline at end of file diff --git a/skills/code-designing/SKILL.md b/skills/code-designing/SKILL.md new file mode 100644 index 0000000..e935476 --- /dev/null +++ b/skills/code-designing/SKILL.md @@ -0,0 +1,221 @@ +--- +name: code-designing +description: Domain type design and architectural planning for Go code. Use when planning new features, designing self-validating types, preventing primitive obsession, or when refactoring reveals need for new types. Focuses on vertical slice architecture and type safety. +--- + +# Code Designing + +Domain type design and architectural planning for Go code. +Use when planning new features or identifying need for new types during refactoring. + +## When to Use +- Planning a new feature (before writing code) +- Refactoring reveals need for new types (complexity extraction) +- Linter failures suggest types should be introduced +- When you need to think through domain modeling + +## Purpose +Design clean, self-validating types that: +- Prevent primitive obsession +- Ensure type safety +- Make validation explicit +- Follow vertical slice architecture + +## Workflow + +### 0. Architecture Pattern Analysis (FIRST STEP) + +**Default: Always use vertical slice architecture** (feature-first, not layer-first). + +Scan codebase structure: +- **Vertical slicing**: `internal/feature/{handler,service,repository,models}.go` ✅ +- **Horizontal layering**: `internal/{handlers,services,domain}/feature.go` ⚠️ + +**Decision Flow**: +1. **Pure vertical** → Continue pattern, implement as `internal/[new-feature]/` +2. **Pure horizontal** → Propose: Start migration with `docs/architecture/vertical-slice-migration.md`, implement new feature as first vertical slice +3. **Mixed (migrating)** → Check for migration docs, continue pattern as vertical slice + +**Always ask user approval with options:** +- Option A: Vertical slice (recommended for cohesion/maintainability) +- Option B: Match existing pattern (if time-constrained) +- Acknowledge: Time pressure, team decisions, consistency needs are valid + +**If migration needed**, create/update `docs/architecture/vertical-slice-migration.md`: +```markdown +# Vertical Slice Migration Plan +## Current State: [horizontal/mixed] +## Target: Vertical slices in internal/[feature]/ +## Strategy: New features vertical, migrate existing incrementally +## Progress: [x] [new-feature] (this PR), [ ] existing features +``` + +See reference.md section #3 for detailed patterns. + +--- + +### 1. Understand Domain +- What is the problem domain? +- What are the main concepts/entities? +- What are the invariants and rules? +- How does this fit into existing architecture? + +### 2. Identify Core Types +Ask for each concept: +- Is this currently a primitive (string, int, float)? +- Does it have validation rules? +- Does it have behavior beyond simple data? +- Is it used across multiple places? + +If yes to any → Consider creating a type + +### 3. Design Self-Validating Types +For each type: +```go +// Type definition +type TypeName underlyingType + +// Validating constructor +func NewTypeName(input underlyingType) (TypeName, error) { + // Validate input + if /* validation fails */ { + return zero, errors.New("why it failed") + } + return TypeName(input), nil +} + +// Methods on type (if behavior needed) +func (t TypeName) SomeMethod() result { + // Type-specific logic +} +``` + +### 4. Plan Package Structure +- **Vertical slices**: Group by feature, not layer +- Each feature gets its own package +- Within package: separate by role (service, repository, handler) + +Good structure: +``` +user/ +├── user.go # Domain types +├── service.go # Business logic +├── repository.go # Persistence +└── handler.go # HTTP/API +``` + +Bad structure: +``` +domain/user.go +services/user_service.go +repository/user_repository.go +``` + +### 5. Design Orchestrating Types +For types that coordinate others: +- Make fields private +- Validate dependencies in constructor +- No nil checks in methods (constructor guarantees validity) + +```go +type Service struct { + repo Repository // private + notifier Notifier // private +} + +func NewService(repo Repository, notifier Notifier) (*Service, error) { + if repo == nil { + return nil, errors.New("repo required") + } + if notifier == nil { + return nil, errors.New("notifier required") + } + return &Service{ + repo: repo, + notifier: notifier, + }, nil +} + +// Methods can trust fields are valid +func (s *Service) DoSomething() error { + // No nil checks needed + return s.repo.Save(...) +} +``` + +### 6. Review Against Principles +Check design against (see reference.md): +- [ ] No primitive obsession +- [ ] Types are self-validating +- [ ] Vertical slice architecture +- [ ] Types designed around intent, not just shape +- [ ] Clear separation of concerns + +## Output Format + +After design phase: + +``` +🎨 DESIGN PLAN + +Feature: [Feature Name] + +Core Domain Types: +✅ UserID (string) - Self-validating, prevents empty IDs +✅ Email (string) - Self-validating, RFC 5322 validation +✅ Age (int) - Self-validating, range 0-150 + +Orchestrating Types: +✅ UserService - Coordinates user operations + Dependencies: Repository, Notifier + Methods: CreateUser, GetUser, UpdateUser + +Package Structure: +📁 user/ + ├── user.go # UserID, Email, Age, User + ├── service.go # UserService + ├── repository.go # Repository interface + implementations + ├── notifier.go # Notifier interface + implementations + └── handler.go # HTTP handlers + +Design Decisions: +- UserID is custom type to prevent passing empty/invalid IDs +- Email validation centralized in NewEmail constructor +- Vertical slice keeps all user logic in one package +- Repository as interface allows multiple backends (Postgres, in-memory for tests) + +Integration Points: +- Consumed by: HTTP API (/users endpoints) +- Depends on: Database, Email service +- Events: UserCreated event published after creation + +Next Steps: +1. Create types with validating constructors +2. Write unit tests for each type +3. Implement UserService +4. Write integration tests + +Ready to implement? Use @testing skill for test structure. +``` + +## Key Principles + +See reference.md for detailed principles: +- Primitive obsession prevention (Yoke design strategy) +- Self-validating types +- Vertical slice architecture +- Types around intent and behavior, not just shape +- Single responsibility per type + +## Pre-Code Review Questions + +Before writing code, ask: +- Can logic be moved into smaller custom types? +- Is this type designed around intent and behavior? +- Have I avoided primitive obsession? +- Is validation in the right place (constructor)? +- Does this follow vertical slice architecture? + +Only after satisfactory answers, proceed to implementation. + +See reference.md for complete design principles and examples. diff --git a/skills/code-designing/reference.md b/skills/code-designing/reference.md new file mode 100644 index 0000000..6c6b81d --- /dev/null +++ b/skills/code-designing/reference.md @@ -0,0 +1,512 @@ +# Code Design Principles + +Core principles for designing Go types and architecture. + +## 1. Primitive Obsession Prevention (Yoke Design Strategy) + +### Principle +When it makes sense, avoid using primitives directly. Instead, create a type with proper named methods. + +### When to Create a Type +A primitive should become a type when: +- It has validation rules +- It has behavior/logic associated with it +- It represents a domain concept +- It's used across multiple places +- Passing an invalid value would be a bug + +### Pattern: Self-Validating Type +```go +// ❌ Primitive obsession +func CreateUser(id string, email string, port int) error { + if id == "" { + return errors.New("id required") + } + if !isValidEmail(email) { + return errors.New("invalid email") + } + if port <= 0 || port >= 9000 { + return errors.New("invalid port") + } + // ... +} + +// ✅ Self-validating types +type UserID string +type Email string +type Port int + +func NewUserID(s string) (UserID, error) { + if s == "" { + return "", errors.New("id required") + } + return UserID(s), nil +} + +func NewEmail(s string) (Email, error) { + if !isValidEmail(s) { + return "", errors.New("invalid email") + } + return Email(s), nil +} + +func NewPort(i int) (Port, error) { + if i <= 0 || i >= 9000 { + return 0, errors.New("port must be between 1 and 8999") + } + return Port(i), nil +} + +func CreateUser(id UserID, email Email, port Port) error { + // No validation needed - types guarantee validity + // Pure business logic +} +``` + +### Benefits +- **Compile-time safety**: Can't pass wrong type +- **Centralized validation**: Rules in one place (constructor) +- **Self-documenting**: Type name explains purpose +- **Easier refactoring**: Change validation in one place + +### Examples from Real Code +```go +// Parser complexity split into roles +type HeaderParser struct { /* ... */ } +type PathParser struct { /* ... */ } +type BodyParser struct { /* ... */ } + +// Instead of one complex Parser with all logic +``` + +--- + +## 2. Self-Validating Types + +### Principle +Types should validate their invariants in constructors. Methods should trust that the object is valid. + +### Pattern: Private Fields + Validating Constructor +```go +// ❌ Non-self-validating +type UserService struct { + Repo Repository // Public, might be nil +} + +func (s *UserService) CreateUser(user User) error { + if s.Repo == nil { // Defensive check in every method + return errors.New("repo is nil") + } + return s.Repo.Save(user) +} + +// ✅ Self-validating +type UserService struct { + repo Repository // Private +} + +func NewUserService(repo Repository) (*UserService, error) { + if repo == nil { + return nil, errors.New("repo is required") + } + return &UserService{repo: repo}, nil +} + +func (s *UserService) CreateUser(user User) error { + // No nil check needed - constructor guarantees validity + return s.repo.Save(user) +} +``` + +### Nil is Not a Valid Value +- Never return nil values (except errors: `nil, err` or `val, nil` is okay) +- Never pass nil into a function +- Check arguments in constructor, not in methods + +### Avoid Defensive Coding +- Don't check for nil fields inside methods +- Constructor should guarantee object validity +- Methods can trust object state + +--- + +## 3. Vertical Slice Architecture + +### Principle +**Group by feature and behavior, not by technical layer.** + +All code for a feature lives together in one directory. + +### Examples + +#### ❌ BAD: Horizontal Layers +``` +internal/ +├── handlers/health_handler.go +├── services/health_service.go +└── models/health.go +``` +Problems: Feature scattered, hard to understand complete behavior, team conflicts + +#### ✅ GOOD: Vertical Slice +``` +internal/health/ +├── handler.go +├── service.go +├── repository.go +└── models.go +``` +Benefits: Feature colocated, easy to understand/extract, parallel work + +### Migration Strategy + +**New features**: Always implement as vertical slices +**Existing horizontal code**: Refactor incrementally + +Create `docs/architecture/vertical-slice-migration.md`: +```markdown +# Vertical Slice Migration Plan +## Current State: [horizontal/mixed description] +## Target: Vertical slices in internal/[feature]/ +## Strategy: New features vertical, refactor existing incrementally +## Progress: [x] new_feature (this PR), [ ] health, [ ] verification +``` + +**Never mix**: Don't have both `health/service.go` AND `services/health_service.go` for same feature. + +--- + +## 4. Types Around Intent and Behavior + +### Principle +Design types around intent and behavior, not just shape. + +### Ask Before Creating a Type +- What is the purpose of this type? +- What invariants must it maintain? +- What behavior does it have? +- Why does it exist (beyond grouping fields)? + +### Pattern: Types with Behavior +```go +// ❌ Type is just data container +type Config struct { + Host string + Port int +} + +// ✅ Type has behavior and validation +type ServerAddress struct { + host string + port int +} + +func NewServerAddress(host string, port int) (ServerAddress, error) { + if host == "" { + return ServerAddress{}, errors.New("host required") + } + if port <= 0 || port > 65535 { + return ServerAddress{}, errors.New("invalid port") + } + return ServerAddress{host: host, port: port}, nil +} + +func (a ServerAddress) String() string { + return fmt.Sprintf("%s:%d", a.host, a.port) +} + +func (a ServerAddress) IsLocal() bool { + return a.host == "localhost" || a.host == "127.0.0.1" +} +``` + +--- + +## 5. Type File Organization + +### Principle +Types with logic should be in their own file. Name the file after the type. + +### Pattern +``` +user/ +├── user.go # User type +├── user_id.go # UserID type with validation +├── email.go # Email type with validation +├── service.go # UserService +└── repository.go # Repository interface +``` + +### When to Extract to Own File +- Type has multiple methods +- Type has complex validation +- Type has significant documentation +- Type is important enough to be easily found + +--- + +## 6. Leaf vs Orchestrating Types + +### Leaf Types +**Definition**: Types not dependent on other custom types + +**Characteristics:** +- Self-contained +- Minimal dependencies +- Pure logic +- Easy to test + +**Example:** +```go +type UserID string +type Email string +type Age int + +// These are leaf types - they depend only on primitives +``` + +**Testing:** +- Should have 100% unit test coverage +- Test only public API +- Use table-driven tests + +### Orchestrating Types +**Definition**: Types that coordinate other types + +**Characteristics:** +- Depend on other types (composition) +- Implement business workflows +- Minimal logic (mostly delegation) + +**Example:** +```go +type UserService struct { + repo Repository + notifier Notifier +} + +// This orchestrates Repository and Notifier +``` + +**Testing:** +- Integration tests covering seams +- Test with real implementations, not mocks +- Can overlap with leaf type coverage + +### Design Goal +**Most logic should be in leaf types.** +- Leaf types are easy to test and maintain +- Orchestrators should be thin wrappers + +--- + +## 7. Abstraction Through Interfaces + +### Principle +Don't create interfaces until you need them (avoid interface pollution). + +### When to Create an Interface +- You have multiple implementations +- You need to inject dependency for testing +- You're defining a clear contract + +### Pattern: Interface at Usage Point +```go +// In user/service.go +type Repository interface { // Defined where used + Save(ctx context.Context, u User) error + Get(ctx context.Context, id UserID) (*User, error) +} + +type UserService struct { + repo Repository // Depends on interface +} + +// In user/postgres.go +type PostgresRepository struct { + db *sql.DB +} + +func (r *PostgresRepository) Save(ctx context.Context, u User) error { + // Implementation +} + +// In user/inmem.go +type InMemoryRepository struct { + users map[UserID]User +} + +func (r *InMemoryRepository) Save(ctx context.Context, u User) error { + // Implementation +} +``` + +### Benefits +- Easy to test (use in-memory implementation) +- Can swap implementations +- Clear contract + +### Don't Over-Abstract +```go +// ❌ Interface pollution +type UserGetter interface { + Get(id UserID) (*User, error) +} + +type UserSaver interface { + Save(u User) error +} + +type UserDeleter interface { + Delete(id UserID) error +} + +// ✅ Single cohesive interface +type Repository interface { + Get(ctx context.Context, id UserID) (*User, error) + Save(ctx context.Context, u User) error + Delete(ctx context.Context, id UserID) error +} +``` + +--- + +## 8. Design Checklist (Pre-Code Review) + +Before writing code, review: + +### Can Logic Move to Smaller Types? +- [ ] Are there primitives that should be types? +- [ ] Can complex logic be split into multiple types? +- [ ] Example: Parser → HeaderParser + PathParser + +### Type Intent +- [ ] Is type designed around behavior, not just shape? +- [ ] Does type have clear responsibility? +- [ ] Why does this type exist? + +### Validation +- [ ] Is validation in constructor? +- [ ] Are fields private? +- [ ] Can methods trust object validity? + +### Architecture +- [ ] Is this a vertical slice (not horizontal layer)? +- [ ] Are related types in same package? +- [ ] Is package name specific (not generic)? + +### Dependencies +- [ ] Are dependencies injected through constructor? +- [ ] Are dependencies interfaces (if needed)? +- [ ] Is constructor validating dependencies? + +Only after satisfactory answers → proceed to write code. + +--- + +## 9. Common Go Anti-Patterns to Avoid + +### Goroutine Leaks +Always ensure goroutines can exit: +```go +// ❌ Goroutine leak +func StartWorker() { + go func() { + for { + // No way to exit + work() + } + }() +} + +// ✅ Goroutine with exit +func StartWorker(ctx context.Context) { + go func() { + for { + select { + case <-ctx.Done(): + return + default: + work() + } + } + }() +} +``` + +### Interface Pollution +Don't create interfaces until you need them. + +### Premature Optimization +Measure before optimizing. + +### Ignoring Context +Always respect context cancellation: +```go +func DoWork(ctx context.Context) error { + // Check context + if err := ctx.Err(); err != nil { + return err + } + // ... +} +``` + +### Mutex in Wrong Scope +Keep mutex close to data it protects: +```go +// ✅ Mutex with data +type SafeCounter struct { + mu sync.Mutex + count int +} + +func (c *SafeCounter) Inc() { + c.mu.Lock() + defer c.mu.Unlock() + c.count++ +} +``` + +--- + +## 10. Naming Conventions + +### Package Names +- Use flatcase: `wekatrace`, not `wekaTrace` or `weka_trace` +- Avoid generic names: `util`, `common`, `helper` +- Avoid stdlib collisions: `metrics` collides with libs, use `wekametrics` + +### Type Names +- Ergonomic: `version.Info` better than `version.VersionInfo` +- Context from package: `user.Service` better than `user.UserService` +- Avoid redundancy: method receiver provides context + +### Method Names +```go +// ❌ Redundant +func (s *UserService) CreateUserAccount(u User) error + +// ✅ Concise +func (s *UserService) Create(u User) error +``` + +### Idiomatic Go +- Write idiomatic Go code +- Follow Go community style and best practices +- Use effective Go guidelines + +--- + +## Summary: Design Principles + +1. **Prevent primitive obsession** - Create types for domain concepts +2. **Self-validating types** - Validate in constructor, trust in methods +3. **Vertical slices** - Group by feature, not layer +4. **Intent and behavior** - Design types around purpose +5. **File per type** - Types with logic get own file +6. **Leaf types** - Most logic in self-contained types +7. **Interfaces when needed** - Don't over-abstract +8. **Pre-code review** - Think before coding +9. **Avoid anti-patterns** - Goroutine leaks, premature optimization, etc. +10. **Idiomatic naming** - Follow Go conventions diff --git a/skills/documentation/SKILL.md b/skills/documentation/SKILL.md new file mode 100644 index 0000000..ba6659c --- /dev/null +++ b/skills/documentation/SKILL.md @@ -0,0 +1,246 @@ +--- +name: documentation +description: Creates comprehensive feature documentation for humans and AI to understand features, resolve bugs, and extend functionality. Use after complete feature implementation (may span multiple commits). Generates feature docs, updates godoc, and creates testable examples. +--- + +# Feature Documentation + +Creates comprehensive feature documentation for humans and AI to use for +future bug resolution, feature extensions, and codebase understanding. + +## When to Use +- After a complete feature is implemented (may span multiple commits) +- When adding significant new functionality to the codebase +- NOT for: individual commits, bug fixes, minor refactors + +## Purpose +Generate documentation that helps: +- **Humans**: Understand what the feature does and how to use it +- **AI**: Context for future bug fixes and feature extensions +- **Team**: Onboarding and knowledge sharing + +This is NOT a changelog - it's an introduction to the feature. + +## Workflow + +### 1. Understand Feature Scope +- Review all commits related to the feature +- Identify all modified/new files +- Understand the problem being solved +- Map out integration points with existing system + +### 2. Analyze Architecture +- Identify core domain types +- Map data/control flow +- Document design decisions (WHY choices were made) +- Note patterns used (vertical slice, self-validating types, etc.) + +### 3. Generate Documentation Artifacts + +**Primary: Feature Documentation** (`docs/[feature-name].md`) +- Problem & solution: What problem does this solve? +- Architecture: How does it work? +- Usage examples: How do I use it? +- Integration: How does it fit into the system? + +**Secondary: Code Comments** +- Update package godoc to reflect feature's role +- Add godoc to key types explaining their purpose +- Create testable examples (Example_* functions) when helpful + +### 4. Validate Documentation +- Can someone unfamiliar understand the feature? +- Can AI use this for bug fixes without reading all code? +- Are design decisions clearly explained? +- Are integration points documented? + +## Documentation Template + +```markdown +# [Feature Name] + +## Problem & Solution +**Problem**: [What user/system problem does this solve?] + +**Solution**: [High-level approach taken] + +## Architecture + +### Core Types +- `TypeName` - [Purpose, why it exists, key responsibility] +- `AnotherType` - [Purpose, why it exists, key responsibility] + +### Design Decisions +- **Why [Decision]**: [Rationale - connects to coding principles] + - Example: "UserID is a custom type (not string) to avoid primitive obsession and ensure validation" +- **Why [Pattern]**: [Rationale] + - Example: "Vertical slice structure groups all user logic together for easier maintenance" + +### Data Flow +``` +[Step-by-step flow diagram or description] +Input → Validation → Processing → Storage → Output +``` + +### Integration Points +- **Consumed by**: [What uses this feature] +- **Depends on**: [What this feature uses] +- **Events/Hooks**: [If applicable] + +## Usage + +### Basic Usage +```go +// Common case example +package main + +func example() { + // Real, runnable code +} +``` + +### Advanced Scenarios +```go +// Complex case example +package main + +func advancedExample() { + // Real, runnable code showing edge cases +} +``` + +## Testing Strategy +- **Unit Tests**: [What's covered, approach] +- **Integration Tests**: [What's covered, approach] +- **Coverage**: [Percentage and rationale] + +## Future Considerations +- [Known limitations] +- [Potential extensions] +- [Related features that might be built on this] + +## References +- [Related packages] +- [External documentation] +- [Design patterns used] +``` + +## Code Comment Guidelines + +**Package-Level Documentation:** +```go +// Package [name] provides [high-level purpose]. +// +// [2-3 sentences about what problem this solves and how] +// +// Core types: +// - Type1: [Purpose] +// - Type2: [Purpose] +// +// Example usage: +// [Simple example showing typical usage] +// +// Design notes: +// - [Key design decision] +// - [Why certain patterns were used] +package name +``` + +**Type-Level Documentation:** +```go +// TypeName represents [domain concept]. +// +// [Explain why this type exists - design decision] +// [Explain validation rules if self-validating] +// [Explain thread-safety if relevant] +// +// Example: +// id, err := NewUserID("usr_123") +// if err != nil { +// // handle validation error +// } +type TypeName struct { + // ... +} +``` + +**Testable Examples:** +```go +// Example_TypeName_Usage demonstrates typical usage of TypeName. +func Example_TypeName_Usage() { + id, _ := NewUserID("usr_123") + fmt.Println(id) + // Output: usr_123 +} + +// Example_TypeName_Validation shows validation behavior. +func Example_TypeName_Validation() { + _, err := NewUserID("") + fmt.Println(err != nil) + // Output: true +} +``` + +## Output Format + +After generating documentation: + +``` +📚 FEATURE DOCUMENTATION COMPLETE + +Feature: [Feature Name] + +Generated Artifacts: +✅ docs/[feature-name].md (created) +✅ Package godoc updated in [package]/[file].go +✅ Type documentation for: + - TypeName1 ([file]:line) + - TypeName2 ([file]:line) +✅ Testable examples: + - Example_TypeName1_Usage + - Example_TypeName2_Validation + +Documentation covers: +- Problem & Solution overview +- Architecture with design decisions +- Core types: [list] +- Data flow diagram +- Integration points: [list] +- Usage examples (basic + advanced) +- Testing strategy +- Future considerations + +📝 Next Steps: +1. Review docs/[feature-name].md for accuracy +2. Run `go test` to verify testable examples execute correctly +3. Consider: Does this help future you/AI understand the feature? + +Would you like to: +1. Commit documentation as-is +2. Refine specific sections +3. Add more examples +4. Add testable examples to code +``` + +## Key Principles + +**Documentation is NOT:** +- A changelog of commits +- Implementation details without context +- API reference without explanation +- Generated automatically without understanding + +**Documentation IS:** +- Explaining WHY decisions were made +- Providing context for future changes +- Showing how pieces fit together +- Helping both humans and AI understand intent + +**AI-Friendly Documentation:** +When AI tools read this documentation for bug fixes or extensions: +- They should understand the problem domain +- They should know which types are central +- They should understand design constraints +- They should see how it integrates with the system + +See reference.md for complete documentation checklist and examples. diff --git a/skills/documentation/reference.md b/skills/documentation/reference.md new file mode 100644 index 0000000..7a0417b --- /dev/null +++ b/skills/documentation/reference.md @@ -0,0 +1,346 @@ +# Feature Documentation Reference + +Complete guide for generating feature documentation that serves both humans and AI. + +## Documentation Quality Checklist + +### Problem & Solution Section +- [ ] Clear problem statement (what user/system pain point?) +- [ ] High-level solution approach +- [ ] Why this solution was chosen over alternatives + +### Architecture Section +- [ ] All core types listed with their purpose +- [ ] Design decisions explained with rationale +- [ ] Connections to coding principles (primitive obsession prevention, vertical slice, etc.) +- [ ] Data flow diagram or clear description +- [ ] Integration points with existing system documented + +### Usage Section +- [ ] Basic usage example with runnable code +- [ ] Advanced usage example showing edge cases +- [ ] Common patterns demonstrated +- [ ] Examples are copy-pasteable + +### Testing Section +- [ ] Unit test approach explained +- [ ] Integration test approach explained +- [ ] Coverage metrics and rationale provided + +### Future Considerations +- [ ] Known limitations documented +- [ ] Potential extensions noted +- [ ] Related features that could build on this mentioned + +## Code Comments Checklist + +### Package Documentation +- [ ] High-level purpose explained +- [ ] Problem domain described +- [ ] Core types listed with brief descriptions +- [ ] Simple usage example included +- [ ] Key design decisions noted + +### Type Documentation +- [ ] Domain concept the type represents +- [ ] Why the type exists (design decision) +- [ ] Validation rules if self-validating +- [ ] Thread-safety guarantees if applicable +- [ ] Usage example for non-trivial types + +### Testable Examples (Example_* functions) +- [ ] At least one Example_* for complex/core types +- [ ] Examples are runnable (not pseudocode) +- [ ] Examples show common use cases +- [ ] Output comments included for verification +- [ ] Examples should be in test files + +## AI-Friendly Documentation Patterns + +### For Bug Fixes +AI needs to know: +- What was the original problem this feature solved? +- What are the core invariants that must be maintained? +- What assumptions were made? +- What are the integration points? + +**Example:** +```markdown +## Design Invariants +- UserID must always be non-empty after construction +- Email validation follows RFC 5322 +- UserService assumes repository is never nil (validated in constructor) +- Password hashes use bcrypt with cost factor 12 +``` + +### For Feature Extensions +AI needs to know: +- What patterns are established? +- Where are the natural extension points? +- What constraints must be maintained? + +**Example:** +```markdown +## Extension Points +- **New validation rules**: Add to NewUserID constructor +- **New storage backends**: Implement Repository interface +- **New notification channels**: Add implementation of Notifier interface +- **New authentication methods**: Implement Authenticator interface +``` + +### For Understanding Data Flow +AI needs to see: +- Entry points (how is this triggered?) +- Key transformation steps (what happens in sequence?) +- Exit points (what are the possible outcomes?) + +**Example:** +```markdown +## Data Flow +1. HTTP handler receives POST /users → CreateUserRequest +2. Request validation → NewUserID, NewEmail (self-validating types) +3. UserService.CreateUser → validates business rules +4. Repository.Save → persists to database +5. Notifier.SendWelcome → sends welcome email (async) +6. Returns: User struct or validation/business error +``` + +## Documentation Anti-Patterns + +### ❌ Implementation Details Without Context +```markdown +## Implementation +The CreateUser function calls validateEmail and then repo.Save. +It returns an error if validation fails. +``` +*Why bad?*: Describes WHAT code does without WHY + +### ✅ Context-Rich Explanation +```markdown +## Design Decision: Validation Before Persistence +CreateUser validates email format before database operations to: +1. Fail fast - avoid unnecessary database round-trips +2. Provide clear error messages - users get immediate feedback +3. Maintain data quality - only valid emails in database + +Email validation is separate from UserID validation because emails +may need external verification (MX record checks) in the future, +while UserIDs are purely format-based. +``` + +### ❌ Feature List Without Purpose +```markdown +## Components +- UserID type +- Email type +- UserService +- Repository interface +- Notifier interface +``` +*Why bad?*: No explanation of relationships or rationale + +### ✅ Purpose-Driven Structure +```markdown +## Architecture + +### Type Safety Layer (Primitive Obsession Prevention) +- **UserID**: Self-validating identifier (prevents empty/malformed IDs) +- **Email**: Self-validating email (prevents invalid formats, RFC 5322) + +These types ensure validation happens once at construction, not repeatedly +throughout the codebase. + +### Business Logic Layer +- **UserService**: Orchestrates user operations (creation, authentication) + - Depends on Repository for persistence + - Depends on Notifier for communication + - Contains no infrastructure code (pure business logic) + +### Abstraction Layer (Dependency Inversion) +- **Repository interface**: Abstracts persistence (allows multiple backends) +- **Notifier interface**: Abstracts communication (email, SMS, push) + +This vertical slice structure keeps all user logic contained in one package, +following the principle: "group by feature and role, not technical layer." +``` + +### ❌ Code Dump as "Example" +```markdown +## Usage +See user_test.go for usage examples. +``` +*Why bad?*: Forces reader to hunt through test code + +### ✅ Inline Runnable Examples +```markdown +## Basic Usage + +Creating a new user with validated types: +```go +package main + +import ( + "context" + "fmt" + "github.com/yourorg/project/user" +) + +func main() { + // Create validated types + id, err := user.NewUserID("usr_12345") + if err != nil { + panic(err) // Invalid ID format + } + + email, err := user.NewEmail("alice@example.com") + if err != nil { + panic(err) // Invalid email format + } + + // Create user service + repo := user.NewPostgresRepository(db) + notifier := user.NewEmailNotifier(smtpConfig) + svc, _ := user.NewUserService(repo, notifier) + + // Create user + u := user.User{ + ID: id, + Email: email, + Name: "Alice", + } + + err = svc.CreateUser(context.Background(), u) + if err != nil { + fmt.Printf("Failed to create user: %v\n", err) + } +} +``` + +## Quality Gates + +Before considering documentation complete, verify: + +### Clarity Test +- Can someone unfamiliar with the code read this and understand the feature? +- Are design decisions explained, not just described? +- Is technical jargon explained or avoided? + +### AI Test +- Can AI use this to fix a bug without reading all implementation code? +- Are integration points clearly documented? +- Are invariants and assumptions explicit? + +### Maintenance Test +- If the feature needs extension, is it clear where to add code? +- Are patterns documented so new code matches existing style? +- Are limitations and future considerations noted? + +### Example Test +- Can examples be copy-pasted and run with minimal setup? +- Do examples demonstrate real-world usage patterns? +- Are edge cases covered in advanced examples? + +## Common Documentation Scenarios + +### Scenario 1: New Domain Type +Document: +- Why this type exists (what primitive obsession does it prevent?) +- What it validates +- How to construct it +- Where it's used in the system + +### Scenario 2: New Service/Orchestrator +Document: +- What business operations it provides +- What dependencies it requires (and why) +- How it fits into existing architecture +- Integration points with other services + +### Scenario 3: New Integration Point +Document: +- What external system/service is integrated +- Why this integration exists +- How data flows in/out +- Error handling strategy +- Retry/fallback behavior + +### Scenario 4: Refactored Architecture +Document: +- What problem the refactor solved +- What changed architecturally +- Why this approach was chosen +- Migration notes (if applicable) +- Before/after comparison + +## Testable Examples Best Practices + +### When to Add Example_* Functions +- Complex types with non-obvious usage +- Types with validation rules +- Common use case patterns +- Non-trivial workflows + +### Example_* Function Structure +```go +// Example_TypeName_Scenario describes what this example demonstrates. +func Example_TypeName_Scenario() { + // Setup (minimal) + input := "example input" + + // Usage (the point of the example) + result, err := SomeFunction(input) + if err != nil { + fmt.Printf("Error: %v\n", err) + return + } + + // Output (demonstrating result) + fmt.Println(result) + // Output: expected output +} +``` + +### Multiple Examples for Same Type +```go +// Example_UserID shows basic UserID creation. +func Example_UserID() { + id, _ := user.NewUserID("usr_123") + fmt.Println(id) + // Output: usr_123 +} + +// Example_UserID_validation shows validation behavior. +func Example_UserID_validation() { + _, err := user.NewUserID("") + fmt.Println(err != nil) + // Output: true +} + +// Example_UserID_invalidFormat shows error handling. +func Example_UserID_invalidFormat() { + _, err := user.NewUserID("invalid") + if err != nil { + fmt.Println("validation failed") + } + // Output: validation failed +} +``` + +## Documentation Location Strategy + +### Feature Documentation +- Location: `docs/[feature-name].md` +- One file per major feature +- Name should match package/feature name + +### Package Documentation +- Location: Package-level godoc in main `.go` file +- Example: `user/user.go` has package documentation + +### Type Documentation +- Location: Godoc on type definition +- Keep close to the code + +### Examples +- Location: Test files (`*_test.go`) +- Use `_test` package for external perspective diff --git a/skills/linter-driven-development/SKILL.md b/skills/linter-driven-development/SKILL.md new file mode 100644 index 0000000..f214dd5 --- /dev/null +++ b/skills/linter-driven-development/SKILL.md @@ -0,0 +1,481 @@ +--- +name: linter-driven-development +description: | + Orchestrates complete autopilot workflow: design → test → lint → refactor → review → commit. + AUTO-INVOKES when user wants to implement code: "implement", "ready", "execute", "continue", + "do step X", "next task", "let's go", "start coding". Runs automatically through all phases + until commit-ready. Uses parallel linter+review analysis and intelligent combined reports. + For: features, bug fixes, refactors. Requires: Go project (go.mod). +--- + +# Linter-Driven Development Workflow + +META ORCHESTRATOR for implementation workflow: design → test → lint → refactor → review → commit. +Use for any commit: features, bug fixes, refactors. + +## When to Use +- Implementing any code change that should result in a commit +- Need automatic workflow management with quality gates +- Want to ensure: clean code + tests + linting + design validation + +## Pre-Flight Check (ALWAYS RUN FIRST) + +Before starting the autopilot workflow, verify all conditions are met: + +### 1. Confirm Implementation Intent +Look for keywords indicating the user wants to implement code: +- **Direct keywords**: "implement", "ready", "execute", "do", "start", "continue", "next", "build", "create" +- **Step references**: "step 1", "task 2", "next task", "do step X" +- **Explicit invocation**: "@linter-driven-development" + +### 2. Verify Go Project +Check that `go.mod` exists in the project root or parent directories. + +### 3. Find Project Commands +Discover test and lint commands by reading project documentation: + +**Search locations** (in order): +1. Project docs: `README.md`, `CLAUDE.md`, `agents.md` +2. Build configs: `Makefile`, `Taskfile.yaml`, `.golangci.yaml` +3. Git repository root for workspace-level commands + +**Extract commands**: +- **Test command**: Look for `go test`, `make test`, `task test`, or similar +- **Lint command**: Look for `golangci-lint run --fix`, `make lint`, `task lintwithfix`, or similar +- **Prefer**: Commands with autofix capability (e.g., `--fix` flag) + +**Fallback defaults** (if not found in docs): +- Tests: `go test ./...` +- Linter: `golangci-lint run --fix` + +**If fallbacks don't work**: +- Ask user: "What commands should I use for testing and linting?" +- Document discovered commands in project docs for future runs + +**Store discovered commands** for use throughout the workflow. + +### 4. Identify Plan Context +Scan conversation history (last 50 messages) for: +- Step-by-step implementation plan +- Which step the user wants to implement +- Any design decisions or architectural context + +### 5. Decision Tree + +✅ **All conditions met → AUTOPILOT ENGAGED** +- Announce: "Engaging autopilot mode for [feature/step description]" +- Proceed directly to Phase 1 + +❓ **Unclear intent or missing context → ASK FOR CONFIRMATION** +- "I detected you want to implement something. Should I start the autopilot workflow?" +- Clarify which step to implement if multiple options exist + +❌ **No plan found → SUGGEST CREATING PLAN FIRST** +- "I don't see an implementation plan. Would you like me to help create one first?" +- Offer to use @code-designing skill for design planning + +❌ **Not a Go project → EXPLAIN LIMITATION** +- "This skill requires a Go project with go.mod. Current project doesn't appear to be Go." + +## Workflow Phases + +### Phase 1: Implementation Foundation + +**Design Architecture** (if new types/functions needed): +- Invoke @code-designing skill +- Output: Type design plan with self-validating domain types +- When in plan mode, invoke with plan mode flag + +**Write Tests First**: +- Invoke @testing skill for guidance +- Write table-driven tests or testify suites +- Target: 100% coverage on new leaf types + +**Implement Code**: +- Follow coding principles from coding_rules.md +- Keep functions <50 LOC, max 2 nesting levels +- Use self-validating types, prevent primitive obsession +- Apply storifying pattern for readable top-level functions + +### Phase 2: Quality Analysis (Agent is the Gate) + +**Invoke quality-analyzer agent** for parallel quality analysis: + +``` +Task(subagent_type: "quality-analyzer") + +Prompt: +"Analyze code quality for this Go project. + +Mode: full + +Project commands: +- Test: [PROJECT_TEST_COMMAND from Pre-Flight Check] +- Lint: [PROJECT_LINT_COMMAND from Pre-Flight Check] + +Files to analyze: +[list files from: git diff --name-only main...HEAD | grep '\.go$'] + +Run all quality gates in parallel and return combined analysis." +``` + +**The quality-analyzer agent automatically**: +- Executes tests, linter, and code review in parallel (40-50% faster) +- Normalizes outputs into common format +- Identifies overlapping issues (same file:line from multiple sources) +- Performs root cause analysis (why multiple issues occur together) +- Prioritizes fixes by impact (issues resolved per fix) +- Returns structured report with one of 4 statuses + +**Route based on agent status**: + +### Status: TEST_FAILURE → Enter Test Focus Mode + +**When**: Agent returns TEST_FAILURE status (tests failed) + +**Action**: Focus exclusively on fixing tests before any quality analysis + +``` +Loop until tests pass: + 1. Display test failures from agent report + 2. Analyze failure root cause + 3. Apply fix to implementation or tests + 4. Re-run quality-analyzer (mode: "full") + 5. Check status: + - Still TEST_FAILURE → Continue loop + - ISSUES_FOUND or CLEAN_STATE → Exit Test Focus Mode, proceed with new status + +Max 10 iterations. If stuck, ask user for guidance. +``` + +**Why Test Focus Mode**: +- Tests are Gate 1 - nothing else matters if tests fail +- Prevents wasting time on linter/review issues when code doesn't work +- Ensures quality analysis runs on working code + +**After tests pass**: Re-run quality-analyzer and continue with new status + +### Status: CLEAN_STATE → Skip to Phase 5 (Documentation) + +**When**: Agent returns CLEAN_STATE status +- ✅ Tests passed +- ✅ Linter clean (0 errors) +- ✅ Review clean (0 findings) + +**Action**: All quality gates passed! Skip fix loop, proceed directly to Phase 5 (Documentation) + +### Status: ISSUES_FOUND → Continue to Phase 3 (Fix Loop) + +**When**: Agent returns ISSUES_FOUND status +- ✅ Tests passed +- ❌ Linter has errors OR ⚠️ Review has findings + +**Action**: Display agent's combined report and proceed to Phase 3 + +**Agent Report Contains**: +- 📊 Summary: Tests, linter, review status +- 🎯 Overlapping issues with root cause analysis +- 📋 Isolated issues (single source only) +- 🔢 Prioritized fix order (by impact) + +**Example Report** (generated by quality-analyzer agent): +``` +═══════════════════════════════════════════════════════ +QUALITY ANALYSIS REPORT +Mode: FULL +Files analyzed: 8 +═══════════════════════════════════════════════════════ + +📊 SUMMARY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Tests: ✅ PASS (coverage: 87%) +Linter: ❌ FAIL (5 errors) +Review: ⚠️ FINDINGS (8 issues: 0 bugs, 3 design, 4 readability, 1 polish) + +Total issues: 13 from 3 sources + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +OVERLAPPING ISSUES ANALYSIS +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Found 3 locations with overlapping issues: + +┌─────────────────────────────────────────────────────┐ +│ pkg/parser.go:45 - function Parse │ +│ OVERLAPPING (4 issues): │ +│ │ +│ ⚠️ Linter: Cognitive complexity 18 (>15) │ +│ ⚠️ Linter: Function length 58 statements (>50) │ +│ 🔴 Review: Mixed abstraction levels │ +│ 🔴 Review: Defensive null checking │ +│ │ +│ 🎯 ROOT CAUSE: │ +│ Function handles multiple responsibilities at │ +│ different abstraction levels (parsing, validation, │ +│ building result). │ +│ │ +│ Impact: HIGH (4 issues) | Complexity: MODERATE │ +│ Priority: #1 CRITICAL │ +└─────────────────────────────────────────────────────┘ + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +PRIORITIZED FIX ORDER +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Priority #1: pkg/parser.go:45 (4 issues, HIGH impact) +Priority #2: pkg/validator.go:23 (3 issues, HIGH impact) +Priority #3: pkg/handler.go:67 (2 issues, MEDIUM impact) + +Isolated issues: 6 (fix individually) + +Total fix targets: 3 overlapping groups + 6 isolated = 9 fixes + +STATUS: ISSUES_FOUND +``` + +### Status: TOOLS_UNAVAILABLE → Report Error + +**When**: Agent returns TOOLS_UNAVAILABLE status (missing tools) + +**Action**: Display agent report with missing tools and suggestions, ask user to install tools + +### Phase 3: Iterative Fix Loop + +**For each prioritized fix** (from agent's report): + +1. **Apply Fix**: + - Invoke @refactoring skill with: + * File and function to fix + * All issues in that area (from agent's overlapping groups or isolated issues) + * Root cause analysis from agent (if available) + * Expected outcome + - @refactoring applies appropriate patterns: + * Early returns (reduce nesting) + * Extract function (break complexity) + * Storifying (uniform abstractions) + * Extract type (create domain types) + * Switch extraction (categorize cases) + * Extract constant (remove magic numbers) + +2. **Verify Fix with Quality-Analyzer Agent (Incremental Mode)**: + + ``` + Task(subagent_type: "quality-analyzer") + + Prompt: + "Re-analyze code quality after refactoring. + + Mode: incremental + + Project commands: + - Test: [PROJECT_TEST_COMMAND] + - Lint: [PROJECT_LINT_COMMAND] + + Files to analyze (changed): + [list files from: git diff --name-only HEAD~1 HEAD | grep '\.go$'] + + Previous findings: + [paste findings from previous quality-analyzer report] + + Run quality gates and return delta report (what changed)." + ``` + + **Agent returns delta report with status**: + - ✅ Fixed: Issues resolved since last run + - ⚠️ Remaining: Issues still present + - 🆕 New: Issues introduced by recent changes + +3. **Route Based on Agent Status**: + + **If status = TEST_FAILURE**: + - → Enter Test Focus Mode (refactoring broke tests) + - Loop until tests pass (same as Phase 2) + - Continue with new status + + **If status = CLEAN_STATE**: + - → All issues resolved! Break out of fix loop + - Continue to Phase 4 (Documentation) + + **If status = ISSUES_FOUND**: + - Check delta report for progress: + * ✅ If issues were fixed → Continue to next fix + * ⚠️ If no progress → Analyze why, try different approach + * 🆕 If new issues introduced → Fix them first + +4. **Safety Limits**: + - Max 10 iterations per fix loop + - IF stuck (no progress after 3 attempts): + * Show current status and delta report + * Ask user for guidance + * User can review: `git diff` + +**Example Delta Report** (from quality-analyzer agent): +``` +═══════════════════════════════════════════════════════ +QUALITY ANALYSIS DELTA REPORT +Mode: INCREMENTAL +Files re-analyzed: 1 (changed since last run) +═══════════════════════════════════════════════════════ + +📊 SUMMARY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Tests: ✅ PASS (coverage: 89% ↑) +Linter: ✅ PASS (0 errors) +Review: ✅ CLEAN (0 findings) + +✅ Fixed: 4 issues from pkg/parser.go:45 +⚠️ Remaining: 0 issues +🆕 New: 0 issues introduced + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +RESOLUTION DETAILS +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +✅ FIXED: + pkg/parser.go:45 | Linter | Cognitive complexity (was 18, now 8) + pkg/parser.go:45 | Linter | Function length (was 58, now 25) + pkg/parser.go:45 | Review | Mixed abstraction levels (resolved) + pkg/parser.go:45 | Review | Defensive null checking (resolved) + +STATUS: CLEAN_STATE ✅ +Ready to proceed with next fix or move to documentation phase. +``` + +**Loop until agent returns CLEAN_STATE**: +- ✅ Tests pass +- ✅ Linter clean +- ✅ Review clean + +### Phase 4: Documentation + +Invoke @documentation skill: + +1. Add/update package-level godoc +2. Add/update type-level documentation +3. Add/update function documentation (WHY not WHAT) +4. Add godoc testable examples (Example_* functions) +5. IF last plan step: + - Add feature documentation to docs/ folder + +**Verify**: +- Run: `go doc -all ./...` +- Ensure examples compile +- Check documentation coverage + +### Phase 5: Commit Ready + +Generate comprehensive summary with suggested commit message. + +- Linter passes ✅ +- Tests pass with coverage ✅ +- Design review complete ✅ +- Documentation complete ✅ +- Present commit message suggestion + +## Output Format + +``` +📋 COMMIT READINESS SUMMARY + +✅ Linter: Passed (0 issues) +✅ Tests: 95% coverage (3 new types, 15 test cases) +⚠️ Design Review: 4 findings (see below) + +🎯 COMMIT SCOPE +Modified: +- user/service.go (+45, -12 lines) +- user/repository.go (+23, -5 lines) + +Added: +- user/user_id.go (new type: UserID) +- user/email.go (new type: Email) + +Tests: +- user/service_test.go (+120 lines) +- user/user_id_test.go (new) +- user/email_test.go (new) + +⚠️ DESIGN REVIEW FINDINGS + +🔴 DESIGN DEBT (Recommended to fix): +- user/service.go:45 - Primitive obsession detected + Current: func GetUserByID(id string) (*User, error) + Better: func GetUserByID(id UserID) (*User, error) + Why: Type safety, validation guarantee, prevents invalid IDs + Fix: Use @code-designing to convert remaining string usages + +🟡 READABILITY DEBT (Consider fixing): +- user/service.go:78 - Mixed abstraction levels in CreateUser + Function mixes high-level steps with low-level validation details + Why: Harder to understand flow at a glance + Fix: Use @refactoring to extract validation helpers + +🟢 POLISH OPPORTUNITIES: +- user/repository.go:34 - Function naming could be more idiomatic + SaveUser → Save (method receiver provides context) + +📝 BROADER CONTEXT: +While reviewing user/service.go, noticed 3 more instances of string-based +IDs throughout the file (lines 120, 145, 203). Consider refactoring the +entire file to use UserID consistently for better type safety. + +💡 SUGGESTED COMMIT MESSAGE +Add self-validating UserID and Email types + +- Introduce UserID type with validation (prevents empty IDs) +- Introduce Email type with RFC 5322 validation +- Refactor CreateUser to use new types +- Achieve 95% test coverage with real repository implementation + +Follows vertical slice architecture and primitive obsession principles. + +──────────────────────────────────────── + +Would you like to: +1. Commit as-is (ignore design findings) +2. Fix design debt only (🔴), then commit +3. Fix design + readability debt (🔴 + 🟡), then commit +4. Fix all findings (🔴 🟡 🟢), then commit +5. Refactor entire file (address broader context), then commit +``` + +## Workflow Control + +**Sequential Phases**: Each phase depends on previous phase completion +- Phase 1: Design and implementation must complete before quality analysis +- Phase 2: Quality analysis (via quality-analyzer agent) determines next phase +- Phase 3: Fix loop continues until all issues resolved (agent returns CLEAN_STATE) +- Phase 4: Documentation only after all quality gates pass +- Phase 5: Commit ready summary presented to user + +**Status-Based Routing**: Agent determines workflow path +- **TEST_FAILURE** → Test Focus Mode (fix tests, retry Phase 2) +- **CLEAN_STATE** → Skip fix loop, go directly to documentation +- **ISSUES_FOUND** → Enter fix loop (Phase 3) +- **TOOLS_UNAVAILABLE** → Report error, ask user to install tools + +**Parallel Execution**: Phase 2 and fix verification run 3 tools simultaneously (40-50% faster) + +**Incremental Review**: After first run, agent only analyzes changed files for faster iteration + +## Integration with Other Skills and Agents + +This orchestrator **invokes** other skills and agents automatically: + +**Skills**: +- @code-designing (Phase 1, if needed for type design) +- @testing (Phase 1, principles applied) +- @refactoring (Phase 3, when issues found) +- @documentation (Phase 4, always) + +**Agents**: +- quality-analyzer agent (Phase 2 and Phase 3 verification) + - Internally delegates to go-code-reviewer agent for design analysis + - Executes tests and linter in parallel + - Returns intelligent combined reports with overlapping issue detection + +After committing, consider: +- If feature complete → Feature fully documented in Phase 4 +- If more work needed → Run this workflow again for next commit diff --git a/skills/linter-driven-development/reference.md b/skills/linter-driven-development/reference.md new file mode 100644 index 0000000..8f5f639 --- /dev/null +++ b/skills/linter-driven-development/reference.md @@ -0,0 +1,100 @@ +# Linter-Driven Development Reference + +This meta-orchestrator skill coordinates other skills. See individual skill reference files for detailed principles: + +## Phase-Specific References + +### Phase 1: Design +See: **code-designing/reference.md** +- Type design principles +- Primitive obsession prevention +- Self-validating types +- Vertical slice architecture + +### Phase 2: Implementation & Testing +See: **testing/reference.md** +- Testing principles +- Table-driven tests +- Testify suites +- Real implementations over mocks + +### Phase 3: Linter & Refactoring +See: **refactoring/reference.md** +- Linter signal interpretation +- Refactoring patterns +- Complexity reduction strategies + +### Phase 4: Pre-Commit Review +See: **pre-commit-review/reference.md** +- Design principles checklist +- Debt categorization +- Review process + +## Linter Commands + +### Primary Command +```bash +task lintwithfix +``` +Runs: +1. `go vet` - Static analysis +2. `golangci-lint fmt` - Format code +3. `golangci-lint run --fix` - Lint with auto-fix + +### Fallback (if no taskfile) +```bash +golangci-lint run --fix +``` + +### Configuration +- Config file: `.golangci.yaml` in project root +- Always use golangci-lint v2 +- Reference: https://github.com/golangci/golangci-lint/blob/HEAD/.golangci.reference.yml + +## Linter Failure Signals + +### Cyclomatic Complexity +**Signal**: Function too complex (too many decision points) +**Action**: Extract functions, simplify logic flow +**Skill**: @refactoring + +### Cognitive Complexity +**Signal**: Function hard to understand (nested logic, mixed abstractions) +**Action**: Storifying, extract helpers, clarify abstraction levels +**Skill**: @refactoring + +### Maintainability Index +**Signal**: Code difficult to maintain +**Action**: Break into smaller pieces, improve naming, reduce coupling +**Skill**: @refactoring + potentially @code-designing + +## Coverage Targets + +### Leaf Types +- **Target**: 100% unit test coverage +- **Why**: Leaf types contain core logic, must be bulletproof +- **Test**: Only public API, use pkg_test package + +### Orchestrating Types +- **Target**: Integration test coverage +- **Why**: Test seams between components +- **Test**: Can overlap with leaf type coverage + +## Commit Readiness Criteria + +All must be true: +- ✅ Linter passes with 0 issues +- ✅ Tests pass +- ✅ Target coverage achieved (100% for leaf types) +- ✅ Design review complete (advisory, but acknowledged) + +## Next Steps After Commit + +### Feature Complete? +→ Invoke @documentation skill to create feature docs + +### More Work Needed? +→ Run @linter-driven-development again for next commit + +### Found Broader Issues During Review? +→ Create new task to address technical debt diff --git a/skills/pre-commit-review/SKILL.md b/skills/pre-commit-review/SKILL.md new file mode 100644 index 0000000..ad0a555 --- /dev/null +++ b/skills/pre-commit-review/SKILL.md @@ -0,0 +1,322 @@ +--- +name: pre-commit-review +description: ADVISORY validation of code against design principles that linters cannot enforce. Use after linter passes and tests pass to validate design quality. Categorizes findings as Design Debt, Readability Debt, or Polish Opportunities. Does NOT block commits. +--- + +# Pre-Commit Design Review + +Expert design analysis that detects issues linters can't catch. Returns detailed report to caller with categorized findings and fix recommendations. + +## What This Skill Does + +**Pure Analysis & Reporting** - Generates report, doesn't fix anything or invoke skills. + +### Input +- Files to review (specific files or all staged changes) +- Review mode: `full` (first run) or `incremental` (subsequent runs) +- Previous findings (optional, for incremental mode) +- Context (invoked by refactoring, orchestrator, subagent, or user) + +### Output +- Structured report with categorized findings +- Each finding: `file:line`, issue, why it matters, fix strategy, effort estimate +- Prioritized by impact and effort +- Format: Parseable for combined analysis (when invoked by orchestrator) + +### Invocation Modes + +**1. Direct Skill Invocation** (User or Orchestrator) +- Full control, can invoke other skills +- Can make changes based on findings +- Interactive mode with user feedback + +**2. Subagent Mode** (Task tool with go-code-reviewer) +- Read-only analysis, returns report only +- Cannot invoke other skills +- Used for parallel execution by orchestrator +- Designed for speed and focused analysis + +### What Reviewer Detects (That Linters Can't) +- Primitive obsession (with juiciness scoring) +- Unstorified functions (mixed abstraction levels) +- Missing domain concepts (implicit types that should be explicit) +- Non-self-validating types (defensive code in methods) +- Poor comment quality (explaining what instead of why) +- File structure issues (too long, too many types) +- Generic package extraction opportunities +- Design bugs (nil deref, ignored errors, resource leaks) +- Test quality (weak assertions, missing use cases, mock overuse, conditionals in tests) + +**See [reference.md](./reference.md) for complete detection checklist with examples** + +## Who Invokes This Skill + +1. **@refactoring skill** - After applying patterns, validates design quality remains high +2. **@linter-driven-development** - Phase 4, checks design quality after linter passes +3. **User** - Manual code review before commit + +## Workflow + +### Full Review Mode (First Run) + +``` +1. Read all files under review (using Read tool) +2. Apply design principles checklist from reference.md (LLM reasoning) +3. Search for usage patterns across codebase (using Grep tool) +4. Categorize findings: + 🐛 Bugs (nil deref, ignored errors, resource leaks) + 🔴 Design Debt (types, architecture, validation) + 🟡 Readability Debt (abstraction, flow, clarity) + 🟢 Polish (naming, docs, minor improvements) +5. Generate structured report with recommendations +6. Return report to caller (doesn't invoke other skills or make fixes) +``` + +### Incremental Review Mode (Subsequent Runs) + +Used after fixes have been applied to verify resolution and detect new issues. + +``` +1. Read ONLY changed files since last review (using git diff) +2. Compare against previous findings: + - Mark resolved issues as ✅ Fixed + - Identify issues that still exist +3. Analyze changed code for NEW issues introduced by fixes +4. Generate delta report: + - ✅ Fixed: Issues from previous run that are now resolved + - ⚠️ Remaining: Issues that still need attention + - 🆕 New: Issues introduced by recent changes +5. Return concise delta report (not full analysis) +``` + +**When to Use Incremental Mode:** +- After @refactoring skill applies fixes +- During iterative fix loop in Phase 4 of autopilot workflow +- User requests re-review after making changes + +**Benefits:** +- Faster execution (only analyzes changed files) +- Clear feedback on what was fixed vs what remains +- Detects regressions introduced by fixes + +## Detection Approach + +**LLM-Powered Analysis** (not AST parsing or metrics calculation): + +The reviewer reads code like a senior developer and applies design principles: +- Reads files with Read tool +- Searches patterns with Grep tool (find usages, duplications) +- Applies checklist from reference.md using LLM reasoning +- Pattern matches against anti-patterns +- Counts occurrences and calculates juiciness scores +- Generates findings with specific locations and fix guidance + +**Division of Labor:** +- **Linter handles**: Complexity metrics, line counts, formatting, syntax +- **Reviewer handles**: Design patterns, domain modeling, conceptual issues + +## Report Format + +### Full Report (First Run) + +``` +📊 CODE REVIEW REPORT +Scope: [files reviewed] +Mode: FULL + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +SUMMARY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Total findings: 18 +🐛 Bugs: 2 (fix immediately) +🔴 Design Debt: 5 (fix before commit) +🟡 Readability Debt: 8 (improves maintainability) +🟢 Polish: 3 (nice to have) + +Estimated fix effort: 3.5 hours + +[Detailed findings by category] +[Recommendations by priority] +[Skills to use for fixes] +``` + +### Incremental Report (Subsequent Runs) + +``` +📊 CODE REVIEW DELTA REPORT +Scope: [changed files only] +Mode: INCREMENTAL + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +SUMMARY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +✅ Fixed: 4 (resolved from previous run) +⚠️ Remaining: 2 (still need attention) +🆕 New: 1 (introduced by recent changes) + +[Detailed delta findings] +``` + +### Structured Output for Orchestrator Parsing + +When invoked as subagent for combined analysis, output follows strict format: + +``` +🐛 BUGS +──────────────────────────────────────────────── +file:line | Issue description | Why it matters | Fix strategy | Effort: [Trivial/Moderate/Significant] + +🔴 DESIGN DEBT +──────────────────────────────────────────────── +file:line | Issue description | Why it matters | Fix strategy | Effort: [Trivial/Moderate/Significant] + +🟡 READABILITY DEBT +──────────────────────────────────────────────── +file:line | Issue description | Why it matters | Fix strategy | Effort: [Trivial/Moderate/Significant] + +🟢 POLISH +──────────────────────────────────────────────── +file:line | Issue description | Why it matters | Fix strategy | Effort: [Trivial/Moderate/Significant] +``` + +**Effort Estimates:** +- **Trivial**: <5 minutes (extract constant, rename variable) +- **Moderate**: 5-20 minutes (extract function, storifying, create simple type) +- **Significant**: >20 minutes (architectural refactoring, complex type extraction) + +**file:line Format:** Must be exact for orchestrator to correlate with linter errors +- Example: `pkg/parser.go:45` +- NOT: `parser.go line 45` or `pkg/parser.go (line 45)` + +**See [examples.md](./examples.md) for complete report examples** + +## What This Skill Does NOT Do + +- ❌ Invoke other skills (@refactoring, @code-designing, @testing) +- ❌ Fix anything or make code changes +- ❌ Make decisions on behalf of user +- ❌ Parse AST or calculate complexity metrics (linter does this) +- ❌ Run linter (caller does this) +- ❌ Iterate or loop (caller decides whether to re-invoke) +- ❌ Block commits (findings are advisory) + +## Integration with Other Skills + +### Invoked by @refactoring +``` +Refactoring completes → invoke reviewer → analyze report: +- Bugs found? → Fix immediately, re-run linter +- Design debt found? → Apply another refactoring pattern +- All clean? → Return success to orchestrator +``` + +### Invoked by @linter-driven-development +``` +Phase 4 (after linter passes): +1. Invoke reviewer on all staged changes +2. Receive categorized report +3. Present findings to user with options: + - Commit as-is (accept debt knowingly) + - Fix critical issues only (bugs + design debt) + - Fix all recommended (bugs + design + readability) + - Fix everything (including polish) +4. Based on user choice: + - Invoke @refactoring or @code-designing for chosen fixes + - Return to Phase 3 (linter loop) + - Iterate until user satisfied +``` + +### Invoked by User +``` +Manual review request: +1. User invokes: @pre-commit-review on path/to/file.go +2. Receive detailed report +3. User decides how to proceed +4. User may invoke @refactoring or @code-designing for fixes +``` + +## Review Scope + +**Primary Scope**: Changed code in commit +- All modified lines +- All new files +- Specific focus on design principle adherence + +**Secondary Scope**: Context around changes +- Entire files containing modifications +- Flag patterns/issues outside commit scope (in BROADER CONTEXT section) +- Suggest broader refactoring opportunities if valuable + +## Advisory Nature + +**This review does NOT block commits.** + +Purpose: +- ✅ Provide visibility into design quality +- ✅ Offer concrete improvement suggestions with examples +- ✅ Help maintain coding principles +- ✅ Guide refactoring decisions + +Caller (or user) decides: +- Commit as-is (accept debt knowingly) +- Fix critical debt before commit (bugs, major design issues) +- Fix all debt before commit (comprehensive cleanup) +- Expand scope to broader refactor (when broader context issues found) + +## Finding Categories + +Findings are categorized by technical debt type and severity: + +### 🐛 Bugs +**Will cause runtime failures or correctness issues** +- Nil dereferences, ignored errors, resource leaks +- Invalid nil returns, race conditions +- Fix immediately before any other work + +### 🔴 Design Debt +**Will cause pain when extending/modifying code** +- Primitive obsession, missing domain types +- Non-self-validating types +- Wrong architecture (horizontal vs vertical) +- Fix before commit recommended + +### 🟡 Readability Debt +**Makes code harder to understand and work with** +- Mixed abstraction levels, not storified +- Functions too long or complex +- Poor naming, unclear intent +- Fix improves team productivity + +### 🟢 Polish Opportunities +**Minor improvements for consistency and quality** +- Non-idiomatic naming, missing examples +- Comment improvements, minor refactoring +- Low priority, nice to have + +**See [reference.md](./reference.md) for detailed principles and examples for each category** + +## Key Capabilities + +**Detects 8 Issue Categories:** +1. Primitive Obsession - with juiciness scoring algorithm +2. Storifying - detects mixed abstraction levels +3. Missing Domain Concepts - identifies implicit types +4. Self-Validating Types - finds defensive code patterns +5. Comment Quality - analyzes what vs why +6. File Structure - checks size and responsibility boundaries +7. Testing Approach - validates test structure and quality +8. Design Bugs - catches common runtime issues + +**For complete detection patterns and examples, see [reference.md](./reference.md)** +**For real-world review scenarios, see [examples.md](./examples.md)** + +## Integration with Orchestrator + +This skill is automatically invoked by @linter-driven-development workflow: +- **Phase 4**: Design review after linter passes +- **Iterative**: Re-invoked after fixes until clean or user accepts debt +- **Advisory**: Never blocks, always presents options + +See [linter-driven-development workflow](../linter-driven-development/SKILL.md) for complete flow. diff --git a/skills/pre-commit-review/examples.md b/skills/pre-commit-review/examples.md new file mode 100644 index 0000000..9b263d5 --- /dev/null +++ b/skills/pre-commit-review/examples.md @@ -0,0 +1,832 @@ +# Pre-Commit Review Examples + +Real-world examples of design review findings with before/after comparisons. + +## Example 0: Detection Workflow + +Shows how the pre-commit-review skill analyzes code step-by-step using LLM reasoning. + +### Input: user/service.go + +```go +package user + +type UserService struct { + Repo Repository +} + +func (s *UserService) CreateUser(id string, email string) error { + // Validate ID + if id == "" { + return errors.New("empty id") + } + + // Validate email + if !strings.Contains(email, "@") { + return errors.New("invalid email") + } + + // Open DB + db, err := sql.Open("postgres", connStr) + if err != nil { + return err + } + defer db.Close() + + // Build query + query := "INSERT INTO users (id, email) VALUES (?, ?)" + _, err = db.Exec(query, id, email) + if err != nil { + return err + } + + // Send email + _ = sendEmail(email, "Welcome!") + + return nil +} +``` + +### LLM Analysis Step-by-Step + +**Step 1: Apply Primitive Obsession Check (reference.md #1)** + +Questions LLM asks: +- Q: Are primitives being validated? +- A: Yes - 'id' string validated at line 9, 'email' string validated at line 14 + +- Q: Is same validation repeated elsewhere? +- A: [Uses Grep tool to search codebase] + - Found in handler.go:23 (same id validation) + - Found in repository.go:45 (same email validation) + +- Q: Would types add value beyond validation? +- A: Yes + - Email: Complex validation pattern, domain concept + - UserID: Validation scattered, type safety benefit + +- Q: Calculate juiciness score (see reference.md): + ``` + UserID: + - Validation in 3 places (+1) + - Simple empty check (+0) + - Used in 8 locations (+2) + Score: 3/10 → MEDIUM priority + + Email: + - Validation in 3 places (+1) + - Regex validation possible (+3) + - Domain/LocalPart methods (+2) + - Used in 8 locations (+2) + Score: 8/10 → HIGH priority + ``` + +**Finding identified:** +``` +🔴 Design Debt: Primitive obsession +- string id → UserID type (MEDIUM juiciness) +- string email → Email type (HIGH juiciness) +``` + +--- + +**Step 2: Apply Storifying Check (reference.md #2)** + +Questions LLM asks: +- Q: Does function read like a story? +- A: No - mixes abstraction levels: + - Lines 7-16: High-level validation + - Lines 18-23: Low-level DB connection setup + - Lines 25-31: SQL implementation details + - Lines 33-34: High-level notification + +- Q: Can I explain flow in 3-5 clear steps? +- A: Should be: validateUser() → saveUser() → notifyUser() + But currently mixes infrastructure and business logic + +**Finding identified:** +``` +🟡 Readability Debt: Function not storified +Mixed abstraction levels in CreateUser +Should extract: validateUser(), saveUser(), notifyUser() +``` + +--- + +**Step 3: Apply Self-Validating Types Check (reference.md #3)** + +Questions LLM asks: +- Q: Do methods check if struct fields are nil/invalid? +- A: No checks in this method, but... + +- Q: Could UserService.Repo be nil? +- A: Yes - struct can be created without validation + ```go + svc := &UserService{} // Repo is nil! + svc.CreateUser(...) // Will panic if we use Repo + ``` + +- Q: Is there a constructor with validation? +- A: [Searches codebase] No NewUserService constructor found + +**Finding identified:** +``` +🔴 Design Debt: Non-self-validating type +UserService can be created with nil Repo +Need NewUserService constructor +``` + +--- + +**Step 4: Apply Design Bugs Check (reference.md #8)** + +Questions LLM asks: +- Q: Potential nil dereferences? +- A: Not obvious in this function (but found UserService.Repo issue above) + +- Q: Are errors ignored? +- A: YES! Line 34: `_ = sendEmail(email, "Welcome!")` + Silent failure, no visibility into delivery failures + +- Q: Resources cleaned up? +- A: Yes, defer db.Close() at line 23 ✓ + +- Q: Context propagated? +- A: No ctx parameter, but no downstream calls needing it + +**Finding identified:** +``` +🐛 Bug: Ignored error +Line 34: _ = sendEmail() silently ignores failure +Should at least log the error +``` + +--- + +**Step 5: Apply Other Checks** + +- Abstraction Levels (#4): Already covered in storifying +- Architecture (#5): Single file, can't assess pattern +- Naming (#6): Names are clear and idiomatic ✓ +- Testing (#7): No test file in scope +- Comment Quality: Missing (no package docs, no function docs) +- File Structure: Single type, reasonable size ✓ + +--- + +### Analysis Complete - Categorize & Prioritize + +**Bugs (Priority 1 - Fix immediately):** +1. Ignored error at line 34 + +**Design Debt (Priority 2 - Fix before commit):** +2. Primitive obsession: string email (HIGH juiciness: 8/10) +3. Primitive obsession: string id (MEDIUM juiciness: 3/10) +4. Non-self-validating: UserService + +**Readability Debt (Priority 3 - Improves maintainability):** +5. Not storified: CreateUser function + +--- + +### Output Report + +``` +📊 CODE REVIEW REPORT +Generated: 2025-11-09 15:45:00 +Scope: user/service.go (1 file, 40 lines) + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +SUMMARY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Total findings: 5 +🐛 Bugs: 1 (fix immediately) +🔴 Design Debt: 3 (fix before commit recommended) +🟡 Readability Debt: 1 (improves maintainability) +🟢 Polish: 0 + +Estimated fix effort: 50 minutes total + - Critical (bugs + high juiciness design): 25 min + - Recommended (medium design + readability): 25 min + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +🐛 BUGS (1) - FIX IMMEDIATELY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +1. Ignored error + Location: user/service.go:34 + Code: _ = sendEmail(email, "Welcome!") + + Issue: Email sending failure silently ignored + Impact: No visibility into delivery failures, hard to debug + + Fix: Log error at minimum: + if err := sendEmail(email, "Welcome!"); err != nil { + log.Printf("failed to send welcome email: %v", err) + } + + Better: Return error if critical: + if err := sendEmail(email, "Welcome!"); err != nil { + return fmt.Errorf("send welcome email: %w", err) + } + + Effort: 2 min + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +🔴 DESIGN DEBT (3) - FIX BEFORE COMMIT RECOMMENDED +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +1. Primitive obsession: string email (HIGH juiciness: 8/10) + Locations: Line 6, 14, 28, 34 + Also found in: handler.go:23, repository.go:45 + + Juiciness Score: 8/10 + - Validation in 3 places (+1) + - Complex regex validation possible (+3) + - Methods: Domain(), LocalPart() (+2) + - Used in 8 locations (+2) + + Current: + func CreateUser(id string, email string) error { + if !strings.Contains(email, "@") { + return errors.New("invalid email") + } + // ... + } + + Better: + type Email string + + func ParseEmail(s string) (Email, error) { + if !emailRegex.MatchString(s) { + return "", fmt.Errorf("invalid email: %s", s) + } + return Email(s), nil + } + + func (e Email) Domain() string { /* ... */ } + func (e Email) LocalPart() string { /* ... */ } + func (e Email) String() string { return string(e) } + + func CreateUser(id string, email Email) error { + // No validation needed, guaranteed valid + } + + Why: Type safety, centralized validation, prevents invalid emails + Fix: Use @code-designing skill → Create Email type + Effort: 20 min + +2. Primitive obsession: string id (MEDIUM juiciness: 3/10) + Locations: Line 6, 9, 28 + Also found in: handler.go:23, repository.go:45 + + Juiciness Score: 3/10 + - Validation in 3 places (+1) + - Simple empty check (+0) + - Used in 8 locations (+2) + + Note: Borderline case. Judgment call on whether to create type. + + Better: + type UserID string + + func ParseUserID(s string) (UserID, error) { + if s == "" { + return "", errors.New("empty user id") + } + return UserID(s), nil + } + + Why: Centralizes validation, type safety + Fix: Use @code-designing skill → Create UserID type + Effort: 10 min + +3. Non-self-validating type: UserService + Location: user/service.go:4 + + Issue: UserService.Repo is public, can be nil + No constructor to validate dependencies + + Current: + type UserService struct { + Repo Repository // Can be nil! + } + + svc := &UserService{} // Invalid state allowed + + Better: + type UserService struct { + repo Repository // Private + } + + func NewUserService(repo Repository) (*UserService, error) { + if repo == nil { + return nil, errors.New("repo required") + } + return &UserService{repo: repo}, nil + } + + func (s *UserService) CreateUser(id string, email Email) error { + // No nil checks needed - constructor guarantees validity + return s.repo.Save(...) + } + + Why: Impossible to create invalid service, eliminates defensive checks + Fix: Use @code-designing skill → Add constructor + Effort: 10 min + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +🟡 READABILITY DEBT (1) - IMPROVES MAINTAINABILITY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +1. Function not storified: CreateUser + Location: user/service.go:6-36 + + Issue: Mixes 3 abstraction levels: + - Lines 7-16: High-level validation + - Lines 18-31: Low-level DB connection/SQL + - Lines 33-34: High-level notification + + Flow not clear at a glance, hard to test pieces independently. + + Better: + func CreateUser(id string, email Email) error { + if err := validateUser(id, email); err != nil { + return err + } + + if err := saveUser(id, email); err != nil { + return err + } + + if err := notifyUser(email); err != nil { + return err + } + + return nil + } + + func validateUser(id string, email Email) error { /* ... */ } + func saveUser(id string, email Email) error { /* ... */ } + func notifyUser(email Email) error { /* ... */ } + + Why: Reads like a story, testable pieces, clear intent + Fix: Use @refactoring skill → Storifying pattern + Effort: 15 min + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +RECOMMENDATIONS +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +Priority 1: Fix immediately (2 min) + ☐ Fix ignored error (log or return) + +Priority 2: Fix before commit (40 min) + ☐ Create Email type (HIGH juiciness) @code-designing + ☐ Create UserID type (MEDIUM juiciness) @code-designing + ☐ Add NewUserService constructor @code-designing + +Priority 3: Improve maintainability (15 min) + ☐ Storify CreateUser function @refactoring + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +SKILLS TO USE +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +@code-designing: For creating Email, UserID types and NewUserService +@refactoring: For storifying CreateUser function +Manual: For fixing ignored error (simple change) + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +END OF REPORT +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +``` + +This example demonstrates how the reviewer skill applies the complete checklist from reference.md systematically, using LLM reasoning to detect issues that linters cannot catch. + +--- + +## Example 1: Primitive Obsession + Self-Validating Types + +### Before (Design Debt 🔴) +```go +package user + +type UserService struct { + DB *sql.DB // Might be nil +} + +func (s *UserService) CreateUser(id string, email string) error { + // Defensive check + if s.DB == nil { + return errors.New("db is nil") + } + + // Primitive validation + if id == "" { + return errors.New("id required") + } + if !strings.Contains(email, "@") { + return errors.New("invalid email") + } + + // Business logic + _, err := s.DB.Exec("INSERT INTO users (id, email) VALUES ($1, $2)", id, email) + return err +} +``` + +**Review Findings:** +- 🔴 **Design Debt**: Primitive obsession on `id` and `email` +- 🔴 **Design Debt**: Non-self-validating type (`UserService.DB` might be nil) + +### After (No Debt) +```go +package user + +type UserID string +type Email string + +func NewUserID(s string) (UserID, error) { + if s == "" { + return "", errors.New("id required") + } + return UserID(s), nil +} + +func NewEmail(s string) (Email, error) { + if !strings.Contains(s, "@") { + return "", errors.New("invalid email") + } + return Email(s), nil +} + +type UserService struct { + db *sql.DB +} + +func NewUserService(db *sql.DB) (*UserService, error) { + if db == nil { + return nil, errors.New("db is required") + } + return &UserService{db: db}, nil +} + +func (s *UserService) CreateUser(id UserID, email Email) error { + // No validation needed - types guarantee validity + _, err := s.db.Exec("INSERT INTO users (id, email) VALUES ($1, $2)", id, email) + return err +} +``` + +--- + +## Example 2: Mixed Abstraction Levels + Storifying + +### Before (Readability Debt 🟡) +```go +func ProcessPayment(orderID string, amount float64) error { + // High-level: validation + if orderID == "" { + return errors.New("invalid order id") + } + if amount <= 0 { + return errors.New("invalid amount") + } + + // Low-level: HTTP client setup + client := &http.Client{Timeout: 10 * time.Second} + req, err := http.NewRequest("POST", "https://api.payment.com/charge", nil) + if err != nil { + return err + } + req.Header.Set("Authorization", "Bearer "+os.Getenv("API_KEY")) + req.Header.Set("Content-Type", "application/json") + + // Low-level: JSON marshaling + body := map[string]interface{}{ + "order_id": orderID, + "amount": amount, + } + jsonBody, err := json.Marshal(body) + if err != nil { + return err + } + req.Body = io.NopCloser(bytes.NewReader(jsonBody)) + + // Low-level: HTTP call + resp, err := client.Do(req) + if err != nil { + return err + } + defer resp.Body.Close() + + // High-level: logging + log.Printf("Payment processed for order %s", orderID) + return nil +} +``` + +**Review Findings:** +- 🟡 **Readability Debt**: Mixed abstraction levels (business + HTTP details) +- 🟡 **Readability Debt**: Function not storified (hard to see flow) + +### After (No Debt) +```go +func ProcessPayment(orderID OrderID, amount Amount) error { + if err := validatePayment(orderID, amount); err != nil { + return err + } + + if err := chargePaymentGateway(orderID, amount); err != nil { + return err + } + + logPaymentSuccess(orderID) + return nil +} + +func validatePayment(orderID OrderID, amount Amount) error { + // Validation logic only (already validated by types, but could have business rules) + return nil +} + +func chargePaymentGateway(orderID OrderID, amount Amount) error { + // HTTP client logic encapsulated + client := newPaymentClient() + return client.Charge(orderID, amount) +} + +func logPaymentSuccess(orderID OrderID) { + log.Printf("Payment processed for order %s", orderID) +} +``` + +--- + +## Example 3: Horizontal Layers → Vertical Slices + +### Before (Design Debt 🔴) +``` +project/ +├── domain/ +│ └── user.go +├── service/ +│ └── user_service.go +├── repository/ +│ └── user_repository.go +└── handler/ + └── user_handler.go +``` + +**Review Finding:** +- 🔴 **Design Debt**: Horizontal layering instead of vertical slices +- Impact: User feature changes require touching 4 different directories + +### After (No Debt) +``` +project/ +└── user/ + ├── user.go # Domain type + ├── service.go # Business logic + ├── repository.go # Persistence + ├── handler.go # HTTP + └── user_test.go +``` + +**Benefits:** +- All user-related code in one place +- Easy to understand complete feature +- Independent testing/deployment + +--- + +## Example 4: Generic Naming + +### Before (Readability Debt 🟡) +```go +package common + +type DataManager struct { + store Storage +} + +func (m *DataManager) ProcessData(data interface{}) (interface{}, error) { + // ... +} + +func HandleRequest(ctx context.Context, data map[string]interface{}) error { + // ... +} +``` + +**Review Findings:** +- 🟡 **Readability Debt**: Generic package name (`common`) +- 🟡 **Readability Debt**: Vague type name (`DataManager`) +- 🟡 **Readability Debt**: Meaningless function names (`ProcessData`, `HandleRequest`) + +### After (No Debt) +```go +package user + +type Service struct { + repo Repository +} + +func (s *Service) Create(ctx context.Context, u User) error { + // ... +} + +func (s *Service) Authenticate(ctx context.Context, credentials Credentials) (Token, error) { + // ... +} +``` + +--- + +## Example 5: Testing Anti-Patterns + +### Before (Design Debt 🔴) +```go +package user // Same package + +// Testing private function +func TestValidateEmailInternal(t *testing.T) { + assert.True(t, validateEmailInternal("test@example.com")) +} + +// Heavy mocking +func TestCreateUser(t *testing.T) { + mockRepo := &MockRepository{} + mockEmailer := &MockEmailer{} + + mockRepo.On("Save", mock.Anything).Return(nil) + mockEmailer.On("Send", mock.Anything).Return(nil) + + svc := &UserService{ + Repo: mockRepo, + Emailer: mockEmailer, + } + + err := svc.CreateUser("123", "test@example.com") + assert.NoError(t, err) + + mockRepo.AssertExpectations(t) +} + +// Flaky with time.Sleep +func TestAsyncOperation(t *testing.T) { + go doAsyncWork() + time.Sleep(100 * time.Millisecond) // ❌ Flaky + assert.True(t, workCompleted) +} +``` + +**Review Findings:** +- 🔴 **Design Debt**: Testing private methods +- 🔴 **Design Debt**: Using mocks instead of real implementations +- 🔴 **Design Debt**: Flaky test with time.Sleep + +### After (No Debt) +```go +package user_test // External package + +// Test public API only +func TestService_CreateUser(t *testing.T) { + // Use real implementations + repo := user.NewInMemoryRepository() + emailer := user.NewTestEmailer() + + svc, err := user.NewUserService(repo, emailer) + require.NoError(t, err) + + id, _ := user.NewUserID("123") + email, _ := user.NewEmail("test@example.com") + + u := user.User{ID: id, Email: email} + err = svc.CreateUser(context.Background(), u) + + assert.NoError(t, err) + + // Verify via public API + retrieved, err := svc.GetUser(context.Background(), id) + assert.NoError(t, err) + assert.Equal(t, email, retrieved.Email) +} + +// No flakiness with channels +func TestAsyncOperation(t *testing.T) { + done := make(chan struct{}) + + go func() { + doAsyncWork() + close(done) + }() + + select { + case <-done: + // Success + case <-time.After(1 * time.Second): + t.Fatal("timeout waiting for async work") + } +} +``` + +--- + +## Example 6: Complete Commit Review Output + +### Scenario +Developer adds user authentication feature with some design issues. + +### Review Output +``` +📋 COMMIT READINESS SUMMARY + +✅ Linter: Passed (0 issues) +✅ Tests: 87% coverage (5 new types, 23 test cases) +⚠️ Design Review: 5 findings (see below) + +🎯 COMMIT SCOPE +Modified: +- user/service.go (+120, -30 lines) +- user/auth.go (new file, +85 lines) + +Added: +- user/user_id.go (new type: UserID) +- user/password.go (new type: Password) + +Tests: +- user/service_test.go (+95 lines) +- user/auth_test.go (new, +140 lines) + +⚠️ DESIGN REVIEW FINDINGS + +🔴 DESIGN DEBT (Recommended to fix): + +1. user/service.go:67 - Primitive obsession on session token + Current: func CreateSession(userID UserID) (string, error) + Better: func CreateSession(userID UserID) (SessionToken, error) + Why: Session tokens should be validated types to prevent empty/invalid tokens + Fix: Use @code-designing to create SessionToken type with validation + +2. user/auth.go:34 - Non-self-validating type + Current: + type Authenticator struct { + HashCost int // Could be invalid + } + Better: + func NewAuthenticator(hashCost int) (*Authenticator, error) { + if hashCost < 4 || hashCost > 31 { + return nil, errors.New("invalid hash cost") + } + // ... + } + Why: Constructor should validate, methods shouldn't need defensive checks + Fix: Use @code-designing to add validating constructor + +🟡 READABILITY DEBT (Consider fixing): + +3. user/auth.go:89 - Mixed abstraction levels in Authenticate() + Function mixes high-level auth flow with low-level bcrypt details + Why: Harder to understand auth logic at a glance + Fix: Use @refactoring to extract password comparison to separate function + +4. user/service.go:45 - Function could be storified better + Current: validateAndCreateUser() does validation + creation in one function + Better: Split into validateUser() and createUser() for clarity + Why: Single responsibility, easier to test each part + Fix: Use @refactoring to split responsibilities + +🟢 POLISH OPPORTUNITIES: + +5. user/auth.go:12 - Less idiomatic naming + Current: ComparePasswordWithHash + Better: PasswordMatches + Why: More concise, Go-style naming + +📝 BROADER CONTEXT: +While reviewing user/service.go, noticed email is still stored as string type +(line 23). Consider refactoring to use Email type consistently across the file +for better type safety (similar to UserID change in this commit). + +💡 SUGGESTED COMMIT MESSAGE +Add user authentication with self-validating types + +- Introduce UserID and Password self-validating types +- Implement Authenticator with bcrypt password hashing +- Add CreateSession and Authenticate methods +- Achieve 87% test coverage with real bcrypt testing + +Follows primitive obsession principles with type-safe IDs and passwords. + +──────────────────────────────────────── + +Would you like to: +1. Commit as-is (5 design findings remain) +2. Fix design debt only (🔴 items 1-2), then commit +3. Fix design + readability debt (🔴🟡 items 1-4), then commit +4. Fix all findings including polish (🔴🟡🟢 all items), then commit +5. Expand scope to refactor email type throughout file, then commit +``` diff --git a/skills/pre-commit-review/reference.md b/skills/pre-commit-review/reference.md new file mode 100644 index 0000000..3b3c408 --- /dev/null +++ b/skills/pre-commit-review/reference.md @@ -0,0 +1,1225 @@ +# Design Principles Checklist + +Complete validation guide with debt-based categorization. + +## How to Use This Reference + +This checklist is applied by the pre-commit-review skill using LLM reasoning to analyze code: + +### Application Process +1. For each file under review, systematically apply all 8 categories below +2. For each detected issue, generate a finding with: + - **Category**: Bug, Design Debt, Readability Debt, or Polish + - **Location**: file:line with specific line numbers + - **Issue**: Description with relevant code snippet + - **Better**: Improved pattern with example code + - **Why**: Impact explanation (maintenance, bugs, productivity) + - **Fix**: Recommended approach (which skill, which pattern) + - **Effort**: Time estimate for fixing + +### Detection Strategy + +**LLM analyzes code by asking questions for each principle:** +- Does this code violate a design principle? → Flag it +- How severe is the impact? → Categorize (Bug > Design > Readability > Polish) +- What's the better pattern? → Provide example +- How much effort to fix? → Estimate time + +**Tools used during detection:** +- **Read tool**: Get file contents for analysis +- **Grep tool**: Find usage patterns, count occurrences, detect duplication across codebase +- **LLM reasoning**: Pattern match anti-patterns, apply heuristics, calculate scores + +### Juiciness Scoring (for Primitive Obsession) + +When detecting potential types, calculate juiciness score: + +**Behavioral (rich behavior):** +- Complex validation (regex, ranges, business rules): +3 +- Multiple meaningful methods (≥2): +2 +- State transitions/transformations: +2 +- Format conversions: +1 + +**Structural (organizing complexity):** +- Parsing unstructured data into fields: +3 +- Grouping related data that travels together: +2 +- Making implicit structure explicit: +2 +- Replacing map[string]interface{}: +2 + +**Usage (simplifies code):** +- Used in 5+ places: +2 +- Used in 3-4 places: +1 +- Significantly simplifies calling code: +1 +- Makes tests cleaner: +1 + +**Scoring:** +- Score ≥4: HIGH priority (clear win, recommend creating type) +- Score 2-3: MEDIUM priority (judgment call, present to user) +- Score 0-1: LOW priority (don't create type, over-engineering) + +## 1. Primitive Obsession [Design Debt 🔴] + +### Detection +Look for: +- [ ] String types representing domain concepts (userID, email, path) +- [ ] Int types representing domain values (Port, Age, StatusCode) +- [ ] Float types representing domain measurements (Price, Distance) +- [ ] Primitive parameters without validation +- [ ] Logic operating directly on primitives + +### Examples + +#### ❌ Design Debt +```go +func CompleteTask(id string) error { + if id == "" { + return ErrInvalidTaskID + } + // continue with logic... + return nil +} + +func CreateUser(id string, email string, age int) error { + if id == "" { + return errors.New("id required") + } + if !strings.Contains(email, "@") { + return errors.New("invalid email") + } + if age < 0 || age > 150 { + return errors.New("invalid age") + } + // ... business logic +} +``` + +Problems: +- Validation scattered across codebase +- No compile-time guarantees +- Easy to pass invalid values +- Harder to change validation rules + +#### ✅ No Debt +```go +type TaskID string + +func NewTaskID(s string) (TaskID, error) { + if s == "" { + return "", ErrInvalidTaskID + } + return TaskID(s), nil +} + +func (s *TaskService) CompleteTask(id TaskID) error { + // logic using validated TaskID - no validation needed + return nil +} + +// More comprehensive example +type UserID string +type Email string +type Age int + +func NewUserID(s string) (UserID, error) { + if s == "" { + return "", errors.New("id required") + } + return UserID(s), nil +} + +func NewEmail(s string) (Email, error) { + if !strings.Contains(s, "@") { + return "", errors.New("invalid email") + } + return Email(s), nil +} + +func NewAge(i int) (Age, error) { + if i < 0 || i > 150 { + return 0, errors.New("invalid age") + } + return Age(i), nil +} + +func CreateUser(id UserID, email Email, age Age) error { + // No validation needed - types guarantee validity + // ... business logic only +} +``` + +Benefits: +- Type safety at compile time +- Validation centralized in constructors +- Self-documenting code +- Easier to refactor + +### Also Check: Enums +```go +// ❌ Design Debt +if status == "READY" + +// ✅ No Debt +type Status string +const StatusReady Status = "READY" +``` + +### Review Questions +- Can this primitive be passed invalid? → Needs type +- Is validation repeated elsewhere? → Needs type +- Does this represent a domain concept? → Needs type + +### Fix +Use @code-designing skill to create self-validating types + +--- + +## 2. Storifying [Readability Debt 🟡] + +### Detection +Look for: +- [ ] Functions mixing high-level steps with low-level details +- [ ] Implementation details obscuring business logic +- [ ] Long functions (>50 LOC) with multiple concerns +- [ ] Unclear flow/sequence of operations + +### Examples + +#### ❌ Readability Debt +```go +func createPizza(order *Order) *Pizza { + pizza := &Pizza{Base: order.Size, + Sauce: order.Sauce, + Cheese: "Mozzarella"} + + // High-level: toppings + if order.kind == "Veg" { + pizza.Toppings = vegToppings + } else if order.kind == "Meat" { + pizza.Toppings = meatToppings + } + + // Low-level: oven temperature control + oven := oven.New() + if oven.Temp != cookingTemp { + for (oven.Temp < cookingTemp) { + time.Sleep(checkOvenInterval) + oven.Temp = getOvenTemp(oven) + } + } + + // Low-level: baking mechanics + if !pizza.Baked { + oven.Insert(pizza) + time.Sleep(cookTime) + oven.Remove(pizza) + pizza.Baked = true + } + + // High-level: boxing + box := box.New() + pizza.Boxed = box.PutIn(pizza) + pizza.Sliced = box.SlicePizza(order.Size) + pizza.Ready = box.Close() + return pizza +} +``` + +Problems: +- Hard to understand flow at a glance +- Mixes abstraction levels (business + infrastructure) +- Difficult to test pieces independently +- Hard to modify one concern without affecting others + +#### ✅ No Debt +```go +func createPizza(order *Order) *Pizza { + pizza := prepare(order) + bake(pizza) + box(pizza) + return pizza +} + +func prepare(order *Order) *Pizza { + pizza := &Pizza{Base: order.Size, + Sauce: order.Sauce, + Cheese: "Mozzarella"} + addToppings(pizza, order.kind) + return pizza +} + +func addToppings(pizza *Pizza, kind string) { + if kind == "Veg" { + pizza.Toppings = vegToppings + } else if kind == "Meat" { + pizza.Toppings = meatToppings + } +} + +func bake(pizza *Pizza) { + oven := oven.New() + heatOven(oven) + bakePizza(pizza, oven) +} + +func heatOven(oven *Oven) { /* ... */ } +func bakePizza(pizza *Pizza, oven *Oven) { /* ... */ } +func box(pizza *Pizza) { /* ... */ } +``` + +Benefits: +- Reads like a story (prepare → bake → box) +- Each function single abstraction level +- Easy to test each step independently +- Clear where to make changes + +### Principle +**Top-level functions should read like a story, not implementation** +- All steps clear and easy to understand at a glance +- Hide nitty-gritty details behind methods with proper names + +### Review Questions +- Does this function read like steps or implementation? → Story = good +- Are there multiple abstraction levels? → Extract helpers +- Could I explain this flow in 3-5 steps? → Should match code structure + +### Fix +Use @refactoring skill to extract functions and clarify abstraction levels + +--- + +## 3. Self-Validating Types [Design Debt 🔴] + +### Detection +Look for: +- [ ] Structs with public fields that need validation +- [ ] Methods checking if fields are nil/empty/invalid +- [ ] Validation happening outside constructors +- [ ] Defensive programming inside methods + +### Examples + +#### ❌ Design Debt +```go +type UserService struct { + Repo Repository // Public, might be nil + EmailSender EmailSender // Public, might be nil +} + +func (s *UserService) CreateUser(ctx context.Context, user User) error { + // Defensive checks in every method + if s.Repo == nil { + return errors.New("repo is nil") + } + if s.EmailSender == nil { + return errors.New("email sender is nil") + } + + // Actual logic + return s.Repo.Save(ctx, user) +} + +func (s *UserService) GetUser(ctx context.Context, id string) (*User, error) { + // Must repeat checks in every method + if s.Repo == nil { + return nil, errors.New("repo is nil") + } + return s.Repo.Get(ctx, id) +} +``` + +Problems: +- Every method must check for nil +- Easy to forget defensive checks +- Can't trust object state +- Wastes time/code on validation + +#### ✅ No Debt +```go +type UserService struct { + repo Repository // Private + emailSender EmailSender // Private +} + +func NewUserService(repo Repository, emailSender EmailSender) (*UserService, error) { + // Validate once in constructor + if repo == nil { + return nil, errors.New("repo is required") + } + if emailSender == nil { + return nil, errors.New("email sender is required") + } + return &UserService{ + repo: repo, + emailSender: emailSender, + }, nil +} + +func (s *UserService) CreateUser(ctx context.Context, user User) error { + // No validation needed - constructor guarantees validity + return s.repo.Save(ctx, user) +} + +func (s *UserService) GetUser(ctx context.Context, id UserID) (*User, error) { + // No nil checks needed + return s.repo.Get(ctx, id) +} +``` + +Benefits: +- Constructor validates once +- Methods trust object state +- Impossible to create invalid objects +- Less defensive code + +### Principle +**Types should be self-validating:** +- Check arguments in constructor +- No need to check for nil object fields inside methods +- Avoid defensive coding + +### Review Questions +- Do methods check field validity? → Move to constructor +- Are fields public when they shouldn't be? → Make private +- Can this object be invalid after construction? → Add validation + +### Fix +Use @code-designing skill to add validating constructors + +--- + +## 4. Abstraction Levels [Readability Debt 🟡] + +### Detection +Look for: +- [ ] Business logic mixed with infrastructure code +- [ ] High-level concepts mixed with low-level operations +- [ ] Function doing "what" AND "how" simultaneously +- [ ] Different conceptual levels in same function + +### Examples + +#### ❌ Readability Debt +```go +func ProcessOrder(order Order) error { + // High-level: validation + if order.ID == "" { + return errors.New("invalid order") + } + for _, item := range order.Items { + if item.Price < 0 { + return errors.New("invalid price") + } + } + + // Low-level: database connection + db, err := sql.Open("postgres", os.Getenv("DB_URL")) + if err != nil { + return fmt.Errorf("db connection: %w", err) + } + defer db.Close() + + // Mixed: transaction handling + tx, err := db.Begin() + if err != nil { + return err + } + + // Low-level: SQL query construction + query := "INSERT INTO orders (id, total) VALUES ($1, $2)" + // ... many more lines of SQL/DB logic + + // High-level: notification + if err := sendEmail(order.CustomerEmail, "Order confirmed"); err != nil { + return err + } + + return nil +} +``` + +Problems: +- Hard to understand flow at a glance +- Mixes business logic with infrastructure +- Difficult to test independently +- Hard to change one concern without affecting others + +#### ✅ No Debt +```go +func ProcessOrder(order Order) error { + if err := validateOrder(order); err != nil { + return err + } + + if err := saveOrder(order); err != nil { + return err + } + + if err := notifyCustomer(order); err != nil { + return err + } + + return nil +} + +func validateOrder(order Order) error { + // Validation logic only +} + +func saveOrder(order Order) error { + // Database logic only +} + +func notifyCustomer(order Order) error { + // Notification logic only +} +``` + +Benefits: +- Reads like a story (validate → save → notify) +- Each function single abstraction level +- Easy to test each step +- Clear separation of concerns + +### Principle +**A function should operate at a single conceptual level** +- Don't mix low-level implementation with high-level business logic +- Don't mix business logic with infrastructure + +### Review Questions +- Does this mix business and infrastructure? → Separate +- Are there different conceptual levels? → Extract layers +- Is the "what" clear or buried in "how"? → Clarify + +### Fix +Use @refactoring skill to separate abstraction layers + +--- + +## 5. Vertical Slice Architecture [Design Debt 🔴 - ADVISORY] + +### Detection +Look for: +- [ ] Features split across domain/, services/, handlers/ directories +- [ ] Horizontal layering vs vertical slicing + +**Note**: This is Design Debt but ADVISORY only. Never blocks. User may have valid reasons (time, team decisions). + +### Examples + +#### ⚠️ Horizontal Layering +``` +internal/{handlers,services,domain}/feature.go +``` +Problems: Feature scattered, coupling, team conflicts + +#### ✅ Vertical Slicing +``` +internal/feature/{handler,service,repository,models}.go +``` +Benefits: Colocated, easy to understand, parallel work + +### Advisory Messages + +**Horizontal pattern**: +``` +🔴 Design Debt (Advisory): Horizontal Layering +Vertical slicing preferred for: cohesion, maintainability, boundaries +Consider: Start migration with docs/architecture/vertical-slice-migration.md +Valid reasons to proceed: time constraints, team agreement +Proceed or refactor? +``` + +**Mixed without docs**: +``` +💡 Polish: Document migration in docs/architecture/vertical-slice-migration.md +Helps team understand pattern and track progress. +``` + +**Vertical slice**: +``` +✅ Architecture: Vertical Slice Pattern +Follows recommended pattern, feature colocated +``` + +### Fix +If user wants refactor: Use @code-designing skill + +--- + +## 6. Naming [Readability Debt 🟡 or Polish 🟢] + +### Detection +Look for: +- [ ] Generic names: utils, common, helpers, manager, handler (without context) +- [ ] Redundant names: UserService.CreateUserAccount +- [ ] Non-idiomatic names: getUserData vs GetUser +- [ ] Colliding names with stdlib or common libraries + +### Examples + +#### 🟡 Readability Debt (Generic/Vague) +```go +package common // Too generic + +type DataManager struct { // Vague + // ... +} + +func ProcessData(data interface{}) interface{} { // No meaning + // ... +} +``` + +#### ✅ Better +```go +package user + +type Service struct { // Context from package + // ... +} + +func (s *Service) Create(u User) error { // Clear action + // ... +} +``` + +#### 🟢 Polish Opportunity (Less Idiomatic) +```go +// Less idiomatic +func (s *Service) CreateUserInDatabase(user User) error + +// More idiomatic +func (s *Service) Create(user User) error // Receiver provides context +``` + +### Principles +- **Write idiomatic Go code** +- **Use flatcase for package names** (e.g., `wekatrace`) +- **Ergonomic naming**: `version.Info` better than `version.VersionInfo` +- **Avoid generic names**: data, utils, common, domain +- **Avoid stdlib collisions**: Don't use `metrics` (collides with libs), use `wekametrics` + +### Review Questions +🟡 Readability: +- Is the name generic/vague? → Make specific +- Does it collide with stdlib? → Choose unique name + +🟢 Polish: +- Is it idiomatic? → Minor naming improvements +- Is it ergonomic? → Reduce redundancy + +### Fix +🟡 Readability: Use @refactoring skill +🟢 Polish: Minor renaming + +--- + +## 7. Testing Approach [Design Debt 🔴] + +### Detection +Look for: +- [ ] Tests in same package (not pkg_test) +- [ ] Testing private methods/functions +- [ ] Heavy use of mocks instead of real implementations +- [ ] Tests with cyclomatic complexity > 1 (conditionals in tests) +- [ ] time.Sleep in tests + +### Examples + +#### ❌ Design Debt +```go +package user // Same package - can test private + +func TestInternalValidation(t *testing.T) { // Testing private + result := validateEmailInternal("test@example.com") + assert.True(t, result) +} + +func TestServiceWithMocks(t *testing.T) { + mockRepo := &MockRepository{} // Heavy mocking + mockEmailer := &MockEmailer{} + + mockRepo.On("Save", mock.Anything).Return(nil) + mockEmailer.On("Send", mock.Anything).Return(nil) + + svc := &UserService{Repo: mockRepo, EmailSender: mockEmailer} + // Test with mocks +} + +func TestWithSleep(t *testing.T) { + go doWork() + time.Sleep(100 * time.Millisecond) // ❌ Flaky + // assert +} +``` + +#### ✅ No Debt +```go +package user_test // External package - tests public API only + +func TestService_CreateUser(t *testing.T) { // Test public API + // Use real implementations + repo := user.NewInMemoryRepository() + emailer := user.NewTestEmailer() + + svc, err := user.NewUserService(repo, emailer) + require.NoError(t, err) + + // Test public behavior + err = svc.CreateUser(context.Background(), testUser) + assert.NoError(t, err) +} + +func TestWithChannel(t *testing.T) { + done := make(chan struct{}) + go func() { + doWork() + close(done) + }() + + select { + case <-done: + // Success + case <-time.After(1 * time.Second): + t.Fatal("timeout") + } +} +``` + +### Test Quality Review + +Beyond structure, review if tests actually test the system properly. + +#### Detection: Does Test Actually Test the SUT? + +Look for: +- [ ] **Weak assertions**: Tests that pass but don't verify behavior +- [ ] **Missing use cases**: Important scenarios not covered +- [ ] **Mock overuse**: Mocking prevents testing real behavior +- [ ] **Test isolation**: Tests depend on each other or shared state +- [ ] **Incomplete verification**: Only checking happy path +- [ ] **Conditionals in tests**: wantErr bool pattern (violates complexity = 1) + +#### ❌ Poor Test Quality + +**Example 1: Weak Assertion** +```go +func TestCreateUser(t *testing.T) { + svc := setupService() + err := svc.CreateUser(ctx, user) + + assert.NoError(t, err) // Only checks no error + // ❌ Doesn't verify user was actually created! + // ❌ Doesn't check user in database + // ❌ Doesn't verify email was sent +} +``` + +**Example 2: Mock Prevents Real Testing** +```go +func TestDataProcessor(t *testing.T) { + mockDB := &MockDatabase{} + mockDB.On("Query", "SELECT...").Return(mockData, nil) + + processor := NewProcessor(mockDB) + result := processor.Process() + + assert.Equal(t, expected, result) + // ❌ Never tests real database interaction + // ❌ Can't catch SQL syntax errors + // ❌ Can't catch data marshaling issues +} +``` + +**Example 3: Missing Important Use Cases** +```go +func TestParseEmail(t *testing.T) { + email, err := ParseEmail("test@example.com") + assert.NoError(t, err) + assert.Equal(t, "test@example.com", email.String()) + + // ❌ Only tests happy path + // ❌ Missing: empty string, invalid format, edge cases +} +``` + +**Example 4: Conditionals in Tests (wantErr anti-pattern)** +```go +func TestParseEmail(t *testing.T) { + tests := []struct { + name string + input string + want Email + wantErr bool // ❌ Anti-pattern + }{ + {name: "valid", input: "test@example.com", want: Email("test@example.com")}, + {name: "empty", input: "", wantErr: true}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := ParseEmail(tt.input) + + if tt.wantErr { // ❌ Conditional in test (complexity > 1) + assert.Error(t, err) + return + } + + assert.NoError(t, err) + assert.Equal(t, tt.want, got) + }) + } +} +``` + +#### ✅ Good Test Quality + +**Example 1: Complete Verification** +```go +func TestCreateUser(t *testing.T) { + // Use real implementations + db := setupTestDB(t) + emailer := &TestEmailer{sent: []Email{}} + + svc := NewUserService(db, emailer) + user := User{Email: "test@example.com", Name: "Test"} + + err := svc.CreateUser(ctx, user) + require.NoError(t, err) + + // ✅ Verify user in database + saved, err := db.GetUser(ctx, user.ID) + require.NoError(t, err) + assert.Equal(t, user.Email, saved.Email) + assert.Equal(t, user.Name, saved.Name) + + // ✅ Verify email sent + assert.Len(t, emailer.sent, 1) + assert.Equal(t, user.Email, emailer.sent[0].To) + assert.Contains(t, emailer.sent[0].Body, "Welcome") +} +``` + +**Example 2: Test Real Database** +```go +func TestUserRepository_Save(t *testing.T) { + // ✅ Use real database (in-memory or testcontainers) + db := setupPostgresTestContainer(t) + // OR: db := setupInMemoryDB(t) + + repo := NewUserRepository(db) + user := User{Email: "test@example.com"} + + // Test real database operations + err := repo.Save(ctx, user) + require.NoError(t, err) + + // Verify by querying database directly + var count int + err = db.QueryRow("SELECT COUNT(*) FROM users WHERE email = ?", + user.Email).Scan(&count) + require.NoError(t, err) + assert.Equal(t, 1, count) +} +``` + +**Example 3: Correct Pattern (Separate Functions, Complexity = 1)** + +Instead of conditionals, use separate test functions: + +```go +// ✅ Success cases - always expect success (no conditionals) +func TestParseEmail_Success(t *testing.T) { + tests := []struct { + name string + input string + want Email + }{ + {name: "simple", input: "test@example.com", want: Email("test@example.com")}, + {name: "with plus", input: "test+tag@example.com", want: Email("test+tag@example.com")}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := ParseEmail(tt.input) + require.NoError(t, err) // No conditionals + assert.Equal(t, tt.want, got) + }) + } +} + +// ✅ Error cases - always expect error (no conditionals) +func TestParseEmail_Error(t *testing.T) { + tests := []struct { + name string + input string + }{ + {name: "empty", input: ""}, + {name: "no @", input: "testexample.com"}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + _, err := ParseEmail(tt.input) + assert.Error(t, err) // No conditionals + }) + } +} +``` + +**See [testing/reference.md](../testing/reference.md) for complete testing patterns and anti-patterns.** + +#### Test Quality Checkpoints + +When reviewing tests, check: + +**1. Real Implementation Usage** +- Database: Use in-memory DB or testcontainers (not mocks) +- Files: Use `t.TempDir()` or `os.CreateTemp()` (not mocks) +- HTTP: Use `httptest.Server` (not mocks) +- External services: Use real test instances or testcontainers +- Only mock when absolutely necessary (external APIs you don't control) + +**2. Complete Verification** +- Assert actual behavior, not just "no error" +- Verify side effects (database changes, files written, messages sent) +- Check state before and after operation + +**3. Use Case Coverage** +- ✅ Happy path + ✅ Edge cases + ✅ Error cases + +**4. Test Independence** +- Tests can run in any order +- Use `t.Cleanup()` for cleanup +- No shared mutable state + +**5. No Conditionals (Complexity = 1)** +- ❌ No `wantErr bool` with if statements +- ✅ Separate success/error test functions + +**6. Meaningful Assertions** +- Use specific assertions with messages +- Verify business logic, not implementation + +**For complete testing patterns and examples, see [testing/reference.md](../testing/reference.md)** + +--- + +### Principles + +**Test Only Public API:** +- Use `pkg_test` package name +- Test types via constructors only +- No testing private methods + +**Avoid Mocks:** +- Use real implementations (HTTP test servers, temp files, in-memory DBs) +- Test with actual dependencies (integration-style) + +**Table-Driven Tests:** +- Good when each case has cyclomatic complexity = 1 +- NO conditionals inside t.Run() +- Separate success/error cases into different test functions + +**Testify Suites:** +- ONLY for complex infrastructure setup +- NOT for simple unit tests + +**Synchronization:** +- Avoid time.Sleep +- Use wait groups or channels + +**Coverage:** +- Leaf types: 100% unit test coverage +- Orchestrating types: Integration tests + +### Review Questions + +**Structure:** +- Are tests in same package? → Use pkg_test +- Testing private methods? → Test public API instead +- Using mocks heavily? → Use real implementations +- Using time.Sleep? → Use channels/wait groups + +**Quality:** +- Does test verify actual behavior? → Add meaningful assertions +- Are important use cases covered? → Add edge cases and error cases +- Using mocks where real implementation possible? → Use testcontainers/in-memory/temp files +- Do tests verify side effects? → Check database/files/messages +- Are tests independent? → Use t.Cleanup(), avoid shared state +- Conditionals in tests (wantErr)? → Separate success and error test functions + +### Fix +Use @testing skill to restructure tests + +--- + +## 8. Design Bugs [Bug 🐛] + +### Detection +Look for: +- [ ] Potential nil dereferences +- [ ] Errors assigned to `_` (silently ignored) +- [ ] Missing defer for resource cleanup +- [ ] Race conditions (shared state without synchronization) +- [ ] Context not propagated (using context.Background() in call chain) +- [ ] Invalid nil returns (returning nil for non-error values) +- [ ] time.Sleep in production code (should use timers/contexts) +- [ ] Goroutine leaks (no way to exit) + +### Examples + +#### ❌ Bug: Potential Nil Dereference +```go +user := getUser() // Can return user with nil Profile +email := user.Profile.Email // Panic if Profile is nil +``` + +**Problems:** +- Crash risk if Profile is nil +- No defensive check +- Hard to debug in production + +#### ✅ Fixed +```go +user := getUser() +if user.Profile == nil { + return errors.New("user has no profile") +} +email := user.Profile.Email +``` + +**Better: Self-validating User type** +```go +func NewUser(..., profile Profile) (*User, error) { + if profile == nil { + return nil, errors.New("profile required") + } + return &User{Profile: profile}, nil +} + +// Now Profile is guaranteed non-nil +func (u *User) GetEmail() string { + return u.Profile.Email // Safe, no check needed +} +``` + +#### ❌ Bug: Ignored Error +```go +_ = client.Record(metric) // Silent failure +``` + +**Problems:** +- If metrics recording fails, no visibility +- Hard to debug production issues +- Violates fail-fast principle + +#### ✅ Fixed +```go +if err := client.Record(metric); err != nil { + log.Printf("failed to record metric: %v", err) +} + +// Better: Return error if it's critical +if err := client.Record(metric); err != nil { + return fmt.Errorf("record metric: %w", err) +} +``` + +#### ❌ Bug: Resource Leak +```go +func processFile(path string) error { + f, err := os.Open(path) + if err != nil { + return err + } + + data, err := io.ReadAll(f) + if err != nil { + return err // File never closed! + } + + f.Close() + return process(data) +} +``` + +**Problems:** +- Early return doesn't close file +- Resource leak +- Can exhaust file descriptors + +#### ✅ Fixed +```go +func processFile(path string) error { + f, err := os.Open(path) + if err != nil { + return err + } + defer f.Close() // Always closes, even on early return + + data, err := io.ReadAll(f) + if err != nil { + return err + } + + return process(data) +} +``` + +#### ❌ Bug: Context Not Propagated +```go +func (s *Service) CreateUser(ctx context.Context, user User) error { + // Ignoring ctx, using Background + return s.repo.Save(context.Background(), user) +} +``` + +**Problems:** +- Cancellation not respected +- Timeouts don't work +- Can't trace requests + +#### ✅ Fixed +```go +func (s *Service) CreateUser(ctx context.Context, user User) error { + return s.repo.Save(ctx, user) // Propagate context +} +``` + +#### ❌ Bug: Invalid Nil Return +```go +func FindUser(id string) *User { + // ... + return nil // Nil is not a valid value for non-error returns +} +``` + +**Problems:** +- Caller must check for nil +- Easy to forget nil check +- Violates "nil is not a valid value" principle + +#### ✅ Fixed +```go +func FindUser(id UserID) (*User, error) { + user, found := users[id] + if !found { + return nil, fmt.Errorf("user not found: %s", id) + } + return &user, nil +} +``` + +#### ❌ Bug: Goroutine Leak +```go +func startWorker() { + go func() { + for { + work := <- workChan + process(work) + // No way to exit this goroutine! + } + }() +} +``` + +**Problems:** +- Goroutine runs forever +- Memory leak +- Can't shutdown cleanly + +#### ✅ Fixed +```go +func startWorker(ctx context.Context) { + go func() { + for { + select { + case work := <- workChan: + process(work) + case <-ctx.Done(): + return // Clean exit + } + } + }() +} +``` + +### Review Questions +- Can anything panic? → Check nil flows +- Are errors handled? → No `_ = ...` +- Are resources cleaned up? → Check defer usage +- Is context propagated? → No context.Background in chains +- Can goroutines exit? → Check cancellation + +### Fix +**Fix bugs immediately before any refactoring work.** +- Nil issues → Add validation or use self-validating types +- Ignored errors → Log at minimum, return if critical +- Resource leaks → Add defer statements +- Context issues → Propagate ctx through call chain +- Goroutine leaks → Add cancellation via context + +--- + +## Review Process Summary + +For each modified file: + +1. **Run Checklist** (#1-8 above) +2. **Categorize Findings**: + - 🐛 Bugs: Nil deref, ignored errors, resource leaks (fix immediately) + - 🔴 Design Debt: Types, architecture, validation + - 🟡 Readability Debt: Abstraction, flow, naming + - 🟢 Polish: Minor improvements + +3. **Check Broader Context**: + - Similar issues in rest of file? + - Pattern worth addressing holistically? + +4. **Generate Report**: + - Specific findings with locations + - Concrete suggestions with examples + - Impact explanations (why it matters) + - Recommended skills to fix + +5. **User Decision**: + - Commit as-is + - Fix specific debt categories + - Expand scope to broader refactor + +## Additional Principles from coding_rules.md + +### Function Complexity +- Keep functions under 50 LOC +- Max 2 nesting levels +- Deeply nested if/else → Extract functions or early returns + +### Nil Handling +- Never return nil values (except for errors: `nil, err` or `val, nil` is ok) +- Never pass nil into functions +- Avoid defensive nil checks in methods (validate in constructor) + +### Defer Complexity +- If defer functions have cyclomatic complexity > 1 → Extract to separate function + +### Test Coverage Strategy +- Leaf types (not dependent on others): 100% unit test coverage +- Most logic should be in leaf types +- Orchestrating types: Integration tests covering seams + +### Linting +- Never use `nolint` directives without approval +- Try to fix code first +- If false positive, add to exclusions in `.golangci.yaml` +- Fix can be as simple as logging error instead of ignoring + +### Table-Driven Tests +**ALWAYS use named struct fields:** +```go +// ❌ BAD - breaks when linter reorders fields +{name: "test1", 42, "result"}, + +// ✅ GOOD - works regardless of field order +{name: "test1", input: 42, want: "result"}, +``` diff --git a/skills/refactoring/SKILL.md b/skills/refactoring/SKILL.md new file mode 100644 index 0000000..c46328c --- /dev/null +++ b/skills/refactoring/SKILL.md @@ -0,0 +1,759 @@ +--- +name: refactoring +description: Linter-driven refactoring patterns to reduce complexity and improve code quality. Use when linter fails with complexity issues (cyclomatic, cognitive, maintainability) or when code feels hard to read/maintain. Applies storifying, type extraction, and function extraction patterns. +--- + +# Refactoring + +Linter-driven refactoring patterns to reduce complexity and improve code quality. + +## When to Use +- **Automatically invoked** by @linter-driven-development when linter fails +- **Automatically invoked** by @pre-commit-review when design issues detected +- **Complexity failures**: cyclomatic, cognitive, maintainability index +- **Architectural failures**: noglobals, gochecknoinits, gochecknoglobals +- **Design smell failures**: dupl (duplication), goconst (magic strings), ineffassign +- Functions > 50 LOC or nesting > 2 levels +- Mixed abstraction levels in functions +- Manual invocation when code feels hard to read/maintain + +**IMPORTANT**: This skill operates autonomously - no user confirmation needed during execution + +## Learning Resources + +Choose your learning path: +- **Quick Start**: Use the patterns below for common refactoring cases +- **Complete Reference**: See [reference.md](./reference.md) for full decision tree and all patterns +- **Real-World Examples**: See [examples.md](./examples.md) to learn the refactoring thought process + - [Example 1](./examples.md#example-1-storifying-mixed-abstractions-and-extracting-logic-into-leaf-types): Storifying and extracting a single leaf type + - [Example 2](./examples.md#example-2-primitive-obsession-with-multiple-types-and-storifying-switch-statements): Primitive obsession with multiple types and switch elimination + +## Analysis Phase (Automatic) + +Before applying any refactoring patterns, the skill automatically analyzes the context: + +### System Context Analysis +``` +AUTOMATICALLY ANALYZE: +1. Find all callers of the failing function +2. Identify which flows/features depend on it +3. Determine primary responsibility +4. Check for similar functions revealing patterns +5. Spot potential refactoring opportunities +``` + +### Type Discovery +Proactively identify hidden types in the code: + +``` +POTENTIAL TYPES TO DISCOVER: +1. Data being parsed from strings → Parse* types + Example: ParseCommandResult(), ParseLogEntry() + +2. Scattered validation logic → Validated types + Example: Email, Port, IPAddress types + +3. Data that always travels together → Aggregate types + Example: UserCredentials, ServerConfig + +4. Complex conditions → State/status types + Example: DeploymentStatus with IsReady(), CanProceed() + +5. Repeated string manipulation → Types with methods + Example: FilePath with Dir(), Base(), Ext() +``` + +### Analysis Output +The analysis produces a refactoring plan identifying: +- Function's role in the system +- Potential domain types to extract +- Recommended refactoring approach +- Expected complexity reduction + +## Refactoring Signals + +### Linter Failures +**Complexity Issues:** +- **Cyclomatic Complexity**: Too many decision points → Extract functions, simplify logic +- **Cognitive Complexity**: Hard to understand → Storifying, reduce nesting +- **Maintainability Index**: Hard to maintain → Break into smaller pieces + +**Architectural Issues:** +- **noglobals/gochecknoglobals**: Global variable usage → Dependency rejection pattern +- **gochecknoinits**: Init function usage → Extract initialization logic +- **Static/singleton patterns**: Hidden dependencies → Inject dependencies + +**Design Smells:** +- **dupl**: Code duplication → Extract common logic/types +- **goconst**: Magic strings/numbers → Extract constants or types +- **ineffassign**: Ineffective assignments → Simplify logic + +### Code Smells +- Functions > 50 LOC +- Nesting > 2 levels +- Mixed abstraction levels +- Unclear flow/purpose +- Primitive obsession +- Global variable access scattered throughout code + +## Workflow (Automatic) + +### 1. Receive Linter Failures +Automatically receive failures from @linter-driven-development: +``` +user/service.go:45:1: cyclomatic complexity 15 of func `CreateUser` is high (> 10) +user/handler.go:23:1: cognitive complexity 25 of func `HandleRequest` is high (> 15) +``` + +### 2. Automatic Root Cause Analysis +The skill automatically diagnoses each failure: +- Does this code read like a story? → Apply storifying +- Can this be broken into smaller pieces? → Extract functions/types +- Does logic run on primitives? → Check for primitive obsession +- Is function long due to switch statement? → Extract case handlers + +### 3. Automatic Pattern Application +Applies patterns in priority order without user intervention: +- **Early Returns**: Try first (least invasive) +- **Extract Function**: Break up complexity +- **Storifying**: Improve abstraction levels +- **Extract Type**: Create domain types (if juicy) +- **Switch Extraction**: Categorize cases + +### 4. Automatic Verification Loop +- Re-run linter automatically +- If still failing, try next pattern +- Continue until linter passes +- Report final results + +## Automation Flow + +This skill operates completely autonomously once invoked: + +### Automatic Iteration Loop +``` +AUTOMATED PROCESS: +1. Receive trigger: + - From @linter-driven-development (linter failures) + - From @pre-commit-review (design debt/readability debt) +2. Apply refactoring pattern (start with least invasive) +3. Run linter immediately (no user confirmation) +4. If linter still fails OR review finds more issues: + - Try next pattern in priority order + - Repeat until both linter and review pass +5. If patterns exhausted and still failing: + - Report what was tried + - Suggest file splitting or architectural changes +``` + +### Pattern Priority Order +Apply patterns based on failure type: + +**For Complexity Failures** (cyclomatic, cognitive, maintainability): +``` +1. Early Returns → Reduce nesting quickly +2. Extract Function → Break up long functions +3. Storifying → Improve abstraction levels +4. Extract Type → Create domain types (only if "juicy") +5. Switch Extraction → Categorize switch cases +``` + +**For Architectural Failures** (noglobals, singletons): +``` +1. Dependency Rejection → Incremental bottom-up approach +2. Extract Type with dependency injection +3. Push global access up call chain one level +4. Iterate until globals only at entry points (main, handlers) +``` + +**For Design Smells** (dupl, goconst): +``` +1. Extract Type → For repeated values or validation +2. Extract Function → For code duplication +3. Extract Constant → For magic strings/numbers +``` + +### No Manual Intervention +- **NO** asking for confirmation between patterns +- **NO** waiting for user input +- **NO** manual linter runs +- **AUTOMATIC** progression through patterns +- **ONLY** report results at the end + +## Refactoring Patterns + +### Pattern 1: Storifying (Mixed Abstractions) +**Signal**: Function mixes high-level steps with low-level details + +```go +// ❌ Before - Mixed abstractions +func ProcessOrder(order Order) error { + // Validation + if order.ID == "" { + return errors.New("invalid") + } + + // Low-level DB setup + db, err := sql.Open("postgres", connStr) + if err != nil { return err } + defer db.Close() + + // SQL construction + query := "INSERT INTO..." + // ... many lines + + return nil +} + +// ✅ After - Story-like +func ProcessOrder(order Order) error { + if err := validateOrder(order); err != nil { + return err + } + + if err := saveToDatabase(order); err != nil { + return err + } + + return notifyCustomer(order) +} + +func validateOrder(order Order) error { /* ... */ } +func saveToDatabase(order Order) error { /* ... */ } +func notifyCustomer(order Order) error { /* ... */ } +``` + +### Pattern 2: Extract Type (Primitive Obsession) +**Signal**: Complex logic operating on primitives OR unstructured data needing organization + +#### Juiciness Test v2 - When to Create Types + +**BEHAVIORAL JUICINESS** (rich behavior): +- ✅ Complex validation rules (regex, ranges, business rules) +- ✅ Multiple meaningful methods (≥2 methods) +- ✅ State transitions or transformations +- ✅ Format conversions (different representations) + +**STRUCTURAL JUICINESS** (organizing complexity): +- ✅ Parsing unstructured data into fields +- ✅ Grouping related data that travels together +- ✅ Making implicit structure explicit +- ✅ Replacing map[string]interface{} with typed fields + +**USAGE JUICINESS** (simplifies code): +- ✅ Used in multiple places +- ✅ Significantly simplifies calling code +- ✅ Makes tests cleaner and more focused + +**Score**: Need "yes" in at least ONE category to create the type + +#### Examples of Juicy vs Non-Juicy Types + +```go +// ❌ NOT JUICY - Don't create type +func ValidateUserID(id string) error { + if id == "" { + return errors.New("empty id") + } + return nil +} +// Just use: if userID == "" + +// ✅ JUICY (Behavioral) - Complex validation +type Email string + +func ParseEmail(s string) (Email, error) { + if s == "" { + return "", errors.New("empty email") + } + if !emailRegex.MatchString(s) { + return "", errors.New("invalid format") + } + if len(s) > 255 { + return "", errors.New("too long") + } + return Email(s), nil +} + +func (e Email) Domain() string { /* extract domain */ } +func (e Email) LocalPart() string { /* extract local */ } +func (e Email) String() string { return string(e) } + +// ✅ JUICY (Structural) - Parsing complex data +type CommandResult struct { + FailedFiles []string + SuccessFiles []string + Message string + ExitCode int + Warnings []string +} + +func ParseCommandResult(output string) (CommandResult, error) { + // Parse unstructured output into structured fields + // Making implicit structure explicit +} + +// ✅ JUICY (Mixed) - Both behavior and structure +type ServiceEndpoint struct { + host string + port Port +} + +func ParseEndpoint(s string) (ServiceEndpoint, error) { + // Parse "host:port/path" format +} + +func (e ServiceEndpoint) URL() string { } +func (e ServiceEndpoint) IsSecure() bool { } +func (e ServiceEndpoint) WithPath(path string) string { } +``` + +**⚠️ Warning Signs of Over-Engineering:** +- Type with only one trivial method +- Simple validation (just empty check) +- Type that's just a wrapper without behavior +- Good variable naming would be clearer + +**→ See [Example 2](./examples.md#first-refactoring-attempt-the-over-abstraction-trap)** for complete case study. + +### Pattern 3: Extract Function (Long Functions) +**Signal**: Function > 50 LOC or multiple responsibilities + +```go +// ❌ Before - Long function +func CreateUser(data map[string]interface{}) error { + // Validation (15 lines) + // ... + + // Database operations (20 lines) + // ... + + // Email notification (10 lines) + // ... + + // Logging (5 lines) + // ... + + return nil +} + +// ✅ After - Extracted functions +func CreateUser(data map[string]interface{}) error { + user, err := validateAndParseUser(data) + if err != nil { + return err + } + + if err := saveUser(user); err != nil { + return err + } + + if err := sendWelcomeEmail(user); err != nil { + return err + } + + logUserCreation(user) + return nil +} +``` + +### Pattern 4: Early Returns (Deep Nesting) +**Signal**: Nesting > 2 levels + +```go +// ❌ Before - Deeply nested +func ProcessItem(item Item) error { + if item.IsValid() { + if item.IsReady() { + if item.HasPermission() { + // Process + return nil + } else { + return errors.New("no permission") + } + } else { + return errors.New("not ready") + } + } else { + return errors.New("invalid") + } +} + +// ✅ After - Early returns +func ProcessItem(item Item) error { + if !item.IsValid() { + return errors.New("invalid") + } + + if !item.IsReady() { + return errors.New("not ready") + } + + if !item.HasPermission() { + return errors.New("no permission") + } + + // Process + return nil +} +``` + +### Pattern 5: Switch Extraction (Long Switch) +**Signal**: Switch statement with complex cases + +```go +// ❌ Before - Long switch in one function +func HandleRequest(reqType string, data interface{}) error { + switch reqType { + case "create": + // 20 lines of creation logic + case "update": + // 20 lines of update logic + case "delete": + // 15 lines of delete logic + default: + return errors.New("unknown type") + } + return nil +} + +// ✅ After - Extracted handlers +func HandleRequest(reqType string, data interface{}) error { + switch reqType { + case "create": + return handleCreate(data) + case "update": + return handleUpdate(data) + case "delete": + return handleDelete(data) + default: + return errors.New("unknown type") + } +} + +func handleCreate(data interface{}) error { /* ... */ } +func handleUpdate(data interface{}) error { /* ... */ } +func handleDelete(data interface{}) error { /* ... */ } +``` + +### Pattern 6: Dependency Rejection (Architectural Refactoring) +**Signal**: noglobals linter fails OR global/singleton usage detected + +**Goal**: Create "islands of clean code" by incrementally pushing dependencies up the call chain + +**Strategy**: Work from bottom-up, rejecting dependencies one level at a time +- DON'T do massive refactoring all at once +- Start at deepest level (furthest from main) +- Extract clean type with dependency injected +- Push global access up one level +- Repeat until global only at entry points + +**Quick Example**: +```go +// ❌ Before - Global accessed deep in code +func PublishEvent(event Event) error { + conn, err := nats.Connect(env.Configs.NATsAddress) + // ... complex logic +} + +// ✅ After - Dependency rejected up one level +type EventPublisher struct { + natsAddress string // injected, not global +} + +func NewEventPublisher(natsAddress string) *EventPublisher { + return &EventPublisher{natsAddress: natsAddress} +} + +func (p *EventPublisher) Publish(event Event) error { + conn, err := nats.Connect(p.natsAddress) + // ... same logic, now testable +} + +// Caller pushed up (closer to main) +func SetupMessaging() *EventPublisher { + return NewEventPublisher(env.Configs.NATsAddress) // Global only here +} +``` + +**Result**: EventPublisher is now 100% testable without globals + +**Key Principles**: +- **Incremental**: One type at a time, one level at a time +- **Bottom-up**: Start at deepest code, work toward main +- **Pragmatic**: Accept globals at entry points (main, handlers) +- **Testability**: Each extracted type is an island (testable in isolation) + +**→ See [Example 3](./examples.md#example-3-dependency-rejection-pattern) for complete case study with config access patterns** + +## Refactoring Decision Tree + +When linter fails, ask these questions (see reference.md for details): + +1. **Does this read like a story?** + - No → Extract functions for different abstraction levels + +2. **Can this be broken into smaller pieces?** + - By responsibility? → Extract functions + - By task? → Extract functions + - By category? → Extract functions + +3. **Does logic run on primitives?** + - Yes → Is this primitive obsession? → Extract type + +4. **Is function long due to switch statement?** + - Yes → Extract case handlers + +5. **Are there deeply nested if/else?** + - Yes → Use early returns or extract functions + +## Testing Integration + +### Automatic Test Creation +When creating new types or extracting functions during refactoring: + +**ALWAYS invoke @testing skill** to write tests for: +- **Isolated types**: Types with injected dependencies (testable islands) +- **Value object types**: Types that may depend on other value objects but are still isolated +- **Extracted functions**: New functions created during refactoring +- **Parse functions**: Functions that transform unstructured data + +### Island of Clean Code Definition + +A type is an "island of clean code" if: +- ✅ Dependencies are explicit (injected via constructor) +- ✅ No global or static dependencies +- ✅ Can be tested in isolation +- ✅ Has 100% testable public API + +**Examples of testable islands:** +- `NATSClient` with injected `natsAddress` string (no other dependencies) +- `Email` type with validation logic (no dependencies) +- `ServiceEndpoint` that uses `Port` value object (both are testable islands) +- `OrderService` with injected `Repository` and `EventPublisher` (all testable) + +**Note**: Islands can depend on other value objects and still be isolated! + +### Workflow +``` +REFACTORING → TESTING: +1. Extract type during refactoring +2. Immediately invoke @testing skill +3. @testing skill writes appropriate tests: + - Table-driven tests for simple scenarios + - Testify suites for complex infrastructure + - Integration tests for orchestrating types +4. Verify tests pass +5. Continue refactoring +``` + +### Testing Delegation +- **Refactoring skill**: Makes code testable (creates islands) +- **@testing skill**: Writes all tests (structure, patterns, coverage) + +**→ See @testing skill for test structure, patterns, and guidelines** + +## Stopping Criteria + +### When to Stop Refactoring + +**STOP when ALL of these are met:** +``` +✅ Linter passes +✅ All functions < 50 LOC +✅ Nesting ≤ 2 levels +✅ Code reads like a story +✅ No more "juicy" abstractions to extract +``` + +### Don't Over-Refactor + +**Warning Signs of Over-Engineering:** +- Creating types with only one method +- Functions that just call another function +- More abstraction layers than necessary +- Code becomes harder to understand +- Diminishing returns on complexity reduction + +**Pragmatic Approach:** +``` +IF linter passes AND code is readable: + STOP - Don't extract more +EVEN IF you could theoretically extract more: + STOP - Avoid abstraction bloat +``` + +### Example Stopping Decision +``` +Current State: +- Function: 45 LOC (was 120) ✅ +- Complexity: 8 (was 25) ✅ +- Nesting: 2 levels (was 4) ✅ +- Created 2 juicy types (Email, PhoneNumber) ✅ + +Could extract UserID type but: +- Only validation is "not empty" ❌ +- No other methods needed ❌ +- Good naming is sufficient ❌ + +Decision: STOP HERE - Goals achieved, avoid bloat +``` + +## After Refactoring + +### Verify +- [ ] Re-run `task lintwithfix` - Should pass +- [ ] Run tests - Should still pass +- [ ] Check coverage - Should maintain or improve +- [ ] Code more readable? - Get feedback if unsure + +### May Need +- **New types created** → Use @code-designing to validate design +- **New functions added** → Ensure tests cover them +- **Major restructuring** → Consider using @pre-commit-review + +## Output Format + +``` +🔍 CONTEXT ANALYSIS + +Function: CreateUser (user/service.go:45) +Role: Core user creation orchestration +Called by: +- api/handler.go:89 (HTTP endpoint) +- cmd/user.go:34 (CLI command) +- test/fixtures.go:123 (test fixtures) + +Potential types spotted: +- Email: Complex validation logic scattered +- UserID: Generation and validation logic +- UserCreationRequest: Multiple related fields + +🔧 REFACTORING APPLIED + +✅ Patterns Successfully Applied: +1. Early Returns: Reduced nesting from 4 to 2 levels +2. Storifying: Extracted validate(), save(), notify() +3. Extract Type: Created Email and PhoneNumber types + +❌ Patterns Tried but Insufficient: +- Extract Function alone: Still too complex, needed types + +🎯 Types Created (with Juiciness Score): + +✅ Email type (JUICY - Behavioral + Usage): +- Behavioral: ParseEmail(), Domain(), LocalPart() methods +- Usage: Used in 5+ places across codebase +- Island: Testable in isolation +- → Invoke @testing skill to write tests + +✅ PhoneNumber type (JUICY - Behavioral): +- Behavioral: Parse(), Format(), CountryCode() methods +- Validation: Complex international format rules +- Island: Testable in isolation +- → Invoke @testing skill to write tests + +❌ Types Considered but Rejected (NOT JUICY): +- UserID: Only empty check, good naming sufficient +- Status: Just string constants, enum adequate + +🏗️ ARCHITECTURAL REFACTORING (if applicable) + +Trigger: noglobals linter failure + +Global Dependencies Identified: +- env.Configs.NATsAddress: Used in 12 places +- env.Configs.DBHost: Used in 8 places + +Dependency Rejection Applied: +✅ Level 1 (Bottom): Created NATSClient with injected address +✅ Level 2 (Middle): Created OrderService using clean types +⬆️ Pushed env.Configs to: main() and HTTP handlers (2 locations) + +Islands of Clean Code Created: +- messaging/nats_client.go: Ready for testing (isolated, injected deps) +- order/service.go: Ready for testing (isolated, injected deps) +→ Invoke @testing skill to write tests for these islands + +Progress: +- Before: 20 global accesses scattered throughout +- After: 2 global accesses (entry points only) +- Islands created: 2 new testable types + +📊 METRICS + +Complexity Reduction: +- Cyclomatic: 18 → 7 ✅ +- Cognitive: 25 → 8 ✅ +- LOC: 120 → 45 ✅ +- Nesting: 4 → 2 ✅ + +📝 FILES MODIFIED + +Modified: +- user/service.go (+15, -75 lines) +- user/handler.go (+5, -20 lines) + +Created (Islands of Clean Code): +- user/email.go (new, +45 lines) → Ready for @testing skill +- user/phone_number.go (new, +38 lines) → Ready for @testing skill + +Next: Invoke @testing skill to write tests for new islands + +✅ AUTOMATION COMPLETE + +Stopping Criteria Met: +✅ Linter passes (0 issues) +✅ All functions < 50 LOC +✅ Max nesting = 2 levels +✅ Code reads like a story +✅ No more juicy abstractions + +Ready for: @pre-commit-review phase +``` + +## Learning from Examples + +For real-world refactoring case studies that show the complete thought process: + +**[Example 1: Storifying Mixed Abstractions](./examples.md#example-1-storifying-mixed-abstractions-and-extracting-logic-into-leaf-types)** +- Transforms a 48-line fat function into lean orchestration + isolated type +- Shows how to extract `IPConfig` type for collection and validation logic +- Demonstrates creating testable islands of clean code + +**[Example 2: Primitive Obsession with Multiple Types](./examples.md#example-2-primitive-obsession-with-multiple-types-and-storifying-switch-statements)** +- Transforms a 60-line function into a 7-line story by extracting 4 isolated types +- Shows the Type Alias Pattern for config-friendly types +- Demonstrates eliminating switch statement duplication +- Fixed misleading function name (`validateCIDR` → `alignCIDRArgs`) + +**[Example 3: Dependency Rejection Pattern](./examples.md#example-3-dependency-rejection-pattern)** +- Incremental elimination of global config access (`env.Configs.NATsAddress`) +- Shows bottom-up approach: create clean islands one level at a time +- Demonstrates testability benefits of dependency injection +- Pragmatic stopping point: globals only at entry points + +See [examples.md](./examples.md) for complete case studies with thought process. + +## Integration with Other Skills + +### Invoked By (Automatic Triggering) +- **@linter-driven-development**: Automatically invokes when linter fails (Phase 3) +- **@pre-commit-review**: Automatically invokes when design issues detected (Phase 4) + +### Iterative Loop +``` +1. Linter fails → invoke @refactoring +2. Refactoring complete → re-run linter +3. Linter passes → invoke @pre-commit-review +4. Review finds design debt → invoke @refactoring again +5. Refactoring complete → re-run linter +6. Repeat until both linter AND review pass +``` + +### Invokes (When Needed) +- **@code-designing**: When refactoring creates new types, validate design +- **@testing**: Automatically invoked to write tests for new types/functions +- **@pre-commit-review**: Validates design quality after linting passes + +See [reference.md](./reference.md) for complete refactoring patterns and decision tree. diff --git a/skills/refactoring/code-design-ref.md b/skills/refactoring/code-design-ref.md new file mode 100644 index 0000000..be2d96e --- /dev/null +++ b/skills/refactoring/code-design-ref.md @@ -0,0 +1,147 @@ +# Type Design Subset for Refactoring + +Quick reference for type design principles when refactoring. +For complete type design guidance, see @code-designing skill. + +## When Refactoring Reveals Need for Types + +### Primitive Obsession Signal +During refactoring, if you find: +- Validation repeated across multiple functions +- Complex logic operating on primitives (string, int, float) +- Parameters passed around without type safety + +→ Create a self-validating type + +### Pattern: Self-Validating Type +```go +type TypeName underlyingType + +func NewTypeName(input underlyingType) (TypeName, error) { + // Validate + if /* invalid */ { + return zero, errors.New("why invalid") + } + return TypeName(input), nil +} + +// Add methods if behavior needed +func (t TypeName) SomeMethod() result { + // Type-specific logic +} +``` + +## Type Design Checklist + +When creating types during refactoring: + +- [ ] **Constructor validates** - Check in New* function +- [ ] **Fields are private** - Prevent invalid state +- [ ] **Methods trust validity** - No nil checks +- [ ] **Type has behavior** - Not just data container +- [ ] **Type in own file** - If it has logic + +## Examples + +### Example 1: Port Validation +```go +// Before refactoring - Validation scattered +func StartServer(port int) error { + if port <= 0 || port >= 9000 { + return errors.New("invalid port") + } + // ... +} + +func ConnectTo(host string, port int) error { + if port <= 0 || port >= 9000 { + return errors.New("invalid port") + } + // ... +} + +// After refactoring - Self-validating type +type Port int + +func NewPort(p int) (Port, error) { + if p <= 0 || p >= 9000 { + return 0, errors.New("port must be 1-8999") + } + return Port(p), nil +} + +func StartServer(port Port) error { + // No validation needed + // ... +} + +func ConnectTo(host string, port Port) error { + // No validation needed + // ... +} +``` + +### Example 2: Parser Complexity +```go +// Before refactoring - One complex Parser +type Parser struct { + // Too many responsibilities +} + +func (p *Parser) Parse(input string) (Result, error) { + // 100+ lines parsing headers, path, body, etc. +} + +// After refactoring - Separate types by role +type HeaderParser struct { /* ... */ } +type PathParser struct { /* ... */ } +type BodyParser struct { /* ... */ } + +func (p *HeaderParser) Parse(input string) (Header, error) { + // Focused logic for headers only +} + +func (p *PathParser) Parse(input string) (Path, error) { + // Focused logic for path only +} + +func (p *BodyParser) Parse(input string) (Body, error) { + // Focused logic for body only +} +``` + +## Quick Decision: Create Type or Extract Function? + +### Create Type When: +- Logic operates on a primitive +- Validation is repeated +- Type represents domain concept +- Behavior is cohesive + +### Extract Function When: +- Logic is procedural (no state needed) +- Different abstraction level +- One-time operation +- No validation required + +## Integration with Refactoring + +After creating types during refactoring: +1. Run tests - Ensure they pass +2. Run linter - Should reduce complexity +3. Consider @code-designing - Validate type design +4. Update tests - Ensure new types have 100% coverage + +## File Organization + +When creating types during refactoring: +``` +package/ +├── original.go # Original file +├── new_type.go # New type in own file (if has logic) +└── original_test.go # Tests +``` + +--- + +For complete type design principles, see @code-designing skill. diff --git a/skills/refactoring/examples.md b/skills/refactoring/examples.md new file mode 100644 index 0000000..80af493 --- /dev/null +++ b/skills/refactoring/examples.md @@ -0,0 +1,1677 @@ +# Example 1: Storifying Mixed Abstractions and Extracting Logic into Leaf Types + +This is a real-world example from a production codebase showing how to transform a complex function by extracting logic into a new leaf type. + +## Key Learning: From Fat Function to Lean Orchestration + Leaf Type +The original function contained ALL the logic. After refactoring: +- **Orchestration layer** (thin): `upsertIfaceAddrHost` - reads like a story +- **Leaf type** (juicy logic): `IPConfig` - owns IP collection, validation, testable in isolation +- **Result**: Most complexity moved to testable leaf type with 100% coverage potential + +This is a real world example from a production codebase and what the developer chose to refactor. +It is not perfect, and it could be improved further, but it demonstrates the core refactoring pattern: +## Before refactoring +```go +// upsertIfaceAddrHost sets any IP from iface or returns error if provided IP not match to the interface +func (c *Config) upsertIfaceAddrHost(iface net.Interface) error { + addr, err := iface.Addrs() + if err != nil { + return fmt.Errorf("network addr: %w", err) + } + var ( + addrIP4Added bool + addrIP6Added bool + ) + for _, a := range addr { + ipnet, ok := a.(*net.IPNet) + if !ok || !ipnet.IP.IsGlobalUnicast() { + logger.Debug().Str("addr", a.String()).Msg("Not a global unicast address") + continue + } + if ipnet.IP.To4() == nil { // validate IP6 + if addrIP6Added { // already added. skip + continue + } + if !c.parseIP6(ipnet) { + return fmt.Errorf("IP6 %q address is not valid", c.IP6) + } + logger.Debug().Str("ip6", c.IP6).Msg("set IP6") + addrIP6Added = true + continue + } + if addrIP4Added { + continue // already added. skip + } + if !c.parseIP4(ipnet) { + return fmt.Errorf("IP4 %q address is not valid", c.IP4) + } + logger.Debug().Str("ip4", c.IP6).Msg("set IP4") + addrIP4Added = true + } + + if !addrIP4Added && !addrIP6Added { + return fmt.Errorf("IP address is not valid. IP4: %q, IP6: %q", c.IP4, c.IP6) + } + + return nil +} + +func (c *Config) isIP4Set() bool { + return len(c.IP4) > 0 +} + +func (c *Config) isIP6Set() bool { + return len(c.IP6) > 0 +} + +func (c *Config) parseIP4(ipnet *net.IPNet) bool { + if c.IP4 == ipnet.IP.To4().String() { + logger.Debug().Str("addr", ipnet.IP.To4().String()).Msg("IP4 match to interface") + return true + } + if c.IP4 == anyIPv4 || c.IP4 == "" { + logger.Debug().Str("addr", ipnet.IP.To4().String()).Msg("Using interface IP for NodeIP") + // use first ip found from interface + c.IP4 = ipnet.IP.To4().String() + return true + } + return false +} + +func (c *Config) parseIP6(ipnet *net.IPNet) bool { + if c.IP6 == ipnet.IP.To16().String() { + logger.Debug().Str("addr", ipnet.IP.To16().String()).Msg("IP6 match to interface") + return true + } + if c.IP6 == anyIPv6 || c.IP6 == "" { + logger.Debug().Str("addr", ipnet.IP.To16().String()).Msg("Using interface IP for NodeIP") + // use first ip found from interface + c.IP6 = ipnet.IP.To16().String() + return true + } + return false +} +``` + +## Code Smells Identified + +The `upsertIfaceAddrHost` function suffers from: + +1. **Fat Function Anti-Pattern** - All logic crammed into one function (48 lines, complexity 12) +2. **Hidden Side Effects** - `parseIP4/parseIP6` names hide mutation +3. **Mixed Abstraction Levels** - Combines low-level iteration with high-level business logic +4. **No Leaf Types** - All logic lives in methods, nothing is extracted to testable types +5. **Flow Control Complexity** - Nested ifs, continues, boolean flags tracking state +6. **Poor Testability** - Must mock `net.Interface` to test anything + +**The Core Problem**: All the juicy logic is trapped in a complex orchestration function. We need to extract it into a leaf type. + + +## After refactoring +```go +// upsertIfaceAddrHost sets any IP from iface or returns error if provided IP not match to the interface +func (c *Config) upsertIfaceAddrHost(iface net.Interface) error { + addr, err := iface.Addrs() + if err != nil { + return fmt.Errorf("network addr: %w", err) + } + + ipConfig := collectIPConfigFrom(addr) + + if err = c.AlignIPs(ipConfig); err != nil { + return fmt.Errorf("align config IPs err: %w", err) + } + + return nil +} + +func collectIPConfigFrom([]net.Addr addresses) IPConfig { + var ipConfig IPConfig + for _, a := range addresses { + ipConfig.AddAddress(a) + } + return ipConfig +} + + +type IPConfig struct { + IP4 string + IP6 string +} + +func (c *IPConfig) AddAddress(a net.Addr) { + ipnet, ok := a.(*net.IPNet) + if !ok || !ipnet.IP.IsGlobalUnicast() { + logger.Debug().Str("addr", a.String()).Msg("Not a global unicast address") + + return + } + + if ipnet.IP.To4() != nil { + if len(c.IP4) > 0 { + return // already added + } + c.IP4 = ipnet.IP.To4().String() + } + + if ipnet.IP.To4() == nil { + if len(c.IP6) > 0 { + return // already added + } + c.IP6 = ipnet.IP.To16().String() + + return + } +} + +func (c *IPConfig) Validate() error { + if len(c.IP4) == 0 && len(c.IP6) == 0 { + return errors.New("IP addresses are not found") + } + + return nil +} + +func (c *Config) AlignIPs(ipConfig IPConfig) error { + if err := ipConfig.Validate(); err != nil { + return fmt.Errorf("ip config is not valid: %w", err) + } + + if err := c.alignIPv4(ipConfig.IP4); err != nil { + return fmt.Errorf("align IPv4 err: %w", err) + } + if err := c.alignIPv6(ipConfig.IP6); err != nil { + return fmt.Errorf("align IPv6 err: %w", err) + } + if len(c.IPv4) == 0 { + c.ExistingClusterStartedIPv4Only = false + } + if c.ExistingClusterStartedIPv4Only && len(c.IPv6) > 0 { + logger.Warn(). + Str("IPv6", c.IPv6). + Str("IPv4", c.IPv4). + Msg("existing cluster is running in IPv4 only. Dual stack is not possible.") + } + + return nil +} + +func (c *Config) alignIPv4(ip string) error { + if c.IPv4 == ip { + logger.Debug().Str("addr", ip).Msg("IP4 match to interface") + + return nil + } + if c.IPv4 == anyIPv4 || c.IPv4 == "" { + logger.Debug().Str("addr", ip).Msg("Using interface IP for NodeIP") + // use first ip found from interface + c.IPv4 = ip + + return nil + } + + return fmt.Errorf("existing IPv4 [%s] mismatch configured [%s]", ip, c.IPv4) +} + +func (c *Config) alignIPv6(ip string) error { + if c.IPv6 == ip { + logger.Debug().Str("addr", ip).Msg("IPv6 match to interface") + + return nil + } + if c.IPv6 == anyIPv6 || c.IPv6 == "" { + logger.Debug().Str("addr", ip).Msg("Using interface IP for NodeIP") + // use first ip found from interface + c.IPv6 = ip + + return nil + } + + return fmt.Errorf("existing IPv6 [%s] mismatch configured [%s]", ip, c.IPv6) +} + +``` + +## Refactoring Thought Process + +### Step 1: Identify What's Orchestration vs. Logic +The original function does 3 things: +1. **Collects** IP addresses from interface (LOGIC) +2. **Validates** them (LOGIC) +3. **Aligns** Config state with discovered IPs (orchestration + logic) + +**Decision**: Extract the collection logic into a new type + +### Step 2: Create a Leaf Type to Hold the Juicy Logic +Instead of keeping all logic in `Config` methods: +→ **Created `IPConfig` type** - a leaf type (no dependencies on other types) +→ **Moved collection logic** into `IPConfig.AddAddress()` method +→ **Moved validation logic** into `IPConfig.Validate()` method + +**Why this matters**: +- `IPConfig` is now a **leaf type** with testable logic +- Can achieve 100% unit test coverage without mocking anything +- Logic is isolated and reusable + +### Step 3: Make Orchestration Read Like a Story +`upsertIfaceAddrHost` now reads: +1. Get addresses from interface +2. Collect them into IPConfig +3. Align our config with what we collected + +No nested ifs, no continues, no boolean flags - just clear steps. + +### Step 4: Honest Naming for Side Effects +`parseIP4/parseIP6` → `alignIPv4/alignIPv6` +The word "align" signals mutation, "parse" suggested read-only. + +## Key Improvements + +### Architecture +* **Fat function became lean orchestration** - 48 lines → 12 lines in main function +* **Created leaf type `IPConfig`** - Holds all the juicy IP collection logic +* **Separated concerns** - Collection (IPConfig) vs. Alignment (Config methods) + +### Readability +* **Storified orchestration** - `upsertIfaceAddrHost` reads like: collect → align → done +* **Honest naming** - `align*` reveals side effects vs. `parse*` hiding them +* **Single level of abstraction** - Each function operates at one conceptual level + +### Testability +* **Leaf type with 100% coverage** - `IPConfig` can be fully unit tested without mocks +* **Testable in isolation**: + ```go + // Test collection logic without network code + func TestIPConfig_AddAddress(t *testing.T) { + cfg := &IPConfig{} + cfg.AddAddress(createIPv4Addr("192.168.1.1")) + assert.Equal(t, "192.168.1.1", cfg.IP4) + } + ``` +* **Integration tests for orchestration** - Test the seams between IPConfig and Config + +### Complexity Reduction +**Before**: Cognitive complexity 18, cyclomatic complexity 12 +**After**: Max complexity 6 per function + +## Refactoring Patterns Applied + +1. **Type Extraction** → Created `IPConfig` leaf type for IP collection +2. **Storifying** → Top-level reads: collect → validate → align +3. **Honest Naming** → `align*` instead of `parse*` reveals mutation +4. **Single Responsibility** → Each function does ONE thing +5. **Early Returns** → Replaced `continue` with `return` for clarity + +## The Leaf Type Strategy + +**Before**: All logic trapped in one place +``` +Config.upsertIfaceAddrHost() { + // ALL the logic here: iteration, validation, collection, alignment + // 48 lines, complexity 12, impossible to test separately +} +``` + +**After**: Logic extracted to leaf type +``` +IPConfig (LEAF TYPE - no dependencies) + ├─ AddAddress() // Collection logic (juicy!) + └─ Validate() // Validation logic (juicy!) + +Config (ORCHESTRATOR) + ├─ upsertIfaceAddrHost() // Thin story: collect → align + └─ AlignIPs() // Thin coordination +``` + +**Result**: Most of the complexity now lives in `IPConfig`, a leaf type with 100% test coverage potential. + + +# Example 2: Primitive Obsession with Multiple Types and Storifying Switch Statements + +This real-world example shows how to transform a 60-line function with nested switches and boolean flags into a 7-line story by extracting multiple leaf types. The original function was named `validateCIDR()` but actually mutated state - a classic naming smell that triggered deeper refactoring. + +## Key Learning: From Primitive Obsession to Type-Rich Design (Without Over-Abstraction!) + +**Before**: All logic operates on raw `[]string` with manual parsing and boolean flags +``` +One 60-line function + └─ Manual string parsing + switch statements + boolean flags +``` + +**After**: Multiple focused leaf types with clear responsibilities +``` +K3SArgs (Leaf Type - string slice wrapper) + ├─ ParseCIDRConfig() → returns domain model + └─ AppendCIDRDefaults() → mutation with explicit dependencies + +CIDRConfig (Leaf Type - domain model with private fields) + ├─ clusterCIDRSet (private bool - controlled mutation) + ├─ serviceCIDRSet (private bool - controlled mutation) + ├─ ClusterCIDRSet() → accessor (read-only) + ├─ ServiceCIDRSet() → accessor (read-only) + └─ AreBothSet() → reads like English + + Note: No CIDRPresence wrapper! Private fields achieve + same safety without wrapper ceremony. + +IPVersionConfig (Leaf Type - configuration) + └─ DefaultCIDRs() → value generator + +Main Function (Orchestrator - 7 lines) + └─ Story: create config → convert to type → append defaults → store back +``` + +**Result**: +- Main function reduced from 60 to 7 lines +- Most complexity lives in 3 leaf types (100% testable) +- Each type can be tested without mocking anything +- Code reads like English: "append CIDR defaults based on IP config" +- **Avoided over-abstraction**: Rejected `CIDRPresence` wrapper, used private fields instead + +## Code Smells Identified + +1. **Misleading Name** - `validateCIDR()` doesn't validate - it mutates! Should return `bool` or `error` if validating +2. **Primitive Obsession (CRITICAL)** - Operating on raw `[]string`, manual parsing everywhere, no encapsulation +3. **Mixed Abstraction Levels** - Jumps between string splitting (`strings.SplitN`) and business logic (`isClusterCIDRSet`) +4. **Boolean Flags Tracking State** - Two booleans tracking related information instead of domain type +5. **Switch Statement Duplication** - Three nearly identical switch cases (IPv4/IPv6/dual) differing only in data values +6. **Fat Function** - 60 lines doing: parse + detect + construct + mutate +7. **Hard to Test** - Must construct entire Config object, can't test parsing independently + +**The Core Problem**: All the juicy logic is trapped in string manipulation and scattered across switch cases. We need multiple leaf types to separate parsing, configuration, and value generation concerns. + +## Before Refactoring + +```go +// Original name was validateCIDR - misleading! +func (c *Config) alignCIDRArgs() { + var ( + isClusterCIDRSet bool + isServerCIDRSet bool + ) + // LOW LEVEL: String parsing + for _, arg := range c.Configuration.K3SArgs { + kv := strings.SplitN(arg, "=", 2) + if len(kv) != 2 { + continue + } + switch kv[0] { + case "--cluster-cidr": + isClusterCIDRSet = true + case "--service-cidr": + isServerCIDRSet = true + } + } + // HIGH LEVEL: Business logic + if isClusterCIDRSet && isServerCIDRSet { + return // both set, nothing to do + } + + // DUPLICATION: Same pattern repeated 3 times with different values + switch { + case c.isIP4Set() && c.isIP6Set(): + if !isClusterCIDRSet { + c.Configuration.K3SArgs = append(c.Configuration.K3SArgs, + fmt.Sprintf("--cluster-cidr=%s,%s", clusterCIDRIPv4, clusterCIDRIPv6)) + } + if !isServerCIDRSet { + c.Configuration.K3SArgs = append(c.Configuration.K3SArgs, + fmt.Sprintf("--service-cidr=%s,%s", serviceCIDRIPv4, serviceCIDRIPv6)) + } + case c.isIP4Set(): + if !isClusterCIDRSet { + c.Configuration.K3SArgs = append(c.Configuration.K3SArgs, + "--cluster-cidr="+clusterCIDRIPv4) + } + if !isServerCIDRSet { + c.Configuration.K3SArgs = append(c.Configuration.K3SArgs, + "--service-cidr="+serviceCIDRIPv4) + } + case c.isIP6Set(): + if !isClusterCIDRSet { + c.Configuration.K3SArgs = append(c.Configuration.K3SArgs, + "--cluster-cidr="+clusterCIDRIPv6) + } + if !isServerCIDRSet { + c.Configuration.K3SArgs = append(c.Configuration.K3SArgs, + "--service-cidr="+serviceCIDRIPv6) + } + } +} +``` + +## First Refactoring Attempt: The Over-Abstraction Trap + +Before showing the final solution, let's see a common mistake: **over-abstracting booleans**. + +### What We Tried (Over-Abstraction ❌) + +```go +// CIDRPresence - A wrapper that adds NO value +type CIDRPresence bool + +const ( + cidrPresent CIDRPresence = true +) + +func (p CIDRPresence) IsSet() bool { + return bool(p) // Just unwraps the bool! +} + +type CIDRConfig struct { + ClusterCIDR CIDRPresence // Wrapped bool + ServiceCIDR CIDRPresence // Wrapped bool +} + +func (c CIDRConfig) AreBothSet() bool { + return c.ClusterCIDR.IsSet() && c.ServiceCIDR.IsSet() +} +``` + +### Why This Is Over-Abstraction + +**Problems with CIDRPresence**: +1. ❌ **8 lines of code** for a trivial wrapper +2. ❌ **One method** that just unwraps: `return bool(p)` +3. ❌ **No type safety** - still just a bool underneath +4. ❌ **Not more readable** - compare: + - `config.ClusterCIDR.IsSet()` (with wrapper) + - `config.ClusterCIDRSet` (with good naming) +5. ❌ **No validation, no logic, no invariants** - pure ceremony +6. ❌ **Increases cognitive load** - one more type to understand + +**The Honest Question**: Is `config.ClusterCIDR.IsSet()` **significantly** clearer than `config.ClusterCIDRSet`? + +**Answer**: No! Good naming achieves the same clarity. + +**The Real Need**: We DO need controlled mutation (only parser should set these values), but we don't need a wrapper type to achieve it. + +### The Better Solution: Private Fields + +Instead of wrapping with `CIDRPresence`, use **private fields with accessor methods**: + +```go +// ✅ Simple, safe, clear +type CIDRConfig struct { + clusterCIDRSet bool // Private: can only be set by ParseCIDRConfig + serviceCIDRSet bool // Private: can only be set by ParseCIDRConfig +} + +// Read-only accessors +func (c CIDRConfig) ClusterCIDRSet() bool { return c.clusterCIDRSet } +func (c CIDRConfig) ServiceCIDRSet() bool { return c.serviceCIDRSet } + +func (c CIDRConfig) AreBothSet() bool { + return c.clusterCIDRSet && c.serviceCIDRSet +} +``` + +**Why This Is Better**: +- ✅ **4 lines** vs 8 lines for CIDRPresence wrapper +- ✅ **Same safety** - compiler enforces that only parser can set values +- ✅ **Same readability** - `ClusterCIDRSet()` is just as clear +- ✅ **No wrapper ceremony** - fields are what they are: bools +- ✅ **Controlled mutation** - private fields can't be set externally + +**Key Lesson**: Not every primitive needs a type. Use private fields when you need controlled mutation without wrapper overhead. + +--- + +## After Refactoring (Final Solution) + +```go +// Main function: Now a 7-line story! +func (c *Config) alignCIDRArgs() { + ipConfig := IPVersionConfig{ + IPv4Enabled: c.isIP4Set(), + IPv6Enabled: c.isIP6Set(), + } + + k3sArgs := K3SArgs(c.K3SArgs) + k3sArgs.AppendCIDRDefaults(ipConfig) + c.K3SArgs = []string(k3sArgs) +} + +// ==================== LEAF TYPE 1: K3SArgs ==================== +// K3SArgs represents K3S command-line arguments. +// Encapsulates ALL argument list operations. +// Design choice: Type alias (not struct) allows direct use in JSON configs: +// type Config struct { +// K3SArgs K3SArgs `json:"k3sArgs,omitempty"` +// } +type K3SArgs []string + +// ParseCIDRConfig extracts which CIDRs are already configured. +// This is the ONLY place where CIDR flags can be set. +func (args K3SArgs) ParseCIDRConfig() CIDRConfig { + var config CIDRConfig + + for _, arg := range args { + key, _, found := parseK3SArgument(arg) + if !found { + continue + } + + switch key { + case "--cluster-cidr": + config.clusterCIDRSet = true // ✓ Controlled mutation in parser + case "--service-cidr": + config.serviceCIDRSet = true // ✓ Controlled mutation in parser + } + } + + return config +} + +// AppendCIDRDefaults adds missing CIDR arguments based on IP configuration. +func (args *K3SArgs) AppendCIDRDefaults(ipConfig IPVersionConfig) { + existing := args.ParseCIDRConfig() + + if existing.AreBothSet() { + return // nothing to do + } + + defaults := ipConfig.DefaultCIDRs() + + if !existing.ClusterCIDRSet() { // ✓ Read-only access via method + *args = append(*args, defaults.ClusterCIDRArg()) + } + + if !existing.ServiceCIDRSet() { // ✓ Read-only access via method + *args = append(*args, defaults.ServiceCIDRArg()) + } +} + +// parseK3SArgument splits a K3S argument into key and value. +func parseK3SArgument(arg string) (key, value string, ok bool) { + parts := strings.SplitN(arg, "=", 2) + if len(parts) != 2 { + return "", "", false + } + return parts[0], parts[1], true +} + +// ==================== LEAF TYPE 2: CIDRConfig ==================== +// CIDRConfig represents which CIDR configurations are present. +// Uses private fields for controlled mutation - can only be set by ParseCIDRConfig. +type CIDRConfig struct { + clusterCIDRSet bool // Private: controlled mutation + serviceCIDRSet bool // Private: controlled mutation +} + +// ClusterCIDRSet returns true if cluster CIDR is configured. +func (c CIDRConfig) ClusterCIDRSet() bool { + return c.clusterCIDRSet +} + +// ServiceCIDRSet returns true if service CIDR is configured. +func (c CIDRConfig) ServiceCIDRSet() bool { + return c.serviceCIDRSet +} + +// AreBothSet returns true if both cluster and service CIDRs are configured. +func (c CIDRConfig) AreBothSet() bool { + return c.clusterCIDRSet && c.serviceCIDRSet +} + +// ==================== LEAF TYPE 3: IPVersionConfig ==================== +// IPVersionConfig describes which IP versions are enabled. +type IPVersionConfig struct { + IPv4Enabled bool + IPv6Enabled bool +} + +func (cfg IPVersionConfig) DefaultCIDRs() DefaultCIDRValues { + return DefaultCIDRValues{ + ipv4Enabled: cfg.IPv4Enabled, + ipv6Enabled: cfg.IPv6Enabled, + } +} + +// DefaultCIDRValues generates default CIDR arguments based on IP config. +type DefaultCIDRValues struct { + ipv4Enabled bool + ipv6Enabled bool +} + +func (d DefaultCIDRValues) ClusterCIDRArg() string { + return "--cluster-cidr=" + d.clusterCIDRValue() +} + +func (d DefaultCIDRValues) ServiceCIDRArg() string { + return "--service-cidr=" + d.serviceCIDRValue() +} + +func (d DefaultCIDRValues) clusterCIDRValue() string { + var cidrs []string + if d.ipv4Enabled { + cidrs = append(cidrs, defaultClusterCIDRIPv4) + } + if d.ipv6Enabled { + cidrs = append(cidrs, defaultClusterCIDRIPv6) + } + return strings.Join(cidrs, ",") +} + +func (d DefaultCIDRValues) serviceCIDRValue() string { + var cidrs []string + if d.ipv4Enabled { + cidrs = append(cidrs, defaultServiceCIDRIPv4) + } + if d.ipv6Enabled { + cidrs = append(cidrs, defaultServiceCIDRIPv6) + } + return strings.Join(cidrs, ",") +} +``` + +## Refactoring Thought Process + +### Step 1: Recognize Primitive Obsession - The Root Cause + +**What's happening**: Function operates on raw `[]string` with manual parsing scattered throughout +```go +// Config struct uses primitive type +type Config struct { + K3SArgs []string `json:"k3sArgs,omitempty"` // Just a slice! +} + +// Parsing logic mixed into business logic +for _, arg := range c.K3SArgs { + kv := strings.SplitN(arg, "=", 2) // String parsing + if len(kv) != 2 { continue } // Validation + switch kv[0] { ... } // Business logic +} +``` + +→ **Decision**: Extract a `K3SArgs` type alias to encapsulate argument list operations + +```go +type K3SArgs []string // Type alias, not struct + +type Config struct { + K3SArgs K3SArgs `json:"k3sArgs,omitempty"` // Now has methods! +} +``` + +**Why type alias vs struct?** +- ✅ Can use directly in JSON config structs (serializes as array) +- ✅ Can convert to/from `[]string` easily: `K3SArgs(slice)` and `[]string(k3sArgs)` +- ✅ No wrapper overhead +- ✅ Backward compatible with existing JSON configs + +**Why this matters**: +- Once you have a type, you can move ALL operations on that data into methods +- Type can be used directly as a config field with JSON tags +- Creates a testable boundary +- Methods travel with the data everywhere it's used + +### Step 2: Identify What Logic Belongs Where + +**Analysis of the original function**: +1. **Parse existing arguments** → Belongs in `K3SArgs.ParseCIDRConfig()` +2. **Track which CIDRs exist** → Needs domain type: `CIDRConfig` +3. **Determine defaults based on IP version** → Needs config type: `IPVersionConfig` +4. **Generate CIDR strings** → Needs value generator: `DefaultCIDRValues` + +→ **Decision**: Extract 4 different types, each with one responsibility + +**Why this matters**: Instead of one 60-line function, we get 4 small leaf types that are independently testable. + +### Step 3: Replace Boolean Flags with Domain Type + +**Before**: Two booleans tracking related state +```go +var isClusterCIDRSet bool +var isServerCIDRSet bool +if isClusterCIDRSet && isServerCIDRSet { return } +``` + +**After**: Domain model with query method +```go +type CIDRConfig struct { + clusterCIDRSet bool // Private fields + serviceCIDRSet bool +} + +func (c CIDRConfig) AreBothSet() bool { + return c.clusterCIDRSet && c.serviceCIDRSet +} + +if existing.AreBothSet() { return } +``` + +→ **Why this transformation matters**: +- Reads like English: "are both set?" +- Encapsulates the logic in one place +- Extensible: easy to add DNS CIDR field +- Groups related state + +### Step 3.5: Recognize Over-Abstraction (Critical Decision!) + +**Temptation**: Wrap the bool in a type +```go +// ❌ Over-abstraction! +type CIDRPresence bool +func (p CIDRPresence) IsSet() bool { return bool(p) } + +type CIDRConfig struct { + ClusterCIDR CIDRPresence + ServiceCIDR CIDRPresence +} +``` + +**Questions to ask**: +1. Does `CIDRPresence` add meaningful methods? → **NO** (just `.IsSet()` which unwraps) +2. Does it enforce invariants? → **NO** (still just a bool) +3. Does it need controlled mutation? → **YES!** (should only be set by parser) +4. Is `.ClusterCIDR.IsSet()` clearer than `.ClusterCIDRSet()`? → **NO!** + +→ **Decision**: Don't create `CIDRPresence` wrapper. Instead, use **private fields** for controlled mutation: + +```go +// ✅ Better: Private fields + accessor methods +type CIDRConfig struct { + clusterCIDRSet bool // Private: only parser can set + serviceCIDRSet bool +} + +func (c CIDRConfig) ClusterCIDRSet() bool { return c.clusterCIDRSet } +func (c CIDRConfig) ServiceCIDRSet() bool { return c.serviceCIDRSet } +``` + +**Why this matters**: +- Achieves same safety (compiler-enforced controlled mutation) +- 4 fewer lines than wrapper approach +- No ceremonial type wrapping +- Just as readable: `ClusterCIDRSet()` vs `ClusterCIDR.IsSet()` + +**Key lesson**: Not every primitive needs a type. Use private fields when you need controlled mutation without wrapper overhead. + +### Step 4: Eliminate Switch Statement Duplication + +**Problem identified**: Same pattern repeated 3 times +```go +case c.isIP4Set() && c.isIP6Set(): + if !isClusterCIDRSet { append(..., IPv4+IPv6) } + if !isServerCIDRSet { append(..., IPv4+IPv6) } +case c.isIP4Set(): + if !isClusterCIDRSet { append(..., IPv4) } + if !isServerCIDRSet { append(..., IPv4) } +case c.isIP6Set(): + // Same pattern again! +``` + +**What differs**: Only the CIDR values (IPv4 vs IPv6 vs both) + +→ **Decision**: Extract value generation into `DefaultCIDRValues` type + +**Result**: The pattern disappears entirely - replaced by: +```go +defaults := ipConfig.DefaultCIDRs() +if !existing.ClusterCIDR.IsSet() { + *args = append(*args, defaults.ClusterCIDRArg()) +} +``` + +**Why this matters**: Duplication eliminated by separating data selection from flow control. + +### Step 5: Storify the Main Function + +**Goal**: Make it read like a story at ONE abstraction level + +**Process**: +```go +// Step 1: Create configuration object (HIGH LEVEL) +ipConfig := IPVersionConfig{ + IPv4Enabled: c.isIP4Set(), + IPv6Enabled: c.isIP6Set(), +} + +// Step 2: Convert to typed wrapper (HIGH LEVEL) +k3sArgs := K3SArgs(c.K3SArgs) + +// Step 3: Apply business logic (HIGH LEVEL) +k3sArgs.AppendCIDRDefaults(ipConfig) + +// Step 4: Store result (HIGH LEVEL) +c.K3SArgs = []string(k3sArgs) +``` + +**Read it aloud**: "Create IP config, convert args to typed wrapper, append CIDR defaults, store back." + +→ **Result**: All implementation details (parsing, switching, string building) are hidden in leaf types + +## Key Improvements + +### Architecture +* **Fat function became lean orchestrator** - 60 lines → 7 lines +* **Created 3 leaf types** - Each handles one concern: + - `K3SArgs`: Argument list operations (parsing, appending) - **usable as config field** + - `CIDRConfig`: Domain model with **private fields for safety** + - `IPVersionConfig` + `DefaultCIDRValues`: CIDR value generation +* **Clear separation** - Parsing vs Detection vs Value Generation vs Orchestration +* **Type alias pattern** - `K3SArgs` as type alias enables direct use in config structs with JSON serialization +* **Avoided over-abstraction** - Rejected `CIDRPresence` wrapper, used private fields instead (4 fewer lines, same safety) + +### Readability +* **Storified main function** - Reads like: create config → convert → append → store +* **Fixed misleading name** - `validateCIDR()` → `alignCIDRArgs()` (now accurately describes mutation) +* **Query methods read like English**: + ```go + if existing.AreBothSet() { return } + if !existing.ClusterCIDR.IsSet() { /* ... */ } + ``` +* **Single abstraction level** - Main function operates entirely at HIGH level + +### Testability +* **All leaf types testable independently**: + ```go + // Test argument parsing without Config + func TestK3SArgs_ParseCIDRConfig(t *testing.T) { + args := K3SArgs{"--cluster-cidr=10.0.0.0/8", "--other-flag=value"} + config := args.ParseCIDRConfig() + assert.True(t, config.ClusterCIDR.IsSet()) + assert.False(t, config.ServiceCIDR.IsSet()) + } + + // Test CIDR value generation without network code + func TestDefaultCIDRValues_ClusterCIDRArg(t *testing.T) { + values := DefaultCIDRValues{ipv4Enabled: true, ipv6Enabled: true} + arg := values.ClusterCIDRArg() + assert.Equal(t, "--cluster-cidr=10.42.0.0/16,fd00:42::/56", arg) + } + + // Test domain logic without parsing + func TestCIDRConfig_AreBothSet(t *testing.T) { + config := CIDRConfig{ + ClusterCIDR: cidrPresent, + ServiceCIDR: cidrPresent, + } + assert.True(t, config.AreBothSet()) + } + ``` +* **No mocking needed** - Each type constructed with simple values +* **100% coverage achievable** - All logic in leaf types + +### Complexity Reduction +**Before**: +- 60 lines in one function +- Cyclomatic complexity: 12 +- Cognitive complexity: 18 +- 3 nesting levels + +**After**: +- Main function: 7 lines, complexity 1 +- Largest helper: 15 lines, complexity 4 +- Max nesting: 2 levels +- **Most complexity in leaf types** (easily testable) + +### Avoiding Over-Abstraction +* **Rejected CIDRPresence wrapper** - Recognized it added no value: + - Would be 8 lines for a trivial bool wrapper + - Only one method: `.IsSet()` that just unwraps the bool + - Not more readable than good naming + - No validation, no logic, no invariants +* **Used private fields instead** - Achieved same safety with less code: + - Compiler-enforced controlled mutation + - Only parser can set values + - 4 fewer lines than wrapper approach +* **Key decision**: Compared `config.ClusterCIDR.IsSet()` vs `config.ClusterCIDRSet()` honestly + - **Answer**: Good naming is just as clear as method call + - **Lesson**: Not every primitive needs a type + +## Refactoring Patterns Applied + +1. **Replace Primitive with Domain Type (Type Alias Pattern)** → Created `K3SArgs` type alias for `[]string` (usable in config fields) +2. **Extract Multiple Leaf Types** → Created 3 leaf types (`K3SArgs`, `CIDRConfig`, `IPVersionConfig`) instead of one complex function +3. **Storifying** → Main function reads: create config → convert → append → store (all at same abstraction level) +4. **Replace Boolean Flags with Domain Model** → `isClusterCIDRSet, isServerCIDRSet` → `CIDRConfig` with **private fields** and query methods +5. **Eliminate Switch Duplication** → Extracted value generation to `DefaultCIDRValues`, eliminated 3 duplicate cases +6. **Introduce Parameter Object** → Created `IPVersionConfig` to pass related configuration together +7. **Query Method Pattern** → `AreBothSet()`, `ClusterCIDRSet()`, `ServiceCIDRSet()` read like English questions +8. **Avoid Over-Abstraction** → Rejected `CIDRPresence` wrapper, used private fields with accessors for controlled mutation + +## The Type Extraction Strategy + +**Before**: All logic in one place +``` +Config.alignCIDRArgs() { + // 60 lines of: + // - String parsing (strings.SplitN, validation) + // - Boolean flag tracking + // - Switch statements with duplication + // - String building (fmt.Sprintf, string concatenation) + // - Slice mutation +} +``` + +**After**: Multiple focused leaf types +``` +K3SArgs (LEAF TYPE - no external dependencies) + ├─ ParseCIDRConfig() // Parsing logic (juicy!) + ├─ AppendCIDRDefaults() // Mutation logic (juicy!) + └─ parseK3SArgument() // Helper (juicy!) + +CIDRConfig (LEAF TYPE - domain model with private fields) + ├─ clusterCIDRSet (private bool) + ├─ serviceCIDRSet (private bool) + ├─ ClusterCIDRSet() // Accessor (read-only) + ├─ ServiceCIDRSet() // Accessor (read-only) + └─ AreBothSet() // Domain logic (juicy!) + + Note: No CIDRPresence wrapper! Private fields achieve + same safety with less ceremony. + +IPVersionConfig (LEAF TYPE - configuration) + └─ DefaultCIDRs() → DefaultCIDRValues + +DefaultCIDRValues (LEAF TYPE - value generator) + ├─ ClusterCIDRArg() // String building (juicy!) + ├─ ServiceCIDRArg() // String building (juicy!) + ├─ clusterCIDRValue() // IPv4/IPv6 selection (juicy!) + └─ serviceCIDRValue() // IPv4/IPv6 selection (juicy!) + +Config (ORCHESTRATOR) + └─ alignCIDRArgs() // Thin story: 7 lines +``` + +**Result**: +- Main function is 7 lines of pure orchestration +- ALL complexity moved to leaf types +- Each leaf type achieves 100% unit test coverage +- No mocking required for any test + +## Linter Metrics + +**Before**: +- Lines: 60 +- Cyclomatic complexity: 12 +- Cognitive complexity: 18 +- Functions: 1 (doing everything) +- Testable units: 1 (requires full Config) + +**After**: +- Main function: 7 lines, complexity 1 +- Total lines: ~146 (across 5 types + helpers) +- Max complexity per function: 4 +- Testable units: 9 (all independently testable) +- Leaf types: 3 (all with 100% coverage potential) + +## Abstraction Balance: Comparison Table + +| Approach | Total Lines | Types | Readability | Safety | Ceremony | Verdict | +|----------|-------------|-------|-------------|--------|----------|---------| +| **CIDRPresence wrapper** | ~150 | 6 | Good | Low | High | ❌ Over-abstraction | +| **Public bool fields** | ~142 | 5 | Good | Low | Low | ⚠️ Acceptable for small teams | +| **Private bool + accessors** | ~146 | 5 | Good | **High** | Low | ✅ **Recommended** | + +**Why Private Fields Win**: +- Only 4 extra lines vs public fields (2 accessor methods) +- 4 fewer lines than CIDRPresence wrapper +- Compiler-enforced mutation control (can only be set in `ParseCIDRConfig`) +- Same readability as public fields +- Best safety-to-complexity ratio +- No wrapper ceremony + +## Remaining Opportunities + +**What could still be improved** (and why we stopped): + +### 1. Why We Rejected CIDRPresence Wrapper ❌ + +**Could have done**: +```go +type CIDRPresence bool +func (p CIDRPresence) IsSet() bool { return bool(p) } +``` + +**Why we didn't**: +- ❌ 8 lines for a trivial bool wrapper +- ❌ Only one method that just unwraps: `return bool(p)` +- ❌ Not more readable: `config.ClusterCIDR.IsSet()` vs `config.ClusterCIDRSet()` +- ❌ No validation, no logic, no invariants +- ❌ Would add ceremony without benefit + +**What we did instead**: Private bool fields with accessor methods +- ✅ Same safety (compiler-enforced controlled mutation) +- ✅ 4 fewer lines +- ✅ No wrapper overhead +- ✅ Just as readable + +**Lesson**: **Not every primitive needs a type.** Ask: "Does this wrapper add meaningful logic or just ceremony?" + +### 2. Why We Chose Private Fields Over Public Fields + +**Could have used public fields**: +```go +type CIDRConfig struct { + ClusterCIDRSet bool // Public + ServiceCIDRSet bool // Public +} +``` + +**Why we used private fields**: +- ✅ Compiler enforces that only `ParseCIDRConfig` can set values +- ✅ Single source of truth for where values come from +- ✅ Easy to debug: only one place to check +- ✅ Only 4 extra lines (2 accessor methods) +- ✅ Public fields would work for small, disciplined teams, but private fields are safer + +**Lesson**: **Use private fields when mutation should be controlled.** Only 4 lines for compile-time safety. + +### 3. DefaultCIDRValues Has Similar Methods + +**Could extract**: +- `clusterCIDRValue()` and `serviceCIDRValue()` are similar +- Could extract common pattern with constants as parameters + +**Why we stopped**: +- Only 2 cases - extraction would be premature abstraction +- Current code is clear and straightforward +- YAGNI principle applies + +### 4. K3SArgs Could Support More Operations + +**Could add**: +- `Remove()`, `Update()`, `HasFlag()` methods + +**Why we stopped**: +- YAGNI - only need parsing and appending for now +- Add methods when you need them, not before + +### 5. IPVersionConfig Is Just Two Bools + +**Could use enum**: +```go +type IPVersion int +const ( + IPv4Only IPVersion = iota + IPv6Only + DualStack +) +``` + +**Why we stopped**: +- Two bools are clear and simple enough +- Enum would add complexity without clarity benefit +- Current code is self-documenting + +### 6. Why Type Alias Over Struct for K3SArgs + +```go +// ❌ Struct would require unwrapping for JSON +type K3SArgs struct { + args []string +} +type Config struct { + K3SArgs K3SArgs // JSON: {"k3sArgs": {"args": [...]}} +} + +// ✅ Type alias works directly +type K3SArgs []string +type Config struct { + K3SArgs K3SArgs `json:"k3sArgs,omitempty"` // JSON: {"k3sArgs": [...]} +} +``` + +--- + +## Key Lessons: When to Stop Refactoring + +**Good refactoring knows when to stop.** We achieved our goals: +- ✅ Main function reads like a story (7 lines) +- ✅ All logic extracted to testable leaf types +- ✅ No primitive obsession (created `K3SArgs` with real behavior) +- ✅ **Avoided over-abstraction** (rejected `CIDRPresence` wrapper) +- ✅ Switch duplication eliminated +- ✅ Complexity under control +- ✅ Controlled mutation via private fields +- ✅ Type alias pattern enables clean JSON serialization + +**The Balance**: +``` +Too Simple Sweet Spot Over-Engineering + | | | +Raw primitives Domain types Types for everything +[]string K3SArgs CIDRPresence wrapper +bool flags CIDRConfig Every bool wrapped + (private fields) +``` + +**Critical Questions Before Creating a Type**: +1. Does it have >1 meaningful method with logic? (Not just unwrapping) +2. Does it enforce invariants or validation? +3. Does it need controlled mutation? (Use private fields, not wrappers) +4. Is the method call **significantly** clearer than good naming? +5. Does it hide complex implementation? + +**If answers are mostly NO** → Use primitives with good naming (or private fields for safety) + +Further refactoring would be over-engineering at this point. + +# Example 3: Dependency Rejection Pattern - Incremental Global Elimination + +This real-world example shows how to handle `noglobals` linter failures by incrementally pushing global dependencies up the call chain, creating "islands of clean code" that are 100% testable. + +## Key Learning: From Global Chaos to Clean Islands + +**The Problem**: `env.Configs.NATsAddress` accessed throughout codebase (20+ locations) +**The Solution**: Reject dependency up one level at a time (bottom-up approach) +**The Result**: Clean, testable types with globals only at entry points (2 locations) + +This pattern is DIFFERENT from other refactorings: +- **NOT a one-time fix** - it's an incremental journey +- **Start at the bottom** (leaf code) +- **Create one clean island at a time** +- **Slowly push globals toward main()** +- **Pragmatic endpoint** - accept globals at top level + +## Why Not Just Refactor Logger? + +**Important**: Some globals are designed to be global (like `slog.Logger`, `zerolog`). Those are fine! + +**Problem globals**: Configuration access like: +- `env.Configs.NATsAddress` +- `env.Configs.DBHost` +- `env.Configs.RedisURL` +- Any `env.Configs.*` scattered throughout code + +**Why these are problems**: +- Makes code untestable (need to set global state) +- Creates hidden dependencies +- Impossible to run tests in parallel +- Can't swap implementations +- Tight coupling to global config struct + +## Before Refactoring: Global Chaos + +```go +// Global config accessed everywhere +package env + +var Configs struct { + NATsAddress string + DBHost string + RedisURL string +} + +// ❌ Deep in the messaging code - globals everywhere +package messaging + +func PublishEvent(event Event) error { + // Global accessed here + conn, err := nats.Connect(env.Configs.NATsAddress) + if err != nil { + return fmt.Errorf("connect failed: %w", err) + } + defer conn.Close() + + data, err := json.Marshal(event) + if err != nil { + return err + } + + return conn.Publish(event.Topic, data) +} + +func PublishBatch(events []Event) error { + // Global accessed here too + conn, err := nats.Connect(env.Configs.NATsAddress) + if err != nil { + return fmt.Errorf("connect failed: %w", err) + } + defer conn.Close() + + for _, event := range events { + // ... publishing logic + } + return nil +} + +// ❌ In the order service - more global access +package order + +func ProcessOrder(orderID string) error { + // More globals + db := connectDB(env.Configs.DBHost) + defer db.Close() + + // Even more globals + err := messaging.PublishEvent(orderCreatedEvent) + // ... +} + +// ❌ Testing is a nightmare +func TestPublishEvent(t *testing.T) { + // Must set global state! + env.Configs.NATsAddress = "nats://test:4222" + + // Tests can't run in parallel + // Can't easily test with different addresses + // Global state leaks between tests +} +``` + +**Problems Identified**: +- `env.Configs.NATsAddress` used in 12 places +- `env.Configs.DBHost` used in 8 places +- Testing requires global state mutation +- Parallel tests impossible +- Hidden dependencies everywhere + +## Step-by-Step Refactoring + +### Step 1: Analyze Dependency Chain + +``` +DEPENDENCY MAP: +main() + └─ HTTP handlers (entry points) + ├─ OrderService.ProcessOrder() [USES env.Configs.DBHost] + │ └─ messaging.PublishEvent() [USES env.Configs.NATsAddress] + │ └─ messaging.PublishBatch() [USES env.Configs.NATsAddress] + └─ UserService.CreateUser() + └─ messaging.PublishEvent() [USES env.Configs.NATsAddress] + +DEEPEST USAGE: messaging.PublishEvent/PublishBatch (furthest from main) +START HERE: Extract messaging types first +``` + +### Step 2: Create First Clean Island (Bottom Level) + +**Extract NATSClient - the deepest leaf**: + +```go +// ✅ Clean type with injected dependency +type NATSClient struct { + natsAddress string // Injected, not global! +} + +func NewNATSClient(natsAddress string) *NATSClient { + return &NATSClient{natsAddress: natsAddress} +} + +func (c *NATSClient) PublishEvent(event Event) error { + // Uses injected value, not global + conn, err := nats.Connect(c.natsAddress) + if err != nil { + return fmt.Errorf("connect failed: %w", err) + } + defer conn.Close() + + data, err := json.Marshal(event) + if err != nil { + return err + } + + return conn.Publish(event.Topic, data) +} + +func (c *NATSClient) PublishBatch(events []Event) error { + conn, err := nats.Connect(c.natsAddress) + if err != nil { + return fmt.Errorf("connect failed: %w", err) + } + defer conn.Close() + + for _, event := range events { + // ... publishing logic + } + return nil +} + +// ✅ Now testable without globals! +func TestNATSClient_PublishEvent(t *testing.T) { + // Start local test NATS server + testNATS := startTestNATS(t) + defer testNATS.Stop() + + // Create client with test address - NO GLOBALS! + client := NewNATSClient(testNATS.URL()) + + // Test with real implementation + err := client.PublishEvent(testEvent) + assert.NoError(t, err) + + // Can run in parallel! + // No global state needed! +} +``` + +**Island #1 Created**: `NATSClient` is now 100% testable without global dependencies. + +### Step 3: Push Global Up One Level + +**Now the callers (one level up) need updating**: + +```go +// ❌ Before - OrderService accessed globals +package order + +func ProcessOrder(orderID string) error { + db := connectDB(env.Configs.DBHost) + defer db.Close() + + // This used to access env.Configs.NATsAddress internally + err := messaging.PublishEvent(orderCreatedEvent) + return err +} + +// ✅ After - OrderService gets clean dependency +package order + +type OrderService struct { + dbHost string // Injected + natsClient *NATSClient // Clean dependency! +} + +func NewOrderService(dbHost string, natsClient *NATSClient) *OrderService { + return &OrderService{ + dbHost: dbHost, + natsClient: natsClient, + } +} + +func (s *OrderService) ProcessOrder(orderID string) error { + db := connectDB(s.dbHost) + defer db.Close() + + // Use clean dependency - no globals + err := s.natsClient.PublishEvent(orderCreatedEvent) + return err +} + +// ✅ Now OrderService is testable! +func TestOrderService_ProcessOrder(t *testing.T) { + testNATS := startTestNATS(t) + defer testNATS.Stop() + + natsClient := NewNATSClient(testNATS.URL()) + service := NewOrderService("localhost:5432", natsClient) + + err := service.ProcessOrder("order-123") + assert.NoError(t, err) +} +``` + +**Island #2 Created**: `OrderService` is now testable with clean dependencies. + +### Step 4: Continue Up the Chain to Entry Points + +**Finally, reach the HTTP handlers (entry points)**: + +```go +// ✅ HTTP handler - global accessed only here (top level) +package api + +type OrderHandler struct { + orderService *OrderService +} + +func SetupOrderHandler() *OrderHandler { + // Global accessed only at entry point! + natsClient := NewNATSClient(env.Configs.NATsAddress) + orderService := NewOrderService(env.Configs.DBHost, natsClient) + + return &OrderHandler{ + orderService: orderService, + } +} + +func (h *OrderHandler) HandleCreateOrder(w http.ResponseWriter, r *http.Request) { + // All clean code from here down - no globals! + err := h.orderService.ProcessOrder(orderID) + // ... +} +``` + +**Final state**: +- Globals only in `SetupOrderHandler()` (acceptable!) +- Everything below is clean, testable code +- 2 global accesses (was 20+) + +## Complete Before/After Comparison + +### Before: Globals Everywhere (20+ accesses) + +``` +main() + └─ handlers [env.Configs access] + ├─ OrderService [env.Configs access] + │ └─ messaging funcs [env.Configs access] + └─ UserService [env.Configs access] + └─ messaging funcs [env.Configs access] + +Testing: IMPOSSIBLE without global state mutation +Parallel tests: NO +``` + +### After: Globals Only at Top (2 accesses) + +``` +main() + └─ handlers [env.Configs access - 2 locations ONLY] + ├─ OrderService [clean - injected deps] + │ └─ NATSClient [clean - injected deps] + └─ UserService [clean - injected deps] + └─ NATSClient [clean - injected deps] + +Testing: EASY - inject test values +Parallel tests: YES +Islands of clean code: 3 types (NATSClient, OrderService, UserService) +``` + +## Testing Benefits Demonstrated + +### Before: Global State Nightmare + +```go +func TestPublishEvent(t *testing.T) { + // ❌ Must mutate global + originalAddr := env.Configs.NATsAddress + env.Configs.NATsAddress = "nats://test:4222" + defer func() { + env.Configs.NATsAddress = originalAddr // Restore + }() + + // ❌ Can't run in parallel - global state shared + // ❌ Tests can interfere with each other + // ❌ Hard to test multiple scenarios +} +``` + +### After: Clean Dependency Injection + +```go +func TestNATSClient_PublishEvent(t *testing.T) { + t.Parallel() // ✅ Parallel execution! + + // ✅ No global state needed + testNATS := startTestNATS(t) + defer testNATS.Stop() + + // ✅ Clean injection + client := NewNATSClient(testNATS.URL()) + + // ✅ Easy to test edge cases + tests := []struct { + name string + address string + wantErr bool + }{ + {"valid", testNATS.URL(), false}, + {"invalid", "nats://nonexistent:4222", true}, + {"empty", "", true}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + // Each test gets clean client - no interference! + c := NewNATSClient(tt.address) + err := c.PublishEvent(testEvent) + // ... assertions + }) + } +} +``` + +## Metrics + +### Before Refactoring + +``` +Global Accesses: +- env.Configs.NATsAddress: 12 locations (deep in code) +- env.Configs.DBHost: 8 locations (deep in code) +Total: 20 global accesses scattered + +Testability: +- Testable types: 0 (all depend on globals) +- Parallel tests: Impossible +- Test complexity: High (global state setup/teardown) + +Code Quality: +- Hidden dependencies: Everywhere +- Coupling: Tight (everything to env.Configs) +- Flexibility: Low (can't swap implementations) +``` + +### After Refactoring + +``` +Global Accesses: +- Entry points only: 2 locations (SetupOrderHandler, SetupUserHandler) +- Deep code: 0 global accesses + +Testability: +- Clean testable types: 3 islands (NATSClient, OrderService, UserService) +- Test coverage: 100% on clean types +- Parallel tests: Fully supported +- Test complexity: Low (simple dependency injection) + +Code Quality: +- Hidden dependencies: Eliminated +- Coupling: Loose (injected dependencies) +- Flexibility: High (easy to swap implementations) + +Improvement: +- 90% reduction in global access points (20 → 2) +- 3 new testable types created +- 100% of business logic now testable +``` + +## Key Lessons + +### 1. Incremental is Better Than Perfect + +**DON'T**: +```go +// ❌ Try to eliminate ALL globals at once +// This is overwhelming and error-prone +``` + +**DO**: +```go +// ✅ One type at a time, one level at a time +// Step 1: Extract NATSClient (week 1) +// Step 2: Extract OrderService (week 2) +// Step 3: Continue gradually +``` + +### 2. Bottom-Up Approach Works + +**Start at the leaf** (furthest from main): +1. Identify deepest usage (`messaging.PublishEvent`) +2. Extract clean type (`NATSClient`) +3. Push global up one level (`OrderService`) +4. Repeat until reaching entry points + +### 3. Pragmatic Stopping Point + +**Acceptable to have globals at**: +- `main()` function +- HTTP handler setup +- Application initialization +- Top-level factory functions + +**NOT acceptable deep in**: +- Business logic types +- Data access types +- Service layer +- Utility functions + +### 4. Islands of Clean Code + +Each extracted type is an "island": +- **Fully testable** in isolation +- **No global dependencies** +- **Explicit dependencies** via constructor +- **Can be tested in parallel** +- **Easy to understand** (dependencies visible) + +### 5. Real Testability + +**Before**: "We have 80% test coverage" +(But all tests depend on global state mutation) + +**After**: "We have 95% test coverage" +(All tests use clean dependency injection, run in parallel) + +Real testability means: +- ✅ No global state manipulation +- ✅ Tests run in parallel +- ✅ Tests are isolated +- ✅ Easy to test edge cases +- ✅ Can use real implementations (no mocking) + +## When to Apply This Pattern + +**Apply dependency rejection when**: +- ✅ `noglobals` linter fails +- ✅ Global config accessed throughout codebase +- ✅ Testing requires global state mutation +- ✅ Parallel tests fail due to shared state +- ✅ Code has hidden dependencies + +**DON'T apply to**: +- ❌ Loggers (`slog`, `zerolog`) - designed to be global +- ❌ Constants/enums - these are fine as globals +- ❌ Read-only singletons with no state + +## Refactoring Progression Example + +``` +Iteration 1 (Week 1): +├─ Extract NATSClient +├─ Global accesses: 20 → 14 +└─ Testable types: 1 + +Iteration 2 (Week 2): +├─ Extract OrderService +├─ Global accesses: 14 → 8 +└─ Testable types: 2 + +Iteration 3 (Week 3): +├─ Extract UserService +├─ Global accesses: 8 → 4 +└─ Testable types: 3 + +Iteration 4 (Week 4): +├─ Push to handlers +├─ Global accesses: 4 → 2 +└─ Mission accomplished! ✅ +``` + +**Note**: Each iteration is a working, tested, deployable state. No "big bang" refactoring needed. + +## Conclusion + +Dependency rejection is about **gradual improvement**, not perfection: +- Start at the bottom (leaf code) +- Create one clean island at a time +- Push globals up one level per iteration +- Stop when globals are only at entry points +- Every step improves testability + +**The goal isn't zero globals** - it's **globals only where they belong** (entry points), with all business logic cleanly injectable and testable. \ No newline at end of file diff --git a/skills/refactoring/reference.md b/skills/refactoring/reference.md new file mode 100644 index 0000000..895953a --- /dev/null +++ b/skills/refactoring/reference.md @@ -0,0 +1,745 @@ +# Refactoring Patterns Reference + +Complete guide for linter-driven refactoring with decision tree and patterns. + +## Refactoring Decision Tree + +When linter fails or code feels complex, use this decision tree: + +### Question 1: Does this code read like a story? +**Check**: Does it mix different levels of abstractions? + +```go +// ❌ No - Mixes abstractions +func CreatePizza(order Order) Pizza { + pizza := Pizza{Base: order.Size} // High-level + + // Low-level temperature control + for oven.Temp < cookingTemp { + time.Sleep(checkOvenInterval) + oven.Temp = getOvenTemp(oven) + } + + return pizza +} + +// ✅ Yes - Story-like +func CreatePizza(order Order) Pizza { + pizza := prepare(order) + bake(pizza) + return pizza +} +``` + +**Action**: Break it down to same level of abstraction. Hide nitty-gritty details behind methods with proper names. + +### Question 2: Can this be broken into smaller pieces? +**By what**: Responsibility? Task? Category? + +Breaking down can be done at all levels: +- Extract a variable +- Extract a function +- Create a new type +- Create a new package + +```go +// ❌ Multiple responsibilities +func HandleUserRequest(w http.ResponseWriter, r *http.Request) { + // Parse request + var user User + json.NewDecoder(r.Body).Decode(&user) + + // Validate + if user.Email == "" { /* ... */ } + + // Save to DB + db.Exec("INSERT INTO...") + + // Send response + json.NewEncoder(w).Encode(map[string]string{"status": "ok"}) +} + +// ✅ Separated by responsibility +func HandleUserRequest(w http.ResponseWriter, r *http.Request) { + user, err := parseUser(r) + if err != nil { + respondError(w, err) + return + } + + if err := validateUser(user); err != nil { + respondError(w, err) + return + } + + if err := saveUser(user); err != nil { + respondError(w, err) + return + } + + respondSuccess(w) +} +``` + +### Question 3: Does logic run on a primitive? +**Check**: Is this primitive obsession? + +If logic operates on string/int/float, consider creating a type. + +```go +// ❌ Primitive obsession +func ValidateEmail(email string) bool { + return strings.Contains(email, "@") +} + +func SendEmail(email string, subject, body string) error { + if !ValidateEmail(email) { + return errors.New("invalid email") + } + // Send +} + +// ✅ Custom type +type Email string + +func NewEmail(s string) (Email, error) { + if !strings.Contains(s, "@") { + return "", errors.New("invalid email") + } + return Email(s), nil +} + +func SendEmail(email Email, subject, body string) error { + // No validation needed - type guarantees validity + // Send +} +``` + +**Note**: Cohesion is more important than coupling. Put logic where it belongs, even if it creates dependencies. + +### Question 4: Is function long due to switch statement? +**Check**: Can cases be categorized and extracted? + +```go +// ❌ Long switch statement +func ProcessEvent(eventType string, data interface{}) error { + switch eventType { + case "user_created": + // 20 lines + case "user_updated": + // 25 lines + case "user_deleted": + // 15 lines + // ... more cases + } +} + +// ✅ Extracted case handlers +func ProcessEvent(eventType string, data interface{}) error { + switch eventType { + case "user_created": + return handleUserCreated(data) + case "user_updated": + return handleUserUpdated(data) + case "user_deleted": + return handleUserDeleted(data) + default: + return errors.New("unknown event type") + } +} + +func handleUserCreated(data interface{}) error { /* ... */ } +func handleUserUpdated(data interface{}) error { /* ... */ } +func handleUserDeleted(data interface{}) error { /* ... */ } +``` + +### Question 5: Types with logic? +**Rule**: Types with logic should be in their own file. Name file after type. + +``` +user/ +├── user.go # User type +├── user_id.go # UserID type with logic +├── email.go # Email type with logic +└── service.go # UserService +``` + +--- + +## Detailed Refactoring Patterns + +### 1. Storifying (Abstraction Levels) + +**Signal:** +- Linter: High cognitive complexity +- Code smell: Mixed high-level and low-level code + +**Pattern:** +```go +// Before +func ProcessOrder(order Order) error { + // Validation + if order.ID == "" { return errors.New("invalid") } + if len(order.Items) == 0 { return errors.New("no items") } + for _, item := range order.Items { + if item.Price < 0 { return errors.New("negative price") } + } + + // Database + db, err := sql.Open("postgres", os.Getenv("DB_URL")) + if err != nil { return err } + defer db.Close() + + tx, err := db.Begin() + if err != nil { return err } + + // SQL queries + _, err = tx.Exec("INSERT INTO orders...") + // ... many more lines + + // Email + smtp, err := mail.Dial("smtp.example.com:587") + // ... email sending logic + + return nil +} + +// After +func ProcessOrder(order Order) error { + if err := validateOrder(order); err != nil { + return err + } + + if err := saveToDatabase(order); err != nil { + return err + } + + if err := notifyCustomer(order); err != nil { + return err + } + + return nil +} +``` + +**Benefits:** +- Clear flow (validate → save → notify) +- Each function single responsibility +- Easy to test +- Easy to modify + +**Real-world example:** See [Example 1 in examples.md](./examples.md#example-1-storifying-mixed-abstractions-and-extracting-logic-into-leaf-types) for a production case of storifying mixed abstractions and extracting a leaf type for IP collection logic + +### 2. Extract Type (Primitive Obsession) + +**Signal:** +- Linter: High cyclomatic complexity (due to validation) +- Code smell: Validation repeated across codebase + +**Pattern:** +```go +// Before: Validation scattered +func CreateServer(host string, port int) (*Server, error) { + if host == "" { + return nil, errors.New("host required") + } + if port <= 0 || port > 65535 { + return nil, errors.New("invalid port") + } + // ... +} + +func ConnectToServer(host string, port int) error { + if host == "" { + return errors.New("host required") + } + if port <= 0 || port > 65535 { + return errors.New("invalid port") + } + // ... +} + +// After: Self-validating types +type Host string +type Port int + +func NewHost(s string) (Host, error) { + if s == "" { + return "", errors.New("host required") + } + return Host(s), nil +} + +func NewPort(p int) (Port, error) { + if p <= 0 || p > 65535 { + return 0, errors.New("port must be 1-65535") + } + return Port(p), nil +} + +type ServerAddress struct { + host Host + port Port +} + +func NewServerAddress(host Host, port Port) ServerAddress { + // No validation needed - types are already valid + return ServerAddress{host: host, port: port} +} + +func (a ServerAddress) String() string { + return fmt.Sprintf("%s:%d", a.host, a.port) +} + +func CreateServer(addr ServerAddress) (*Server, error) { + // No validation needed + // ... +} + +func ConnectToServer(addr ServerAddress) error { + // No validation needed + // ... +} +``` + +**Benefits:** +- Validation centralized +- Type safety +- Reduced complexity +- Self-documenting + +**Real-world example:** See [Example 2 in examples.md](./examples.md#example-2-primitive-obsession-with-multiple-types-and-storifying-switch-statements) for extracting multiple types from a 60-line function with primitive obsession. Shows the Type Alias Pattern for creating config-friendly types and eliminating switch statement duplication. + +--- + +### 2.5. The Over-Abstraction Trap ⚠️ + +**Critical**: Not every primitive needs a type. The goal is **clarity**, not **type proliferation**. + +#### Quick Decision Checklist + +**Create types when they**: +- ✅ Have multiple meaningful methods (>1) with real logic +- ✅ Enforce invariants/validation at construction +- ✅ Hide complex implementation +- ✅ Need controlled mutation → use **private fields**, NOT wrappers + +**DON'T create types when they**: +- ❌ Just wrap primitives with one trivial method +- ❌ Add ceremony without benefit +- ❌ Good naming achieves same clarity + +#### Bad vs Good: One Example + +```go +// ❌ Bad: Trivial wrapper - 8 lines, no benefit +type CIDRPresence bool +func (p CIDRPresence) IsSet() bool { return bool(p) } + +// ✅ Good: Private fields - same safety, less code +type CIDRConfig struct { + clusterCIDRSet bool // Only parser can set + serviceCIDRSet bool +} +func (c CIDRConfig) ClusterCIDRSet() bool { return c.clusterCIDRSet } +``` + +#### Complete Teaching & Examples + +**→ See [Example 2: Over-Abstraction Section](./examples.md#first-refactoring-attempt-the-over-abstraction-trap)** + +Full case study includes: +- Complete thought process & comparisons +- 6 questions before creating a type +- Balance diagram & decision tree +- When to stop refactoring + +--- + +### 3. Early Returns (Reduce Nesting) + +**Signal:** +- Linter: High cyclomatic complexity +- Code smell: Nesting > 2 levels + +**Pattern:** +```go +// Before: Deep nesting +func ProcessRequest(req Request) error { + if req.IsValid() { + if req.HasAuth() { + if req.HasPermission() { + // Do work + result, err := doWork(req) + if err != nil { + return err + } + return saveResult(result) + } else { + return errors.New("no permission") + } + } else { + return errors.New("not authenticated") + } + } else { + return errors.New("invalid request") + } +} + +// After: Early returns +func ProcessRequest(req Request) error { + if !req.IsValid() { + return errors.New("invalid request") + } + + if !req.HasAuth() { + return errors.New("not authenticated") + } + + if !req.HasPermission() { + return errors.New("no permission") + } + + result, err := doWork(req) + if err != nil { + return err + } + + return saveResult(result) +} +``` + +**Benefits:** +- Reduced nesting (max 1 level) +- Easier to read (guard clauses up front) +- Lower cyclomatic complexity + +### 4. Extract Function (Long Functions) + +**Signal:** +- Function > 50 LOC +- Multiple distinct concerns + +**Pattern:** +```go +// Before: Long function (80 LOC) +func RegisterUser(data map[string]interface{}) error { + // Parsing (15 lines) + email, ok := data["email"].(string) + if !ok { return errors.New("email required") } + // ... more parsing + + // Validation (20 lines) + if email == "" { return errors.New("email required") } + if !strings.Contains(email, "@") { return errors.New("invalid email") } + // ... more validation + + // Database (25 lines) + db, err := getDB() + if err != nil { return err } + // ... DB operations + + // Email (15 lines) + smtp := getSMTP() + // ... email sending + + // Logging (5 lines) + log.Printf("User registered: %s", email) + // ... + + return nil +} + +// After: Extracted functions +func RegisterUser(data map[string]interface{}) error { + user, err := parseUserData(data) + if err != nil { + return err + } + + if err := validateUser(user); err != nil { + return err + } + + if err := saveUserToDB(user); err != nil { + return err + } + + if err := sendWelcomeEmail(user); err != nil { + return err + } + + logUserRegistration(user) + return nil +} + +func parseUserData(data map[string]interface{}) (*User, error) { + // 15 lines +} + +func validateUser(user *User) error { + // 20 lines +} + +func saveUserToDB(user *User) error { + // 25 lines +} + +func sendWelcomeEmail(user *User) error { + // 15 lines +} + +func logUserRegistration(user *User) { + // 5 lines +} +``` + +**Guidelines:** +- Aim for functions under 50 LOC +- Each function single responsibility +- Top-level function reads like a story + +### 5. Switch Statement Extraction + +**Signal:** +- Long function due to switch statement +- Each case is complex + +**Pattern:** +```go +// Before +func RouteHandler(action string, params map[string]string) error { + switch action { + case "create": + // Validate create params + if params["name"] == "" { return errors.New("name required") } + // ... 15 more lines + return db.Create(...) + + case "update": + // Validate update params + if params["id"] == "" { return errors.New("id required") } + // ... 20 more lines + return db.Update(...) + + case "delete": + // Validate delete params + // ... 12 more lines + return db.Delete(...) + + default: + return errors.New("unknown action") + } +} + +// After +func RouteHandler(action string, params map[string]string) error { + switch action { + case "create": + return handleCreate(params) + case "update": + return handleUpdate(params) + case "delete": + return handleDelete(params) + default: + return errors.New("unknown action") + } +} + +func handleCreate(params map[string]string) error { + // All create logic (15 lines) +} + +func handleUpdate(params map[string]string) error { + // All update logic (20 lines) +} + +func handleDelete(params map[string]string) error { + // All delete logic (12 lines) +} +``` + +### 6. Defer Complexity Extraction + +**Signal:** +- Linter: Defer function has cyclomatic complexity > 1 + +**Pattern:** +```go +// Before: Complex defer +func ProcessFile(filename string) error { + f, err := os.Open(filename) + if err != nil { + return err + } + + defer func() { + if err := f.Close(); err != nil { + if !errors.Is(err, fs.ErrClosed) { + log.Printf("Error closing file: %v", err) + } + } + }() + + // Process file + return nil +} + +// After: Extracted cleanup function +func ProcessFile(filename string) error { + f, err := os.Open(filename) + if err != nil { + return err + } + defer closeFile(f) + + // Process file + return nil +} + +func closeFile(f *os.File) { + if err := f.Close(); err != nil { + if !errors.Is(err, fs.ErrClosed) { + log.Printf("Error closing file: %v", err) + } + } +} +``` + +--- + +## Linter-Specific Refactoring + +### Cyclomatic Complexity +**Cause**: Too many decision points (if, switch, for, &&, ||) + +**Solutions:** +1. Extract functions for different branches +2. Use early returns to reduce nesting +3. Extract type with methods for primitive logic +4. Simplify boolean expressions + +### Cognitive Complexity +**Cause**: Code hard to understand (nested logic, mixed abstractions) + +**Solutions:** +1. Storifying (clarify abstraction levels) +2. Extract nested logic to named functions +3. Use early returns +4. Break into smaller, focused functions + +### Maintainability Index +**Cause**: Code difficult to maintain + +**Solutions:** +1. All of the above +2. Improve naming +3. Add comments for complex logic +4. Reduce coupling + +--- + +## Guidelines for Effective Refactoring + +### Keep Functions Small +- Target: Under 50 LOC +- Max 2 nesting levels +- Single responsibility + +### Prefer Simplicity +- Simple, straightforward solutions over complex ones +- Descriptive variable and function names +- Avoid magic numbers and strings + +### Maintain Tests +- Tests should pass after refactoring +- Add tests for new functions if needed +- Maintain or improve coverage + +### Avoid Global State +- No global variables +- Inject dependencies through constructors +- Keep state localized + +--- + +## Common Refactoring Scenarios + +### Scenario 1: Linter Says "Cyclomatic Complexity Too High" +1. Identify decision points (if, switch, loops) +2. Extract branches to separate functions +3. Consider early returns +4. Check for primitive obsession (move logic to type) + +### Scenario 2: Function Feels Hard to Test +1. Probably doing too much → Extract functions +2. Might have hidden dependencies → Inject through constructor +3. Might mix concerns → Separate responsibilities + +### Scenario 3: Code Duplicated Across Functions +1. Extract common logic to shared function +2. Consider if primitives should be types (with methods) +3. Check if behavior belongs on existing type + +### Scenario 4: Can't Name Function Clearly +1. Probably doing too much → Split responsibilities +2. Might be at wrong abstraction level +3. Reconsider what the function should do + +--- + +## After Refactoring Checklist + +- [ ] Linter passes (`task lintwithfix`) +- [ ] Tests pass (`go test ./...`) +- [ ] Coverage maintained or improved +- [ ] Code more readable +- [ ] Functions under 50 LOC +- [ ] Max 2 nesting levels +- [ ] Each function has clear purpose + +--- + +## Integration with Design Principles + +Refactoring often reveals design issues. After refactoring, consider: + +**Created new types?** +→ Use @code-designing to validate type design + +**Changed architecture?** +→ Ensure still following vertical slice structure + +**Extracted significant logic?** +→ Ensure tests cover new functions (100% for leaf types) + +--- + +## Summary: Refactoring Decision Tree + +``` +Linter fails or code complex + ↓ +1. Does it read like a story? + No → Extract functions for abstraction levels + ↓ +2. Can it be broken into smaller pieces? + Yes → By responsibility/task/category? + Extract functions/types/packages + ↓ +3. Does logic run on primitives? + Yes → Is this primitive obsession? + Create custom type with methods + ↓ +4. Is it long due to switch statement? + Yes → Extract case handlers + ↓ +5. Deeply nested if/else? + Yes → Early returns or extract functions + ↓ +Re-run linter → Should pass +Run tests → Should pass +If new types → Validate with @code-designing +``` + +**Remember**: Cohesion > Coupling. Put logic where it belongs. diff --git a/skills/testing/SKILL.md b/skills/testing/SKILL.md new file mode 100644 index 0000000..ad53897 --- /dev/null +++ b/skills/testing/SKILL.md @@ -0,0 +1,255 @@ +--- +name: testing +description: Automatically invoked to write tests for new types, or use as testing expert advisor for guidance and recommendations. Covers unit, integration, and system tests with emphasis on in-memory dependencies. Use when creating leaf types, after refactoring, during implementation, or when testing advice is needed. Ensures 100% coverage on leaf types with public API testing. +--- + +# Testing Principles + +Principles and patterns for writing effective Go tests. + +## When to Use + +### Automatic Invocation (Proactive) +- **Automatically invoked** by @linter-driven-development during Phase 2 (Implementation) +- **Automatically invoked** by @refactoring when new isolated types are created +- **Automatically invoked** by @code-designing after designing new types +- **After creating new leaf types** - Types that should have 100% unit test coverage +- **After extracting functions** during refactoring that create testable units + +### Manual Invocation +- User explicitly requests tests to be written +- User asks for testing advice, recommendations, or "what to do" +- When testing strategy is unclear (table-driven vs testify suites) +- When choosing between dependency levels (in-memory vs binary vs test-containers) +- When adding tests to existing untested code +- When user needs testing expert guidance or consultation + +**IMPORTANT**: This skill writes tests autonomously based on the code structure and type design, and also serves as a testing expert advisor + +## Testing Philosophy + +**Test only the public API** +- Use `pkg_test` package name +- Test types through their constructors +- No testing private methods/functions + +**Prefer real implementations over mocks** +- Use in-memory implementations (fastest, no external deps) +- Use HTTP test servers (httptest) +- Use temp files/directories +- Test with actual dependencies when beneficial + +**Coverage targets** +- Leaf types: 100% unit test coverage +- Orchestrating types: Integration tests +- Critical workflows: System tests + +## Test Pyramid + +Three levels of testing, each serving a specific purpose: + +**Unit Tests** (Base of pyramid - most tests here) +- Test leaf types in isolation +- Fast, focused, no external dependencies +- 100% coverage target for leaf types +- Use `pkg_test` package, test public API only + +**Integration Tests** (Middle - fewer than unit) +- Test seams between components +- Test workflows across package boundaries +- Use real or in-memory implementations +- Verify components work together correctly + +**System Tests** (Top - fewest tests) +- Black box testing from `tests/` folder +- Test entire system via CLI/API +- Test critical end-to-end workflows +- **Strive for independence in Go** (minimize external deps) + +## Reusable Test Infrastructure + +Build shared test infrastructure in `internal/testutils/`: +- In-memory mock servers with DSL (HTTP, DB, file system) +- Reusable across all test levels +- Test the infrastructure itself! +- Can expose as CLI tools for manual testing + +**Dependency Priority** (minimize external dependencies): +1. **In-memory** (preferred): Pure Go, httptest, in-memory DB +2. **Binary**: Standalone executable via exec.Command +3. **Test-containers**: Programmatic Docker from Go +4. **Docker-compose**: Last resort, manual testing only + +Goal: System tests should be **independent in Go** when possible. + +See reference.md for comprehensive testutils patterns and DSL examples. + +## Workflow + +### Unit Tests Workflow + +**Purpose**: Test leaf types in isolation, 100% coverage target + +1. **Identify leaf types** - Self-contained types with logic +2. **Choose structure** - Table-driven (simple) or testify suites (complex setup) +3. **Write in pkg_test package** - Test public API only +4. **Use in-memory implementations** - From testutils or local implementations +5. **Avoid pitfalls** - No time.Sleep, no conditionals in cases, no private method tests + +**Test structure:** +- Table-driven: Separate success/error test functions (complexity = 1) +- Testify suites: Only for complex infrastructure setup (HTTP servers, DBs) +- Always use named struct fields (linter reorders fields) + +See reference.md for detailed patterns and examples. + +### Integration Tests Workflow + +**Purpose**: Test seams between components, verify they work together + +1. **Identify integration points** - Where packages/components interact +2. **Choose dependencies** - Prefer: in-memory > binary > test-containers +3. **Write tests** - In `pkg_test` or `integration_test.go` with build tags +4. **Test workflows** - Cover happy path and error scenarios across boundaries +5. **Use real or testutils implementations** - Avoid heavy mocking + +**File organization:** +```go +//go:build integration + +package user_test + +// Test Service + Repository + real/mock dependencies +``` + +See reference.md for integration test patterns with dependencies. + +### System Tests Workflow + +**Purpose**: Black box test entire system, critical end-to-end workflows + +1. **Place in tests/ folder** - At project root, separate from packages +2. **Test via CLI/API** - exec.Command for CLI, HTTP client for APIs +3. **Minimize external deps** - Prefer: in-memory mocks > binary > test-containers +4. **Strive for Go independence** - Pure Go tests, no Docker when possible +5. **Test critical workflows** - User journeys, not every edge case + +**Example structure:** +```go +// tests/cli_test.go +func TestCLI_UserWorkflow(t *testing.T) { + mockAPI := testutils.NewMockServer(). + OnGET("/users/1").RespondJSON(200, user). + Build() // In-memory httptest.Server + defer mockAPI.Close() + + cmd := exec.Command("./myapp", "get-user", "1", + "--api-url", mockAPI.URL()) + output, err := cmd.CombinedOutput() + // Assert on output +} +``` + +See reference.md for comprehensive system test patterns. + +## Key Test Patterns + +**Table-Driven Tests:** +- Separate success and error test functions (complexity = 1) +- Always use named struct fields (linter reorders fields) +- No wantErr bool pattern (adds conditionals) + +**Testify Suites:** +- Only for complex infrastructure (HTTP servers, DBs, OpenTelemetry) +- SetupSuite/TearDownSuite for expensive shared setup +- SetupTest/TearDownTest for per-test isolation + +**Synchronization:** +- Never use time.Sleep (flaky, slow) +- Use channels with select/timeout for async operations +- Use sync.WaitGroup for concurrent operations + +See reference.md for complete patterns with code examples. + +## Output Format + +After writing tests: + +``` +✅ TESTING COMPLETE + +📊 Unit Tests: +- user/user_id_test.go: 100% (4 test cases) +- user/email_test.go: 100% (6 test cases) +- user/service_test.go: 100% (8 test cases) + +🔗 Integration Tests: +- user/integration_test.go: 3 workflows tested +- Dependencies: In-memory DB, httptest mock server + +🎯 System Tests: +- tests/cli_test.go: 2 end-to-end workflows +- tests/api_test.go: 1 full API workflow +- Infrastructure: In-memory mocks (pure Go, no Docker) + +Test Infrastructure: +- internal/testutils/httpserver: In-memory mock API with DSL +- internal/testutils/mockdb: In-memory database mock + +Test Execution: +$ go test ./... # All tests +$ go test -tags=integration ./... # Include integration tests +$ go test ./tests/... # System tests only + +✅ All tests pass +✅ 100% coverage on leaf types +✅ No external dependencies required + +Next Steps: +1. Run linter: task lintwithfix +2. If linter fails → use @refactoring skill +3. If linter passes → use @pre-commit-review skill +``` + +## Key Principles + +See reference.md for: +- Table-driven test patterns +- Testify suite guidelines +- Real implementations over mocks +- Synchronization techniques +- Coverage strategies + +## Testing Checklist + +### Unit Tests +- [ ] All unit tests in pkg_test package +- [ ] Testing public API only (no private methods) +- [ ] Table-driven tests use named struct fields +- [ ] No conditionals in test cases (complexity = 1) +- [ ] Using in-memory implementations from testutils +- [ ] No time.Sleep (using channels/waitgroups) +- [ ] Leaf types have 100% coverage + +### Integration Tests +- [ ] Test seams between components +- [ ] Use in-memory or binary dependencies (avoid Docker) +- [ ] Build tags for optional execution (`//go:build integration`) +- [ ] Cover happy path and error scenarios across boundaries +- [ ] Real or testutils implementations (minimal mocking) + +### System Tests +- [ ] Located in tests/ folder at project root +- [ ] Black box testing via CLI/API +- [ ] Uses in-memory testutils mocks (pure Go) +- [ ] No external dependencies (no Docker required) +- [ ] Tests critical end-to-end workflows +- [ ] Fast execution, runs in CI without setup + +### Test Infrastructure +- [ ] Reusable mocks in internal/testutils/ +- [ ] Test infrastructure has its own tests +- [ ] DSL provides readable test setup +- [ ] Can be exposed as CLI for manual testing + +See reference.md for complete testing guidelines and examples. diff --git a/skills/testing/examples/grpc-bufconn.md b/skills/testing/examples/grpc-bufconn.md new file mode 100644 index 0000000..c029469 --- /dev/null +++ b/skills/testing/examples/grpc-bufconn.md @@ -0,0 +1,318 @@ +# gRPC Testing with bufconn and Rich Client Mocks + +## When to Use This Example + +Use this when: +- Testing gRPC servers +- Need bidirectional streaming tests +- Want in-memory gRPC (no network I/O) +- Testing server-client interactions +- Need rich DSL for readable tests + +**Dependency Level**: Level 1 (In-Memory) - Uses `bufconn` for in-memory gRPC connections + +**Key Insight**: When testing a **gRPC server**, mock the **clients** that connect to it. When testing a **gRPC client**, mock the **server**. + +## Implementation + +### Rich gRPC Client Mock with DSL + +When your **System Under Test (SUT) is a gRPC server**, create rich client mocks: + +```go +// internal/testutils/grpc_client_mock.go +package testutils + +import ( + "context" + "io" + "sync" + "testing" + + "github.com/stretchr/testify/require" + "google.golang.org/grpc" + "google.golang.org/grpc/metadata" + + pb "myproject/grpc_api/gen/go/traces/v1" +) + +// TaskmonOnCluster is a rich gRPC client mock with DSL for testing gRPC servers. +// It connects to your gRPC server and provides helper methods for assertions. +type TaskmonOnCluster struct { + clusterRoutingKey string + taskmonID string + stream pb.RemoteTracesService_StreamTracesClient + mu sync.RWMutex + receivedQuery *pb.TracesQuery + receivedQueriesPayloads []string +} + +// OpenTaskmonToWekaHomeStream creates a gRPC client mock that connects to your server. +// This is the constructor for the mock - returns a rich DSL object. +func OpenTaskmonToWekaHomeStream( + ctx context.Context, + client pb.RemoteTracesServiceClient, + clusterRoutingKey, taskmonID string, +) (*TaskmonOnCluster, error) { + // Inject metadata (like session tokens) into context + md := metadata.Pairs("X-Taskmon-session-token", clusterRoutingKey) + ctx = metadata.NewOutgoingContext(ctx, md) + + // Open streaming connection to the server (your SUT) + stream, err := client.StreamTraces(ctx, grpc.Header(&md)) + if err != nil { + return nil, err + } + + return &TaskmonOnCluster{ + stream: stream, + clusterRoutingKey: clusterRoutingKey, + taskmonID: taskmonID, + receivedQueriesPayloads: []string{}, + }, nil +} + +// SessionToken returns the session token (useful for assertions) +func (m *TaskmonOnCluster) SessionToken() string { + return m.clusterRoutingKey +} + +// Close closes the stream (idempotent) +func (m *TaskmonOnCluster) Close() { + if m.stream == nil { + return + } + m.stream.CloseSend() +} + +// ListenToStreamAndAssert is a helper that listens to server messages and asserts. +// This makes tests read like documentation! +func (m *TaskmonOnCluster) ListenToStreamAndAssert( + t *testing.T, + expectedQueryPayload, + resultPayload string, +) { + for { + query, err := m.stream.Recv() + if err == io.EOF { + break + } + require.NoError(t, err, "Failed to receive query from server") + + // Store received data (thread-safe) + m.mu.Lock() + m.receivedQuery = query + m.receivedQueriesPayloads = append(m.receivedQueriesPayloads, string(query.TracesQueryPayload)) + m.mu.Unlock() + + // Assert expected payload + require.Equal(t, expectedQueryPayload, string(query.TracesQueryPayload)) + + // Send response back to server + response := &pb.TracesFromServer{ + TraceServerRoute: query.TraceServerRoute, + TracesPayload: []byte(resultPayload), + MessageId: query.MessageId, + } + err = m.stream.Send(response) + require.NoError(t, err, "Failed to send response") + } +} + +// LastReceivedQuery returns the last received query (thread-safe) +func (m *TaskmonOnCluster) LastReceivedQuery() *pb.TracesQuery { + m.mu.RLock() + defer m.mu.RUnlock() + return m.receivedQuery +} + +// ReceivedQueriesPayloads returns all received payloads (thread-safe) +func (m *TaskmonOnCluster) ReceivedQueriesPayloads() []string { + m.mu.RLock() + defer m.mu.RUnlock() + return m.receivedQueriesPayloads +} +``` + +## Usage in Integration Tests + +### Complete Test Suite Example + +```go +//go:build integration + +package integration_test + +import ( + "context" + "net" + "testing" + "time" + + "github.com/stretchr/testify/suite" + "google.golang.org/grpc" + "google.golang.org/grpc/credentials/insecure" + "google.golang.org/grpc/test/bufconn" + + pb "myproject/grpc_api/gen/go/traces/v1" + "myproject/internal/remotetraces" + "myproject/internal/testutils" +) + +type RemoteTracesTestSuite struct { + suite.Suite + lis *bufconn.Listener // In-memory gRPC connection + ctx context.Context + natsServer *nserver.Server // In-memory NATS +} + +func (suite *RemoteTracesTestSuite) SetupSuite() { + suite.ctx = context.Background() + + // Start in-memory NATS server (Level 1) + natsServer, err := testutils.RunNATsServer() + suite.Require().NoError(err) + suite.natsServer = natsServer + + // Connect to NATS + natsAddress := "nats://" + natsServer.Addr().String() + nc, err := natsremotetraces.ConnectToRemoteTracesSession(suite.ctx, natsAddress, 2, 2, 10) + suite.Require().NoError(err) + + // ** System Under Test: gRPC Server ** + // Use bufconn for in-memory gRPC (no network I/O!) + suite.lis = bufconn.Listen(1024 * 1024) + s := grpc.NewServer() + + // Your gRPC server implementation + remoteTracesServer := remotetraces.NewGRPCServer(nc, 10, 10, time.Second) + pb.RegisterRemoteTracesServiceServer(s, remoteTracesServer) + + go func() { + if err := s.Serve(suite.lis); err != nil { + suite.NoError(err) + } + }() +} + +func (suite *RemoteTracesTestSuite) bufDialer(ctx context.Context, _ string) (net.Conn, error) { + return suite.lis.DialContext(ctx) +} + +func (suite *RemoteTracesTestSuite) TestStreamTraces() { + // Create gRPC client (connects to your server) + conn, err := grpc.NewClient("passthrough:///bufnet", + grpc.WithContextDialer(suite.bufDialer), + grpc.WithTransportCredentials(insecure.NewCredentials()), + ) + suite.Require().NoError(err) + defer conn.Close() + + client := pb.NewRemoteTracesServiceClient(conn) + + // Create rich gRPC client mock (testutils DSL!) + clusterRoutingKey := "test-cluster-123" + taskmonMock, err := testutils.OpenTaskmonToWekaHomeStream( + suite.ctx, client, clusterRoutingKey, "taskmon-1") + suite.Require().NoError(err) + defer taskmonMock.Close() + + expectedQuery := "fetch_traces_query" + expectedResult := "traces_result_data" + + // Start listening (this makes the test readable!) + go taskmonMock.ListenToStreamAndAssert(suite.T(), expectedQuery, expectedResult) + + // Send query to server (via NATS or HTTP API) + // ... your test logic here ... + + // Assert using helper methods + suite.Eventually(func() bool { + return taskmonMock.LastReceivedQuery() != nil && + string(taskmonMock.LastReceivedQuery().TracesQueryPayload) == expectedQuery + }, 5*time.Second, 500*time.Millisecond) +} + +func TestRemoteTracesTestSuite(t *testing.T) { + suite.Run(t, new(RemoteTracesTestSuite)) +} +``` + +## Why This Pattern is Excellent + +1. **Rich DSL** - `OpenTaskmonToWekaHomeStream()` returns friendly object with helper methods +2. **Helper Methods** - `ListenToStreamAndAssert()`, `LastReceivedQuery()`, `ReceivedQueriesPayloads()` +3. **Thread-Safe** - Mutex protects shared state for concurrent access +4. **Readable Tests** - Tests read like documentation, clear intent +5. **In-Memory** - Uses `bufconn` (no network I/O, pure Go) +6. **Reusable** - Same mock for unit, integration, and system tests +7. **Event-Driven** - Can add channels for connection events if needed + +## Key Design Principles + +### Testing Direction + +- **Testing a server?** → Mock the **clients** that connect to it +- **Testing a client?** → Mock the **server** it connects to + +### DSL Benefits + +- Use rich DSL objects with helper methods +- Make tests read like documentation +- Hide complexity behind clean interfaces +- Provide thread-safe state tracking +- Enable fluent assertions + +### In-Memory with bufconn + +`bufconn` provides an in-memory, full-duplex network connection: +- No network I/O overhead +- No port allocation needed +- Faster than TCP loopback +- Perfect for CI/CD +- Deterministic behavior + +## Benefits + +- **No Docker required** - Pure Go, works anywhere +- **No binary downloads** - Everything in-memory +- **No network I/O** - Unless testing actual network code +- **Perfect for CI/CD** - Fast, reliable, no external dependencies +- **Lightning fast** - Microsecond startup time +- **Thread-safe** - Concurrent test execution safe + +## Alternative: Testing gRPC Clients + +If you're testing a **gRPC client**, mock the **server** instead: + +```go +// internal/testutils/grpc_server_mock.go +type MockGRPCServer struct { + pb.UnimplementedRemoteTracesServiceServer + mu sync.Mutex + receivedQueries []*pb.TracesQuery +} + +func (m *MockGRPCServer) StreamTraces(stream pb.RemoteTracesService_StreamTracesServer) error { + // Mock server implementation + // Store received queries, send responses + // ... + return nil +} + +// Usage +server := testutils.NewMockGRPCServer() +lis := bufconn.Listen(1024 * 1024) +s := grpc.NewServer() +pb.RegisterRemoteTracesServiceServer(s, server) +// ... test your client against this mock server +``` + +## Key Takeaways + +1. **bufconn is Level 1** - In-memory, no external dependencies +2. **Mock the opposite end** - Server → mock clients, Client → mock server +3. **Rich DSL makes tests readable** - Helper methods, clear intent +4. **Thread-safe state tracking** - Use mutexes for concurrent access +5. **Reusable across test levels** - Same infrastructure everywhere +6. **Check for official test harnesses first** - Many libraries provide them (like NATS) diff --git a/skills/testing/examples/httptest-dsl.md b/skills/testing/examples/httptest-dsl.md new file mode 100644 index 0000000..025e34e --- /dev/null +++ b/skills/testing/examples/httptest-dsl.md @@ -0,0 +1,281 @@ +# HTTP Test Server with DSL Pattern + +## When to Use This Example + +Use this when: +- Testing HTTP clients or APIs +- Need simple, readable HTTP mocking +- Want to avoid complex mock frameworks +- Testing REST APIs, webhooks, or HTTP integrations + +**Dependency Level**: Level 1 (In-Memory) - Uses stdlib `httptest.Server` + +## Basic httptest.Server Pattern + +### Simple HTTP Mock + +```go +func TestAPIClient(t *testing.T) { + // Create test server + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Mock API response + w.WriteHeader(http.StatusOK) + json.NewEncoder(w).Encode(map[string]string{"status": "ok"}) + })) + defer server.Close() + + // Use real HTTP client with test server URL + client := NewAPIClient(server.URL) + result, err := client.GetStatus() + + assert.NoError(t, err) + assert.Equal(t, "ok", result.Status) +} +``` + +## DSL Pattern for Readable Tests + +### Without DSL (Verbose) + +```go +func TestUserAPI(t *testing.T) { + handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.Method == "GET" && r.URL.Path == "/users/1" { + w.WriteHeader(200) + json.NewEncoder(w).Encode(map[string]string{"id": "1", "name": "Alice"}) + } else if r.Method == "POST" && r.URL.Path == "/users" { + // ... more complex logic + } else { + w.WriteHeader(404) + } + }) + server := httptest.NewServer(handler) + defer server.Close() + // ... test +} +``` + +### With DSL (Readable) + +```go +func TestUserAPI(t *testing.T) { + mockAPI := httpserver.New(). + OnGET("/users/1"). + RespondJSON(200, User{ID: "1", Name: "Alice"}). + OnPOST("/users"). + WithBodyMatcher(hasRequiredFields). + RespondJSON(201, User{ID: "2", Name: "Bob"}). + Build() + defer mockAPI.Close() + + // Test reads like documentation! + client := NewAPIClient(mockAPI.URL()) + user, err := client.GetUser("1") + // ... assertions +} +``` + +## Implementing the DSL + +### Basic DSL Structure + +```go +// internal/testutils/httpserver/server.go +package httpserver + +import ( + "encoding/json" + "net/http" + "net/http/httptest" +) + +type MockServer struct { + routes map[string]map[string]mockRoute // method -> path -> handler + server *httptest.Server +} + +type mockRoute struct { + statusCode int + response any + matcher func(*http.Request) bool +} + +func New() *MockServerBuilder { + return &MockServerBuilder{ + routes: make(map[string]map[string]mockRoute), + } +} + +type MockServerBuilder struct { + routes map[string]map[string]mockRoute +} + +func (b *MockServerBuilder) OnGET(path string) *RouteBuilder { + return &RouteBuilder{ + builder: b, + method: "GET", + path: path, + } +} + +func (b *MockServerBuilder) OnPOST(path string) *RouteBuilder { + return &RouteBuilder{ + builder: b, + method: "POST", + path: path, + } +} + +type RouteBuilder struct { + builder *MockServerBuilder + method string + path string + statusCode int + response any + matcher func(*http.Request) bool +} + +func (r *RouteBuilder) RespondJSON(statusCode int, response any) *MockServerBuilder { + if r.builder.routes[r.method] == nil { + r.builder.routes[r.method] = make(map[string]mockRoute) + } + r.builder.routes[r.method][r.path] = mockRoute{ + statusCode: statusCode, + response: response, + matcher: r.matcher, + } + return r.builder +} + +func (r *RouteBuilder) WithBodyMatcher(matcher func(*http.Request) bool) *RouteBuilder { + r.matcher = matcher + return r +} + +func (b *MockServerBuilder) Build() *MockServer { + mock := &MockServer{routes: b.routes} + + handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + methodRoutes, ok := mock.routes[r.Method] + if !ok { + w.WriteHeader(http.StatusMethodNotAllowed) + return + } + + route, ok := methodRoutes[r.URL.Path] + if !ok { + w.WriteHeader(http.StatusNotFound) + return + } + + if route.matcher != nil && !route.matcher(r) { + w.WriteHeader(http.StatusBadRequest) + return + } + + w.WriteHeader(route.statusCode) + json.NewEncoder(w).Encode(route.response) + }) + + mock.server = httptest.NewServer(handler) + return mock +} + +func (m *MockServer) URL() string { + return m.server.URL +} + +func (m *MockServer) Close() { + m.server.Close() +} +``` + +## Simple In-Memory Patterns + +### In-Memory Repository + +```go +// user/inmem.go +package user + +type InMemoryRepository struct { + mu sync.RWMutex + users map[UserID]User +} + +func NewInMemoryRepository() *InMemoryRepository { + return &InMemoryRepository{ + users: make(map[UserID]User), + } +} + +func (r *InMemoryRepository) Save(ctx context.Context, u User) error { + r.mu.Lock() + defer r.mu.Unlock() + r.users[u.ID] = u + return nil +} + +func (r *InMemoryRepository) Get(ctx context.Context, id UserID) (*User, error) { + r.mu.RLock() + defer r.mu.RUnlock() + + u, ok := r.users[id] + if !ok { + return nil, ErrNotFound + } + return &u, nil +} +``` + +### Test Email Sender + +```go +// user/test_emailer.go +package user + +import ( + "bytes" + "fmt" + "sync" +) + +type TestEmailer struct { + mu sync.Mutex + buffer bytes.Buffer +} + +func NewTestEmailer() *TestEmailer { + return &TestEmailer{} +} + +func (e *TestEmailer) Send(to Email, subject, body string) error { + e.mu.Lock() + defer e.mu.Unlock() + + fmt.Fprintf(&e.buffer, "To: %s\nSubject: %s\n%s\n\n", to, subject, body) + return nil +} + +func (e *TestEmailer) SentEmails() string { + e.mu.Lock() + defer e.mu.Unlock() + return e.buffer.String() +} +``` + +## Benefits + +- **Simple** - Built on stdlib, no external dependencies +- **Readable** - DSL makes tests self-documenting +- **Fast** - In-memory, microsecond startup +- **Flexible** - Easy to extend with new methods +- **Reusable** - Same pattern for all HTTP testing + +## Key Takeaways + +1. **Start with httptest.Server** - Simple and powerful +2. **Add DSL for readability** - When tests get complex +3. **Keep implementations simple** - In-memory maps, buffers +4. **Thread-safe** - Use mutexes for concurrent access +5. **Test your test infrastructure** - It's production code diff --git a/skills/testing/examples/integration-patterns.md b/skills/testing/examples/integration-patterns.md new file mode 100644 index 0000000..a30650d --- /dev/null +++ b/skills/testing/examples/integration-patterns.md @@ -0,0 +1,248 @@ +# Integration Test Patterns + +## Purpose + +Integration tests verify that components work together correctly. They test the seams between packages, ensure proper data flow, and validate that integrated components behave as expected. + +**When to Write**: After unit testing individual components, test how they interact. + +## File Organization + +### Option 1: In Package with Build Tags (Preferred) + +```go +//go:build integration + +package user_test + +import ( + "testing" + "myproject/internal/testutils" +) + +func TestUserService_Integration(t *testing.T) { + // Integration test +} +``` + +### Option 2: Separate Package + +``` +user/ +├── user.go +├── user_test.go # Unit tests +└── integration/ + └── user_integration_test.go # Integration tests +``` + +## Pattern 1: Service + Repository (In-Memory) + +**Use when**: Testing service logic with data persistence + +```go +//go:build integration + +package user_test + +import ( + "context" + "testing" + + "github.com/stretchr/testify/require" + "myproject/user" +) + +func TestUserService_CreateAndRetrieve(t *testing.T) { + // Setup: In-memory repository (Level 1) + repo := user.NewInMemoryRepository() + svc := user.NewUserService(repo, nil) + + ctx := context.Background() + + // Create user + userID, _ := user.NewUserID("usr_123") + email, _ := user.NewEmail("alice@example.com") + newUser := user.User{ + ID: userID, + Name: "Alice", + Email: email, + } + + err := svc.CreateUser(ctx, newUser) + require.NoError(t, err) + + // Retrieve user + retrieved, err := svc.GetUser(ctx, userID) + require.NoError(t, err) + require.Equal(t, "Alice", retrieved.Name) + require.Equal(t, email, retrieved.Email) +} +``` + +## Pattern 2: Testing with Real External Service + +**Use when**: Need to test against real service behavior (Victoria Metrics, NATS, etc.) + +```go +//go:build integration + +package metrics_test + +import ( + "context" + "testing" + + "github.com/stretchr/testify/require" + "myproject/internal/testutils" + "myproject/metrics" +) + +func TestMetricsIngest_WithVictoriaMetrics(t *testing.T) { + // Start real Victoria Metrics (Level 2 - binary) + vmServer, err := testutils.RunVictoriaMetricsServer() + require.NoError(t, err) + defer vmServer.Shutdown() + + // Create service with real dependency + svc := metrics.NewIngester(vmServer.WriteURL()) + + // Test ingestion + err = svc.IngestMetric(context.Background(), "test_metric", 42.0) + require.NoError(t, err) + + // Force flush and verify + vmServer.ForceFlush(context.Background()) + results, err := testutils.QueryVictoriaMetrics(vmServer.QueryURL(), "test_metric") + require.NoError(t, err) + require.Len(t, results, 1) +} +``` + +## Pattern 3: Multi-Component Workflow + +**Use when**: Testing complete workflows across multiple components + +```go +//go:build integration + +package workflow_test + +import ( + "context" + "testing" + "time" + + "github.com/stretchr/testify/suite" + "myproject/internal/testutils" + "myproject/user" + "myproject/notification" +) + +type UserWorkflowSuite struct { + suite.Suite + userRepo *user.InMemoryRepository + emailer *user.TestEmailer + natsServer *nserver.Server + userService *user.UserService + notifSvc *notification.NotificationService +} + +func (s *UserWorkflowSuite) SetupSuite() { + // Setup in-memory NATS (Level 1) + natsServer, err := testutils.RunNATsServer() + s.Require().NoError(err) + s.natsServer = natsServer + + // Setup components + s.userRepo = user.NewInMemoryRepository() + s.emailer = user.NewTestEmailer() + s.userService = user.NewUserService(s.userRepo, s.emailer) + + natsAddr := "nats://" + natsServer.Addr().String() + s.notifSvc = notification.NewService(natsAddr) +} + +func (s *UserWorkflowSuite) TearDownSuite() { + s.natsServer.Shutdown() +} + +func (s *UserWorkflowSuite) TestCreateUser_TriggersNotification() { + ctx := context.Background() + + // Subscribe to notifications + received := make(chan string, 1) + s.notifSvc.Subscribe("user.created", func(msg string) { + received <- msg + }) + + // Create user + userID, _ := user.NewUserID("usr_123") + email, _ := user.NewEmail("alice@example.com") + newUser := user.User{ID: userID, Name: "Alice", Email: email} + + err := s.userService.CreateUser(ctx, newUser) + s.Require().NoError(err) + + // Verify notification sent + select { + case msg := <-received: + s.Contains(msg, "Alice") + case <-time.After(2 * time.Second): + s.Fail("timeout waiting for notification") + } + + // Verify email sent + emails := s.emailer.SentEmails() + s.Contains(emails, "alice@example.com") +} + +func TestUserWorkflowSuite(t *testing.T) { + suite.Run(t, new(UserWorkflowSuite)) +} +``` + +## Dependency Priority + +1. **Level 1: In-Memory** (Preferred) - httptest, in-memory maps, NATS harness +2. **Level 2: Binary** (When needed) - Victoria Metrics, standalone services +3. **Level 3: Test-containers** (Last resort) - Docker containers, slow startup + +## Best Practices + +### DO: +- Test seams between components +- Use in-memory implementations when possible +- Test happy path and error scenarios +- Use testify suites for complex setup +- Focus on data flow and integration points + +### DON'T: +- Don't test business logic (that's unit tests) +- Don't use heavy mocking (use real implementations) +- Don't require Docker unless absolutely necessary +- Don't duplicate unit test coverage +- Don't skip cleanup (always defer) + +## Running Integration Tests + +```bash +# Skip integration tests (default) +go test ./... + +# Run with integration tests +go test -tags=integration ./... + +# Run only integration tests +go test -tags=integration ./... -run Integration + +# With coverage +go test -tags=integration -coverprofile=coverage.out ./... +``` + +## Key Takeaways + +1. **Test component interactions** - Not individual units +2. **Prefer real implementations** - Over mocks when possible +3. **Use build tags** - Keep unit tests fast +4. **Reuse testutils** - Same infrastructure across tests +5. **Test workflows** - Not just individual operations diff --git a/skills/testing/examples/jsonrpc-mock.md b/skills/testing/examples/jsonrpc-mock.md new file mode 100644 index 0000000..c3059af --- /dev/null +++ b/skills/testing/examples/jsonrpc-mock.md @@ -0,0 +1,263 @@ +# JSON-RPC Server Mock with DSL + +## When to Use This Example + +Use this when: +- Testing JSON-RPC clients +- Need to mock JSON-RPC server responses +- Want configurable mock behavior per method +- Need to track and assert on received requests +- Testing with OpenTelemetry trace propagation + +**Dependency Level**: Level 1 (In-Memory) - Uses `httptest.Server` for in-memory HTTP + +**Key Insight**: When testing a **JSON-RPC client**, mock the **server** it calls. Use rich DSL for readable test setup. + +## Implementation + +### Rich JSON-RPC Server Mock + +```go +// internal/testutils/jrpc_server_mock.go +package testutils + +import ( + "errors" + "fmt" + "net/http" + "net/http/httptest" + + "github.com/gorilla/rpc/v2/json2" + "go.opentelemetry.io/otel" + "go.opentelemetry.io/otel/propagation" + "go.opentelemetry.io/otel/trace" +) + +var ErrMethodNotFound = errors.New("method not found") + +// TraceQuery holds received JSON-RPC queries for assertions +type TraceQuery struct { + Method string + Params string +} + +// JrpcTraceServerMock is a rich JSON-RPC server mock with DSL. +// Uses httptest.Server for in-memory HTTP (Level 1). +type JrpcTraceServerMock struct { + tracer trace.Tracer + server *httptest.Server + mockResponses map[string]any // method -> response + queriesReceived []TraceQuery // for assertions +} + +// StartJrpcTraceServerMock starts an in-memory JSON-RPC server. +// Returns a rich DSL object for configuring mock responses. +func StartJrpcTraceServerMock() *JrpcTraceServerMock { + mock := &JrpcTraceServerMock{ + mockResponses: make(map[string]any), + tracer: otel.Tracer("trace-server-mock"), + } + + mux := mock.createHTTPHandlers() + mock.server = httptest.NewServer(mux) + + return mock +} + +// AddMockResponse configures the mock to return a response for a method. +// This is the DSL - chain multiple calls for different methods! +func (m *JrpcTraceServerMock) AddMockResponse(method string, response any) { + m.mockResponses[method] = response +} + +// GetQueriesReceived returns all queries received (for assertions) +func (m *JrpcTraceServerMock) GetQueriesReceived() []TraceQuery { + return m.queriesReceived +} + +// Close shuts down the server (idempotent) +func (m *JrpcTraceServerMock) Close() { + m.server.Close() +} + +// Address returns the server address (for client configuration) +func (m *JrpcTraceServerMock) Address() string { + return m.server.Listener.Addr().String() +} + +func (m *JrpcTraceServerMock) createHTTPHandlers() *http.ServeMux { + mux := http.NewServeMux() + codec := json2.NewCodec() + + mux.HandleFunc("/reader", func(w http.ResponseWriter, r *http.Request) { + // Extract OpenTelemetry context for realistic testing + reqCtx := r.Context() + reqCtx = otel.GetTextMapPropagator().Extract(reqCtx, propagation.HeaderCarrier(r.Header)) + reqCtx, span := m.tracer.Start(reqCtx, "jrpc-trace-server", + trace.WithSpanKind(trace.SpanKindServer)) + defer span.End() + + if r.Method != http.MethodPost { + w.WriteHeader(http.StatusMethodNotAllowed) + return + } + + receivedReq := codec.NewRequest(r) + method, err := receivedReq.Method() + if err != nil { + w.WriteHeader(http.StatusBadRequest) + return + } + + // Check if we have a mock response configured + if response, exists := m.mockResponses[method]; exists { + args := struct{}{} + if err := receivedReq.ReadRequest(&args); err != nil { + receivedReq.WriteError(w, http.StatusBadRequest, err) + return + } + + // Store query for assertions + m.queriesReceived = append(m.queriesReceived, TraceQuery{ + Method: method, + Params: fmt.Sprintf("%+v", args), + }) + + // Write mock response + receivedReq.WriteResponse(w, response) + return + } + + // Method not configured + params := []string{} + receivedReq.ReadRequest(¶ms) + receivedReq.WriteError(w, http.StatusBadRequest, ErrMethodNotFound) + }) + + return mux +} +``` + +## Usage Examples + +### Setup in Test Suite + +```go +func (suite *TaskmonTestSuite) SetupSuite() { + // Start in-memory JSON-RPC server mock (Level 1) + suite.jrpcServerMock = testutils.StartJrpcTraceServerMock() + + // Configure mock responses using DSL + suite.jrpcServerMock.AddMockResponse("protocol", struct { + Version string `json:"version"` + Date string `json:"date"` + }{ + Version: "3.18.0", + Date: "Sep-04-2018", + }) + + suite.jrpcServerMock.AddMockResponse("get_traces", struct { + Traces []string `json:"traces"` + }{ + Traces: []string{"trace1", "trace2"}, + }) + + // Configure your client to use the mock server + client := jrpc.NewClient(suite.jrpcServerMock.Address() + "/reader") +} + +func (suite *TaskmonTestSuite) TearDownSuite() { + suite.jrpcServerMock.Close() +} +``` + +### Test with Assertions + +```go +func (suite *TaskmonTestSuite) TestProtocolVersion() { + // Call your code that makes JSON-RPC requests + version, err := suite.taskmon.GetProtocolVersion() + suite.Require().NoError(err) + suite.Equal("3.18.0", version.Version) + + // Assert on received queries + queries := suite.jrpcServerMock.GetQueriesReceived() + suite.Require().Len(queries, 1) + suite.Equal("protocol", queries[0].Method) +} + +func (suite *TaskmonTestSuite) TestGetTraces() { + // Call your code + traces, err := suite.taskmon.GetTraces() + suite.Require().NoError(err) + suite.Equal([]string{"trace1", "trace2"}, traces) + + // Verify the right method was called + queries := suite.jrpcServerMock.GetQueriesReceived() + suite.Require().Len(queries, 2) // protocol + get_traces + suite.Equal("get_traces", queries[1].Method) +} +``` + +## Why This Pattern is Excellent + +1. **Rich DSL** - `AddMockResponse()` for easy, readable configuration +2. **Readable Setup** - Tests are self-documenting, clear intent +3. **In-Memory** - Uses `httptest.Server` (Level 1, no network I/O) +4. **Query Tracking** - `GetQueriesReceived()` for assertions on what was called +5. **OpenTelemetry Integration** - Realistic trace propagation for observability testing +6. **Idempotent Cleanup** - Safe to call `Close()` multiple times +7. **Flexible** - Configure any method/response combination dynamically + +## Key Design Principles + +### DSL for Configuration + +Mock setup should read like configuration: +```go +mock.AddMockResponse("method_name", expectedResponse) +mock.AddMockResponse("another_method", anotherResponse) +``` + +### Query Tracking for Assertions + +Always track what was received: +- Method names called +- Parameters passed +- Order of calls +- Number of calls + +### Built on httptest.Server + +httptest.Server provides: +- In-memory HTTP (no network I/O) +- Automatic address allocation +- Clean lifecycle management +- Standard library, no dependencies + +## Pattern Comparison + +| Pattern | Use When | +|---------|----------| +| **httptest.Server** | Simple HTTP mocking | +| **NATS test harness** | Need real NATS (pub/sub) | +| **gRPC client mock** | Testing gRPC **server** | +| **JSON-RPC server mock** | Testing JSON-RPC **client** | + +## Benefits + +- **In-Memory** - No network I/O, pure Go +- **Fast** - Microsecond startup time +- **Configurable** - Dynamic response configuration per test +- **Trackable** - Full visibility into received requests +- **OpenTelemetry-aware** - Realistic trace propagation +- **Reusable** - Same infrastructure across test levels + +## Key Takeaways + +1. **Mock servers should have rich DSL** - Makes setup readable +2. **Track received requests** - Essential for assertions +3. **Use httptest.Server** - Perfect for HTTP-based protocols +4. **Make setup read like configuration** - Self-documenting tests +5. **Support trace propagation** - Realistic observability testing +6. **Idempotent cleanup** - Safe resource management diff --git a/skills/testing/examples/nats-in-memory.md b/skills/testing/examples/nats-in-memory.md new file mode 100644 index 0000000..9ae05a9 --- /dev/null +++ b/skills/testing/examples/nats-in-memory.md @@ -0,0 +1,175 @@ +# NATS In-Memory Test Server + +## When to Use This Example + +Use this when: +- Testing message queue integrations with NATS +- Need pub/sub functionality in tests +- Want fast, in-memory NATS server (no Docker, no binary) +- Testing event-driven architectures + +**Dependency Level**: Level 1 (In-Memory) - Pure Go, official test harness + +## Implementation + +### Setup Test Infrastructure + +Many official SDKs provide test harnesses. Here's NATS: + +```go +// internal/testutils/nats.go +package testutils + +import ( + nserver "github.com/nats-io/nats-server/v2/server" + natsserver "github.com/nats-io/nats-server/v2/test" + "github.com/projectdiscovery/freeport" +) + +// RunNATsServer runs a NATS server in-memory for testing. +// Uses the official NATS SDK test harness - no binary download needed! +func RunNATsServer() (*nserver.Server, error) { + opts := natsserver.DefaultTestOptions + + // Allocate free port to prevent conflicts in parallel tests + tcpPort, err := freeport.GetFreePort("127.0.0.1", freeport.TCP) + if err != nil { + return nil, err + } + + opts.Port = tcpPort.Port + + // Start NATS server in-memory (pure Go!) + return natsserver.RunServer(&opts), nil +} + +// RunNATsServerWithJetStream runs NATS with JetStream enabled +func RunNATsServerWithJetStream() (*nserver.Server, error) { + opts := natsserver.DefaultTestOptions + + tcpPort, err := freeport.GetFreePort("127.0.0.1", freeport.TCP) + if err != nil { + return nil, err + } + + opts.Port = tcpPort.Port + opts.JetStream = true + + return natsserver.RunServer(&opts), nil +} +``` + +## Usage in Integration Tests + +### Basic Pub/Sub Test + +```go +//go:build integration + +package integration_test + +import ( + "context" + "testing" + "time" + "github.com/nats-io/nats.go" + "github.com/stretchr/testify/require" + "myproject/internal/testutils" +) + +func TestNATSPubSub_Integration(t *testing.T) { + // Start NATS server in-memory (Level 1 - pure Go!) + natsServer, err := testutils.RunNATsServer() + require.NoError(t, err) + defer natsServer.Shutdown() + + // Connect to in-memory NATS + natsAddress := "nats://" + natsServer.Addr().String() + nc, err := nats.Connect(natsAddress) + require.NoError(t, err) + defer nc.Close() + + // Test pub/sub + received := make(chan string, 1) + _, err = nc.Subscribe("test.subject", func(msg *nats.Msg) { + received <- string(msg.Data) + }) + require.NoError(t, err) + + // Publish message + err = nc.Publish("test.subject", []byte("hello")) + require.NoError(t, err) + + // Wait for message + select { + case msg := <-received: + require.Equal(t, "hello", msg) + case <-time.After(1 * time.Second): + t.Fatal("timeout waiting for message") + } +} +``` + +### Real-World Usage Example (gRPC + NATS) + +```go +// tests/gointegration/remote_traces_test.go +type RemoteTracesTestSuite struct { + suite.Suite + natsServer *nserver.Server + natsAddress string + nc *nats.Conn + // ... other fields +} + +func (suite *RemoteTracesTestSuite) SetupSuite() { + // Start NATS server in-memory + natsServer, err := testutils.RunNATsServer() + suite.Require().NoError(err) + + suite.natsServer = natsServer + suite.natsAddress = "nats://" + natsServer.Addr().String() + + // Connect application to in-memory NATS + suite.nc, err = natsremotetraces.ConnectToRemoteTracesSession( + suite.ctx, suite.natsAddress, numWorkers, numWorkers, channelSize) + suite.Require().NoError(err) + + // Start gRPC server with NATS backend + // ... rest of setup +} + +func (suite *RemoteTracesTestSuite) TearDownSuite() { + suite.nc.Close() + suite.natsServer.Shutdown() // Clean shutdown +} + +func (suite *RemoteTracesTestSuite) TestMessageFlow() { + // Test your application logic that uses NATS + // ... +} +``` + +## Why This is Excellent + +- **Pure Go** - NATS server imported as library (no binary download) +- **Official** - Uses NATS SDK's official test harness +- **Fast** - Starts in microseconds +- **Reliable** - Same behavior as production NATS +- **Portable** - Works anywhere Go runs +- **No Docker** - No external dependencies +- **Parallel-Safe** - Free port allocation prevents conflicts + +## Other Libraries with Test Harnesses + +- **Redis**: `github.com/alicebob/miniredis` - Pure Go in-memory Redis +- **NATS**: `github.com/nats-io/nats-server/v2/test` (shown above) +- **PostgreSQL**: `github.com/jackc/pgx/v5/pgxpool` with pgx mock +- **MongoDB**: `github.com/tryvium-travels/memongo` - In-memory MongoDB + +## Key Takeaways + +1. **Check for official test harnesses first** - Many popular libraries provide them +2. **Use free port allocation** - Prevents conflicts in parallel tests +3. **Clean shutdown** - Always call `Shutdown()` in teardown +4. **Reusable infrastructure** - Same setup for unit, integration, and system tests diff --git a/skills/testing/examples/system-patterns.md b/skills/testing/examples/system-patterns.md new file mode 100644 index 0000000..0583a7a --- /dev/null +++ b/skills/testing/examples/system-patterns.md @@ -0,0 +1,288 @@ +# System Test Patterns + +## Purpose + +System tests are black-box tests that verify the entire application works correctly from an external perspective. They test via CLI or API, simulating real user interactions. + +**Location**: `tests/` directory at project root (separate from package code) + +## Principles + +### Black Box Testing +- Test only via public interfaces (CLI, API) +- No access to internal packages +- Simulate real user behavior +- Test critical workflows end-to-end + +### Independence in Go +- Strive for pure Go tests (no Docker required) +- Use in-memory mocks from `testutils` +- Binary dependencies when needed +- Avoid docker-compose in CI + +## CLI Testing Patterns + +### Pattern 1: Simple Command Execution + +```go +// tests/cli_test.go +package tests + +import ( + "os/exec" + "strings" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestCLI_Version(t *testing.T) { + // Execute CLI command + cmd := exec.Command("./myapp", "version") + output, err := cmd.CombinedOutput() + + require.NoError(t, err) + require.Contains(t, string(output), "myapp version") +} + +func TestCLI_Help(t *testing.T) { + cmd := exec.Command("./myapp", "--help") + output, err := cmd.CombinedOutput() + + require.NoError(t, err) + require.Contains(t, string(output), "Usage:") +} +``` + +### Pattern 2: CLI with In-Memory Mocks + +```go +// tests/cli_metrics_test.go +package tests + +import ( + "context" + "os/exec" + "testing" + + "github.com/stretchr/testify/require" + "myproject/internal/testutils" +) + +func TestCLI_MetricsIngest(t *testing.T) { + // Start Victoria Metrics (Level 2 - binary) + vmServer, err := testutils.RunVictoriaMetricsServer() + require.NoError(t, err) + defer vmServer.Shutdown() + + // Test CLI against real Victoria Metrics + cmd := exec.Command("./myapp", "ingest", + "--metrics-url", vmServer.WriteURL(), + "--metric-name", "cli_test_metric", + "--value", "100") + + output, err := cmd.CombinedOutput() + require.NoError(t, err) + require.Contains(t, string(output), "Metric ingested successfully") + + // Verify with helpers + vmServer.ForceFlush(context.Background()) + results, err := testutils.QueryVictoriaMetrics(vmServer.QueryURL(), "cli_test_metric") + require.NoError(t, err) + require.Len(t, results, 1) +} +``` + +### Pattern 3: CLI with File System + +```go +// tests/cli_config_test.go +package tests + +import ( + "os" + "os/exec" + "path/filepath" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestCLI_ConfigFile(t *testing.T) { + // Create temp directory + tempDir := t.TempDir() + configPath := filepath.Join(tempDir, "config.yaml") + + // Write config file + configContent := ` +server: + port: 8080 + host: localhost +` + err := os.WriteFile(configPath, []byte(configContent), 0644) + require.NoError(t, err) + + // Test CLI with config file + cmd := exec.Command("./myapp", "start", "--config", configPath, "--dry-run") + output, err := cmd.CombinedOutput() + + require.NoError(t, err) + require.Contains(t, string(output), "Server would start on localhost:8080") +} +``` + +## API Testing Patterns + +### Pattern 1: HTTP API with In-Memory Mocks + +```go +// tests/api_test.go +package tests + +import ( + "bytes" + "encoding/json" + "io" + "net/http" + "os/exec" + "testing" + "time" + + "github.com/stretchr/testify/require" + "myproject/internal/testutils" +) + +func TestAPI_UserWorkflow(t *testing.T) { + // Start in-memory NATS (Level 1) + natsServer, err := testutils.RunNATsServer() + require.NoError(t, err) + defer natsServer.Shutdown() + + natsAddr := "nats://" + natsServer.Addr().String() + + // Start API server + cmd := exec.Command("./myapp", "serve", + "--port", "0", // Random free port + "--nats-url", natsAddr) + + // Start in background + err = cmd.Start() + require.NoError(t, err) + defer cmd.Process.Kill() + + // Wait for API to be ready + time.Sleep(500 * time.Millisecond) + + // Get actual port (from logs or endpoint) + apiURL := "http://localhost:8080" // Or parse from logs + + // Test API workflow + // 1. Create user + createReq := map[string]string{ + "name": "Alice", + "email": "alice@example.com", + } + body, _ := json.Marshal(createReq) + + resp, err := http.Post(apiURL+"/users", "application/json", bytes.NewBuffer(body)) + require.NoError(t, err) + defer resp.Body.Close() + require.Equal(t, http.StatusCreated, resp.StatusCode) + + // Parse response + var createResp map[string]string + json.NewDecoder(resp.Body).Decode(&createResp) + userID := createResp["id"] + + // 2. Retrieve user + resp, err = http.Get(apiURL + "/users/" + userID) + require.NoError(t, err) + defer resp.Body.Close() + require.Equal(t, http.StatusOK, resp.StatusCode) + + var user map[string]string + json.NewDecoder(resp.Body).Decode(&user) + require.Equal(t, "Alice", user["name"]) +} +``` + +## Architecture for Independence + +### Dependency Injection Pattern + +Design your application to accept dependency URLs: + +```go +// cmd/myapp/main.go +func main() { + // Allow overriding dependencies via flags + natsURL := flag.String("nats-url", "nats://localhost:4222", "NATS server URL") + metricsURL := flag.String("metrics-url", "http://localhost:8428", "Metrics server URL") + flag.Parse() + + // Use provided URLs (allows in-memory mocks in tests) + app := app.New(*natsURL, *metricsURL) + app.Run() +} +``` + +### Test with In-Memory Dependencies + +```go +// tests/app_test.go +func TestApp_WithMocks(t *testing.T) { + // Start all mocks + natsServer, _ := testutils.RunNATsServer() + defer natsServer.Shutdown() + + vmServer, _ := testutils.RunVictoriaMetricsServer() + defer vmServer.Shutdown() + + // Test app with mocked dependencies (pure Go, no Docker!) + cmd := exec.Command("./myapp", "serve", + "--nats-url", "nats://"+natsServer.Addr().String(), + "--metrics-url", vmServer.WriteURL()) + + // ... test application +} +``` + +## Running System Tests + +```bash +# Build application first +go build -o myapp ./cmd/myapp + +# Run system tests +go test -v ./tests/... + +# With coverage +go test -v -coverprofile=coverage.out ./tests/... + +# Specific test +go test -v ./tests/... -run TestCLI_MetricsIngest +``` + +## Best Practices + +### DO: +- Test via CLI/API only (black box) +- Use in-memory mocks from testutils +- Test critical end-to-end workflows +- Build binary before running tests +- Use temp directories for file operations + +### DON'T: +- Don't import internal packages +- Don't test every edge case (that's unit/integration tests) +- Don't require Docker in CI +- Don't use sleep for timing (use polling/channels) +- Don't skip cleanup + +## Key Takeaways + +1. **Black box only** - Test via public interfaces +2. **Independent in Go** - No Docker required +3. **Use testutils mocks** - Reuse infrastructure +4. **Test critical paths** - Not every scenario +5. **Fast execution** - Should run quickly in CI diff --git a/skills/testing/examples/test-organization.md b/skills/testing/examples/test-organization.md new file mode 100644 index 0000000..36be39b --- /dev/null +++ b/skills/testing/examples/test-organization.md @@ -0,0 +1,260 @@ +# Test Organization and File Structure + +## File Organization + +### Basic Structure + +``` +user/ +├── user.go +├── user_test.go # Unit tests for user.go +├── service.go +├── service_test.go # Unit tests for service.go +├── repository.go +└── repository_test.go # Unit tests for repository.go +``` + +### With Integration and System Tests + +``` +project/ +├── user/ +│ ├── user.go +│ ├── user_test.go # Unit tests (pkg_test) +│ ├── service.go +│ ├── service_test.go # Unit tests (pkg_test) +│ └── integration_test.go # Integration tests with //go:build integration +├── internal/ +│ └── testutils/ # Reusable test infrastructure +│ ├── nats.go # In-memory NATS server +│ ├── victoria.go # Victoria Metrics binary management +│ └── httpserver/ # HTTP mock DSL +│ ├── server.go +│ └── server_test.go # Test the infrastructure! +└── tests/ # System tests (black box) + ├── cli_test.go # CLI testing via exec.Command + └── api_test.go # API testing via HTTP client +``` + +## Package Naming + +### Use `pkg_test` for Unit Tests + +```go +// ✅ External package - tests public API only +package user_test + +import ( + "testing" + "github.com/yourorg/project/user" +) + +func TestService_CreateUser(t *testing.T) { + // Test through public API + svc, _ := user.NewUserService(repo, notifier) + err := svc.CreateUser(ctx, testUser) + // ... +} +``` + +### Avoid Same Package Testing + +```go +// ❌ Same package - can test private methods (don't do this) +package user + +import "testing" + +func TestInternalValidation(t *testing.T) { + // Testing private function - bad practice + result := validateEmailInternal("test@example.com") + // ... +} +``` + +## Build Tags for Integration Tests + +### Using Build Tags + +```go +//go:build integration + +package user_test + +import ( + "context" + "testing" + "myproject/internal/testutils" +) + +func TestUserService_Integration(t *testing.T) { + // Integration test with real dependencies + natsServer, _ := testutils.RunNATsServer() + defer natsServer.Shutdown() + + // Test with real NATS + // ... +} +``` + +### Running Tests + +```bash +# Run only unit tests (default - no build tags) +go test ./... + +# Run unit + integration tests +go test -tags=integration ./... + +# Run specific package integration tests +go test -tags=integration ./user + +# Run system tests only +go test ./tests/... + +# Run all tests +go test -tags=integration ./... +``` + +## Makefile/Taskfile Integration + +### Taskfile.yml Example + +```yaml +version: '3' + +tasks: + test: + desc: Run unit tests + cmds: + - go test -v -race ./... + + test:integration: + desc: Run integration tests + cmds: + - go test -v -race -tags=integration ./... + + test:system: + desc: Run system tests + cmds: + - go test -v -race ./tests/... + + test:all: + desc: Run all tests + cmds: + - task: test:integration + - task: test:system + + test:coverage: + desc: Run tests with coverage + cmds: + - go test -v -race -coverprofile=coverage.out ./... + - go tool cover -html=coverage.out -o coverage.html +``` + +### Makefile Example + +```makefile +.PHONY: test test-integration test-system test-all coverage + +test: + go test -v -race ./... + +test-integration: + go test -v -race -tags=integration ./... + +test-system: + go test -v -race ./tests/... + +test-all: test-integration test-system + +coverage: + go test -v -race -coverprofile=coverage.out ./... + go tool cover -html=coverage.out -o coverage.html +``` + +## Test File Naming + +### Unit Tests +- `*_test.go` - Standard test files +- Located next to the code being tested +- Use `pkg_test` package name + +### Integration Tests +- `integration_test.go` or `*_integration_test.go` +- Use `//go:build integration` tag +- Can be in same directory or separate `integration/` folder +- Use `pkg_test` package name + +### System Tests +- `*_test.go` in `tests/` directory at project root +- No build tags needed (separate directory) +- Use `tests` or `main_test` package name + +## CI/CD Integration + +### GitHub Actions Example + +```yaml +name: Tests + +on: [push, pull_request] + +jobs: + unit-tests: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-go@v4 + with: + go-version: '1.21' + - name: Run unit tests + run: go test -v -race ./... + + integration-tests: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-go@v4 + with: + go-version: '1.21' + - name: Run integration tests + run: go test -v -race -tags=integration ./... + + system-tests: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-go@v4 + with: + go-version: '1.21' + - name: Build application + run: go build -o myapp ./cmd/myapp + - name: Run system tests + run: go test -v -race ./tests/... +``` + +## testutils Package Structure + +``` +internal/testutils/ +├── nats.go # NATS in-memory server helpers +├── victoria.go # Victoria Metrics binary management +├── prometheus.go # Prometheus payload helpers +├── grpc_client_mock.go # gRPC client mock with DSL +├── jrpc_server_mock.go # JSON-RPC server mock with DSL +└── httpserver/ # HTTP mock server with DSL + ├── server.go + ├── server_test.go # Test the infrastructure! + ├── dsl.go + └── README.md +``` + +## Key Principles + +1. **Co-locate unit tests** - Next to the code being tested +2. **Use pkg_test package** - Forces public API testing +3. **Build tags for integration** - Keep unit tests fast by default +4. **Separate system tests** - In `tests/` directory +5. **Test your test infrastructure** - Treat testutils as production code +6. **Reusable infrastructure** - Share across all test levels diff --git a/skills/testing/examples/victoria-metrics.md b/skills/testing/examples/victoria-metrics.md new file mode 100644 index 0000000..fe1dc7e --- /dev/null +++ b/skills/testing/examples/victoria-metrics.md @@ -0,0 +1,566 @@ +# Victoria Metrics Binary Test Server + +## When to Use This Example + +Use this when: +- Testing Prometheus Remote Write integrations +- Need real Victoria Metrics for testing metrics ingestion +- Testing PromQL queries +- Want production-like behavior without Docker +- Testing metrics pipelines end-to-end + +**Dependency Level**: Level 2 (Binary) - Standalone executable via `exec.Command` + +**Why Binary Instead of In-Memory:** +- Victoria Metrics is complex; reimplementing as in-memory mock isn't practical +- Need real PromQL engine behavior +- Need actual data persistence and querying +- Binary startup is fast (< 1 second) and requires no Docker + +## Implementation + +### Victoria Server Infrastructure + +This example shows how to download, manage, and run Victoria Metrics binary for testing: + +```go +// internal/testutils/victoria.go +package testutils + +import ( + "context" + "errors" + "fmt" + "net/http" + "os" + "os/exec" + "path/filepath" + "runtime" + "sync" + "time" + + "github.com/projectdiscovery/freeport" +) + +const ( + DefaultVictoriaMetricsVersion = "v1.128.0" + VictoriaMetricsVersionEnvVar = "TEST_VICTORIA_METRICS_VERSION" +) + +var ( + ErrVictoriaMetricsNotHealthy = errors.New("victoria metrics did not become healthy") + ErrDownloadFailed = errors.New("download failed") + + // binaryDownloadMu protects concurrent downloads (prevent race conditions) + binaryDownloadMu sync.Mutex +) + +// VictoriaServer represents a running Victoria Metrics test instance +type VictoriaServer struct { + cmd *exec.Cmd + port int + dataPath string + writeURL string + queryURL string + version string + binaryPath string + shutdownOnce sync.Once + shutdownErr error +} + +// WriteURL returns the URL for writing metrics (Prometheus Remote Write endpoint) +func (vs *VictoriaServer) WriteURL() string { + return vs.writeURL +} + +// QueryURL returns the URL for querying metrics (Prometheus-compatible query endpoint) +func (vs *VictoriaServer) QueryURL() string { + return vs.queryURL +} + +// Port returns the port Victoria Metrics is listening on +func (vs *VictoriaServer) Port() int { + return vs.port +} + +// ForceFlush forces Victoria Metrics to flush buffered samples from memory to disk, +// making them immediately queryable. This is useful for testing to avoid waiting +// for the automatic flush cycle. +func (vs *VictoriaServer) ForceFlush(ctx context.Context) error { + url := fmt.Sprintf("http://localhost:%d/internal/force_flush", vs.port) + req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) + if err != nil { + return fmt.Errorf("failed to create force flush request: %w", err) + } + + resp, err := http.DefaultClient.Do(req) + if err != nil { + return fmt.Errorf("failed to force flush: %w", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + return fmt.Errorf("force flush failed: status %d", resp.StatusCode) + } + + return nil +} + +// Shutdown stops Victoria Metrics and cleans up resources. +// Safe to call multiple times (idempotent). +func (vs *VictoriaServer) Shutdown() error { + vs.shutdownOnce.Do(func() { + if vs.cmd == nil || vs.cmd.Process == nil { + return + } + + // Send interrupt signal for graceful shutdown + if err := vs.cmd.Process.Signal(os.Interrupt); err != nil { + vs.shutdownErr = err + return + } + + // Wait for process to exit (with timeout) + done := make(chan error, 1) + go func() { + done <- vs.cmd.Wait() + }() + + select { + case <-time.After(5 * time.Second): + vs.cmd.Process.Kill() + vs.shutdownErr = errors.New("shutdown timeout") + case err := <-done: + if err != nil && err.Error() != "signal: interrupt" { + vs.shutdownErr = err + } + } + + // Cleanup data directory + if vs.dataPath != "" { + os.RemoveAll(vs.dataPath) + } + }) + return vs.shutdownErr +} + +// RunVictoriaMetricsServer starts a Victoria Metrics instance for testing. +// It downloads the binary if needed, starts the server, and waits for it to be healthy. +func RunVictoriaMetricsServer() (*VictoriaServer, error) { + version := getVictoriaMetricsVersion() + + // Ensure binary exists (downloads if missing) + binaryPath, err := ensureVictoriaBinary(version) + if err != nil { + return nil, err + } + + // Get free port (prevents conflicts in parallel tests) + freePort, err := freeport.GetFreePort("127.0.0.1", freeport.TCP) + if err != nil { + return nil, fmt.Errorf("failed to get free port: %w", err) + } + port := freePort.Port + + // Create temporary data directory + dataPath, err := os.MkdirTemp("", "victoria-metrics-test-*") + if err != nil { + return nil, fmt.Errorf("failed to create temp directory: %w", err) + } + + // Start Victoria Metrics + cmd := exec.Command( + binaryPath, + fmt.Sprintf("-httpListenAddr=:%d", port), + "-storageDataPath="+dataPath, + "-retentionPeriod=1d", + "-inmemoryDataFlushInterval=1ms", // Force immediate data flush for testing + ) + + cmd.Stdout = os.Stdout + cmd.Stderr = os.Stderr + + if err := cmd.Start(); err != nil { + os.RemoveAll(dataPath) + return nil, fmt.Errorf("failed to start victoria metrics: %w", err) + } + + baseURL := fmt.Sprintf("http://localhost:%d", port) + server := &VictoriaServer{ + cmd: cmd, + port: port, + dataPath: dataPath, + writeURL: baseURL + "/api/v1/write", + queryURL: baseURL + "/api/v1/query", + version: version, + binaryPath: binaryPath, + } + + // Wait for server to become healthy + if err := waitForHealth(baseURL); err != nil { + server.Shutdown() + return nil, err + } + + return server, nil +} + +func getVictoriaMetricsVersion() string { + if version := os.Getenv(VictoriaMetricsVersionEnvVar); version != "" { + return version + } + return DefaultVictoriaMetricsVersion +} + +// ensureVictoriaBinary ensures the Victoria Metrics binary exists, downloading if necessary. +// Thread-safe with double-check locking to prevent race conditions. +func ensureVictoriaBinary(version string) (string, error) { + binaryName := fmt.Sprintf("victoria-metrics-%s-%s-%s", version, runtime.GOOS, getVMArch()) + binaryPath := filepath.Join(".bin", binaryName) + + // Quick check without lock (optimization) + if _, err := os.Stat(binaryPath); err == nil { + return binaryPath, nil + } + + // Acquire lock to prevent concurrent downloads + binaryDownloadMu.Lock() + defer binaryDownloadMu.Unlock() + + // Double-check after acquiring lock (another goroutine might have downloaded it) + if _, err := os.Stat(binaryPath); err == nil { + return binaryPath, nil + } + + // Create .bin directory + if err := os.MkdirAll(".bin", 0755); err != nil { + return "", fmt.Errorf("failed to create .bin directory: %w", err) + } + + // Download to temporary location with unique name + tempPath := fmt.Sprintf("%s.tmp.%d", binaryPath, os.Getpid()) + defer os.Remove(tempPath) + + downloadURL := fmt.Sprintf( + "https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/%s/victoria-metrics-%s-%s-%s.tar.gz", + version, runtime.GOOS, getVMArch(), version, + ) + + if err := downloadAndExtract(downloadURL, tempPath); err != nil { + return "", fmt.Errorf("failed to download: %w", err) + } + + if err := os.Chmod(tempPath, 0755); err != nil { + return "", fmt.Errorf("failed to make binary executable: %w", err) + } + + // Atomic rename - only one goroutine succeeds if multiple try + if err := os.Rename(tempPath, binaryPath); err != nil { + // If rename fails, check if another goroutine succeeded + if _, statErr := os.Stat(binaryPath); statErr == nil { + return binaryPath, nil // Another goroutine won the race + } + return "", fmt.Errorf("failed to rename binary: %w", err) + } + + return binaryPath, nil +} + +func getVMArch() string { + switch runtime.GOARCH { + case "amd64": + return "amd64" + case "arm64": + return "arm64" + default: + return runtime.GOARCH + } +} + +func waitForHealth(baseURL string) error { + healthURL := baseURL + "/health" + maxRetries := 30 + retryInterval := time.Second + + ctx := context.Background() + for range maxRetries { + req, err := http.NewRequestWithContext(ctx, http.MethodGet, healthURL, nil) + if err != nil { + time.Sleep(retryInterval) + continue + } + + resp, err := http.DefaultClient.Do(req) + if err == nil { + statusOK := resp.StatusCode == http.StatusOK + resp.Body.Close() + if statusOK { + return nil + } + } + + time.Sleep(retryInterval) + } + + return ErrVictoriaMetricsNotHealthy +} +``` + +### Helper Functions for Prometheus/Victoria Metrics Testing + +Add practical helpers that make tests clear and maintainable: + +```go +// internal/testutils/prometheus.go +package testutils + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "net/http" + "net/url" + "testing" + "time" + + "github.com/gogo/protobuf/proto" + "github.com/golang/snappy" + "github.com/prometheus/prometheus/prompb" + "github.com/stretchr/testify/require" +) + +var ( + ErrQueryFailed = errors.New("victoria metrics query failed") + ErrQueryNonSuccess = errors.New("query returned non-success status") +) + +// CreatePrometheusPayload creates a valid Prometheus Remote Write payload +// with a sample metric. The payload is protobuf-encoded and snappy-compressed, +// ready to be sent to Victoria Metrics' /api/v1/write endpoint. +func CreatePrometheusPayload(metricName string, value float64, labels map[string]string) ([]byte, error) { + // Create timestamp (current time in milliseconds) + timestampMs := time.Now().UnixMilli() + + // Build label pairs + labelPairs := make([]prompb.Label, 0, len(labels)+1) + labelPairs = append(labelPairs, prompb.Label{ + Name: "__name__", + Value: metricName, + }) + for name, val := range labels { + labelPairs = append(labelPairs, prompb.Label{ + Name: name, + Value: val, + }) + } + + // Create a single time series with one sample + timeseries := []prompb.TimeSeries{ + { + Labels: labelPairs, + Samples: []prompb.Sample{ + { + Value: value, + Timestamp: timestampMs, + }, + }, + }, + } + + // Create WriteRequest + writeRequest := &prompb.WriteRequest{ + Timeseries: timeseries, + } + + // Marshal to protobuf + data, err := proto.Marshal(writeRequest) + if err != nil { + return nil, fmt.Errorf("failed to marshal protobuf: %w", err) + } + + // Compress with snappy + compressed := snappy.Encode(nil, data) + + return compressed, nil +} + +// VMQueryResult represents a single result from a Victoria Metrics query. +type VMQueryResult struct { + Metric map[string]string // label name -> label value + Value []any // [timestamp, value_string] +} + +// VMQueryResponse represents the full Victoria Metrics API response. +type VMQueryResponse struct { + Status string `json:"status"` + Data struct { + ResultType string `json:"result_type"` + Result []VMQueryResult `json:"result"` + } `json:"data"` +} + +// QueryVictoriaMetrics executes a PromQL query against Victoria Metrics. +// The query is performed via the /api/v1/query endpoint with time buffer +// for clock skew and delayed indexing. +func QueryVictoriaMetrics(queryURL, query string) ([]VMQueryResult, error) { + // Query with current time + 1 minute to catch any clock skew or delayed indexing + currentTime := time.Now().Add(1 * time.Minute) + fullURL := fmt.Sprintf("%s?query=%s&time=%d", queryURL, url.QueryEscape(query), currentTime.Unix()) + + // Execute HTTP request + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + + req, err := http.NewRequestWithContext(ctx, http.MethodGet, fullURL, nil) + if err != nil { + return nil, fmt.Errorf("failed to create request: %w", err) + } + + resp, err := http.DefaultClient.Do(req) + if err != nil { + return nil, fmt.Errorf("failed to execute query: %w", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + return nil, fmt.Errorf("%w: %s", ErrQueryFailed, resp.Status) + } + + // Read response body + bodyBytes, err := io.ReadAll(resp.Body) + if err != nil { + return nil, fmt.Errorf("failed to read response body: %w", err) + } + + // Parse JSON response + var queryResp VMQueryResponse + if err := json.Unmarshal(bodyBytes, &queryResp); err != nil { + return nil, fmt.Errorf("failed to decode response: %w", err) + } + + if queryResp.Status != "success" { + return nil, fmt.Errorf("%w: %s", ErrQueryNonSuccess, queryResp.Status) + } + + return queryResp.Data.Result, nil +} + +// AssertLabelExists checks if at least one result contains a label with the given name and value. +// Fails the test if the label is not found. +func AssertLabelExists(t *testing.T, results []VMQueryResult, labelName, labelValue string) { + t.Helper() + + for _, result := range results { + if val, exists := result.Metric[labelName]; exists && val == labelValue { + return // Found it! + } + } + + // Label not found - fail with helpful message + require.Fail(t, "Label not found", + "Expected to find label %s=%s in query results, but it was not present", + labelName, labelValue) +} +``` + +## Usage Examples + +### Integration Test + +```go +// internal/api/stats/prometheus_ingest_test.go +func TestPrometheusIngest_WithVictoriaMetrics(t *testing.T) { + // Start real Victoria Metrics server (Level 2) + vmServer, err := testutils.RunVictoriaMetricsServer() + require.NoError(t, err) + defer vmServer.Shutdown() + + // Create valid Prometheus payload using helper + payload, err := testutils.CreatePrometheusPayload("test_metric", 42.0, map[string]string{ + "service": "api", + "env": "test", + }) + require.NoError(t, err) + + // Send to Victoria Metrics + req := httptest.NewRequest(http.MethodPost, vmServer.WriteURL(), bytes.NewBuffer(payload)) + req.Header.Set("Content-Type", "application/x-protobuf") + req.Header.Set("Content-Encoding", "snappy") + + resp, err := http.DefaultClient.Do(req) + require.NoError(t, err) + defer resp.Body.Close() + + assert.Equal(t, http.StatusNoContent, resp.StatusCode) + + // Force flush to make data queryable immediately + err = vmServer.ForceFlush(context.Background()) + require.NoError(t, err) + + // Query using helper + results, err := testutils.QueryVictoriaMetrics(vmServer.QueryURL(), `test_metric{service="api"}`) + require.NoError(t, err) + require.Len(t, results, 1) + + // Assert using helper + testutils.AssertLabelExists(t, results, "env", "test") +} +``` + +### System Test + +```go +// tests/prometheus_ingestion_test.go +func TestE2E_PrometheusIngestion(t *testing.T) { + // Same Victoria Metrics infrastructure! + vmServer, err := testutils.RunVictoriaMetricsServer() + require.NoError(t, err) + defer vmServer.Shutdown() + + // Test CLI against real Victoria Metrics + cmd := exec.Command("./myapp", "ingest", + "--metrics-url", vmServer.WriteURL(), + "--metric-name", "cli_test_metric", + "--value", "100") + + output, err := cmd.CombinedOutput() + require.NoError(t, err) + assert.Contains(t, string(output), "Metric ingested successfully") + + // Verify with helpers + vmServer.ForceFlush(context.Background()) + results, err := testutils.QueryVictoriaMetrics(vmServer.QueryURL(), "cli_test_metric") + require.NoError(t, err) + require.Len(t, results, 1) +} +``` + +## Key Features + +- **Binary download with OS/arch detection** - Works on macOS/Linux, amd64/arm64 +- **Thread-safe download** - Mutex + double-check locking prevents race conditions +- **Free port allocation** - Prevents conflicts in parallel tests +- **Idempotent shutdown** - Safe to call multiple times with `sync.Once` +- **Resource cleanup** - Proper temp directory and process cleanup +- **Helper functions** - `ForceFlush()` for immediate data availability +- **Prometheus helpers** - Create payloads, query, assert on results + +## Benefits + +- **Production-like testing** - Testing against REAL Victoria Metrics, not mocks +- **Reusable** - Same `testutils` infrastructure for unit, integration, and system tests +- **Readable** - Helper functions make tests read like documentation +- **No Docker** - No Docker required, works in any environment +- **Fast** - Binary starts in < 1 second +- **Portable** - Works anywhere Go runs +- **Maintainable** - Changes to test infrastructure are centralized + +## Key Takeaways + +1. **Binary level is good for complex services** - When in-memory is too complex +2. **Download management is critical** - Thread-safe, cached, version-controlled +3. **Helper functions make tests readable** - DSL for common operations +4. **Reuse across test levels** - Same infrastructure for unit, integration, system +5. **Force flush is essential** - Make data immediately queryable in tests diff --git a/skills/testing/reference.md b/skills/testing/reference.md new file mode 100644 index 0000000..bb58723 --- /dev/null +++ b/skills/testing/reference.md @@ -0,0 +1,689 @@ +# Testing Reference + +Complete guide to Go testing principles and patterns. + +## Core Testing Principles + +### 1. Test Only Public API +- **Use `pkg_test` package name** - Forces external perspective +- **Test types via constructors** - No direct struct initialization +- **No testing private methods** - If you need to test it, make it public or rethink design + +```go +// ✅ Good +package user_test + +import "github.com/yourorg/project/user" + +func TestService_CreateUser(t *testing.T) { + svc, _ := user.NewUserService(repo, notifier) + err := svc.CreateUser(ctx, testUser) + // ... +} +``` + +### 2. Avoid Mocks - Use Real Implementations + +Instead of mocks, use: +- **HTTP test servers** (`httptest` package) +- **Temp files/directories** (`os.CreateTemp`, `os.MkdirTemp`) +- **In-memory databases** (SQLite in-memory, or custom implementations) +- **Test implementations** (TestEmailer that writes to buffer) + +**Benefits:** +- Tests are more reliable +- Tests verify actual behavior +- Easier to maintain + +### 3. Coverage Strategy + +**Leaf Types** (self-contained): +- **Target**: 100% unit test coverage +- **Why**: Core logic must be bulletproof + +**Orchestrating Types** (coordinate others): +- **Target**: Integration test coverage +- **Why**: Test seams between components + +**Goal**: Most logic in leaf types (easier to test and maintain) + +--- + +## Table-Driven Tests + +### When to Use +- Each test case has **cyclomatic complexity = 1** +- No conditionals inside t.Run() +- Simple, focused testing scenarios + +### ❌ Anti-Pattern: wantErr bool + +**DO NOT** use `wantErr bool` pattern - it violates complexity = 1 rule: + +```go +// ❌ BAD - Has conditionals (complexity > 1) +func TestNewUserID(t *testing.T) { + tests := []struct { + name string + input string + want UserID + wantErr bool // ❌ Anti-pattern + }{ + {name: "valid ID", input: "usr_123", want: UserID("usr_123"), wantErr: false}, + {name: "empty ID", input: "", wantErr: true}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := NewUserID(tt.input) + if tt.wantErr { // ❌ Conditional + assert.Error(t, err) + return + } + assert.NoError(t, err) + assert.Equal(t, tt.want, got) + }) + } +} +``` + +### ✅ Correct Pattern: Separate Functions + +**Always separate success and error cases:** + +```go +// ✅ Success cases - Complexity = 1 +func TestNewUserID_Success(t *testing.T) { + tests := []struct { + name string + input string + want UserID + }{ + {name: "valid ID", input: "usr_123", want: UserID("usr_123")}, + {name: "with numbers", input: "usr_456", want: UserID("usr_456")}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := NewUserID(tt.input) + require.NoError(t, err) // ✅ No conditionals + assert.Equal(t, tt.want, got) + }) + } +} + +// ✅ Error cases - Complexity = 1 +func TestNewUserID_Error(t *testing.T) { + tests := []struct { + name string + input string + }{ + {name: "empty ID", input: ""}, + {name: "whitespace only", input: " "}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + _, err := NewUserID(tt.input) + assert.Error(t, err) // ✅ No conditionals + }) + } +} +``` + +### Critical Rule: Named Struct Fields + +**ALWAYS use named struct fields** - Linter reorders fields, breaking unnamed initialization: + +```go +// ❌ BAD - Breaks when linter reorders fields +tests := []struct { + name string + input int + want string +}{ + {"test1", 42, "result"}, // Will break +} + +// ✅ GOOD - Works regardless of field order +tests := []struct { + name string + input int + want string +}{ + {name: "test1", input: 42, want: "result"}, // Always works +} +``` + +--- + +## Testify Suites + +### When to Use + +ONLY for complex test infrastructure setup: +- Mock HTTP servers +- Database connections +- OpenTelemetry testing setup +- Temporary files/directories needing cleanup +- Shared expensive setup/teardown + +### When NOT to Use +- Simple unit tests (use table-driven instead) +- Tests without complex setup + +### Pattern + +```go +package user_test + +import ( + "net/http/httptest" + "testing" + "github.com/stretchr/testify/suite" +) + +type ServiceSuite struct { + suite.Suite + server *httptest.Server + svc *user.UserService +} + +func (s *ServiceSuite) SetupSuite() { + s.server = httptest.NewServer(testHandler) +} + +func (s *ServiceSuite) TearDownSuite() { + s.server.Close() +} + +func (s *ServiceSuite) SetupTest() { + s.svc = user.NewUserService(s.server.URL) +} + +func (s *ServiceSuite) TestCreateUser() { + err := s.svc.CreateUser(ctx, testUser) + s.NoError(err) +} + +func TestServiceSuite(t *testing.T) { + suite.Run(t, new(ServiceSuite)) +} +``` + +--- + +## Synchronization in Tests + +### Never Use time.Sleep + +Use channels or WaitGroups instead. + +### Use Channels + +```go +func TestAsyncOperation(t *testing.T) { + done := make(chan struct{}) + + go func() { + doAsyncWork() + close(done) + }() + + select { + case <-done: + // Success + case <-time.After(1 * time.Second): + t.Fatal("timeout") + } +} +``` + +### Use WaitGroups + +```go +func TestConcurrentOperations(t *testing.T) { + var wg sync.WaitGroup + results := make([]string, 10) + + for i := 0; i < 10; i++ { + wg.Add(1) + go func(index int) { + defer wg.Done() + results[index] = doWork(index) + }(i) + } + + wg.Wait() + // Assert on results +} +``` + +--- + +## Test Organization + +### File Structure + +``` +user/ +├── user.go +├── user_test.go # Tests for user.go (pkg_test) +├── service.go +├── service_test.go # Tests for service.go (pkg_test) +``` + +### Package Naming + +```go +// ✅ External package - tests public API only +package user_test + +import ( + "testing" + "github.com/yourorg/project/user" +) +``` + +--- + +## Real Implementation Patterns + +### In-Memory Repository + +```go +package user + +type InMemoryRepository struct { + mu sync.RWMutex + users map[UserID]User +} + +func NewInMemoryRepository() *InMemoryRepository { + return &InMemoryRepository{ + users: make(map[UserID]User), + } +} + +func (r *InMemoryRepository) Save(ctx context.Context, u User) error { + r.mu.Lock() + defer r.mu.Unlock() + r.users[u.ID] = u + return nil +} + +func (r *InMemoryRepository) Get(ctx context.Context, id UserID) (*User, error) { + r.mu.RLock() + defer r.mu.RUnlock() + u, ok := r.users[id] + if !ok { + return nil, ErrNotFound + } + return &u, nil +} +``` + +### Test Email Sender + +```go +package user + +import ( + "bytes" + "fmt" + "sync" +) + +type TestEmailer struct { + mu sync.Mutex + buffer bytes.Buffer +} + +func NewTestEmailer() *TestEmailer { + return &TestEmailer{} +} + +func (e *TestEmailer) Send(to Email, subject, body string) error { + e.mu.Lock() + defer e.mu.Unlock() + fmt.Fprintf(&e.buffer, "To: %s\nSubject: %s\n%s\n\n", to, subject, body) + return nil +} + +func (e *TestEmailer) SentEmails() string { + e.mu.Lock() + defer e.mu.Unlock() + return e.buffer.String() +} +``` + +--- + +## Testable Examples (GoDoc Examples) + +### When to Add +- Non-trivial types +- Types with validation +- Common usage patterns + +### Pattern + +```go +// Example_UserID demonstrates basic usage. +func Example_UserID() { + id, _ := user.NewUserID("usr_123") + fmt.Println(id) + // Output: usr_123 +} + +// Example_UserID_validation shows validation behavior. +func Example_UserID_validation() { + _, err := user.NewUserID("") + fmt.Println(err != nil) + // Output: true +} +``` + +--- + +## Testing Checklist + +### Before Considering Tests Complete + +**Structure:** +- [ ] Tests in `pkg_test` package +- [ ] Testing public API only +- [ ] Table-driven tests use named fields +- [ ] No conditionals in test cases + +**Implementation:** +- [ ] Using real implementations, not mocks +- [ ] No time.Sleep (using channels/waitgroups) +- [ ] Testify suites only for complex setup + +**Coverage:** +- [ ] Leaf types: 100% unit test coverage +- [ ] Orchestrating types: Integration tests +- [ ] Happy path, edge cases, error cases covered + +--- + +## Summary + +**The Golden Rule**: Cyclomatic complexity = 1 in all test cases + +**Test Structure Choices:** +- **Table-driven tests**: Simple, focused scenarios +- **Testify suites**: Complex infrastructure setup only + +**Test Philosophy:** +- Test only public API (`pkg_test` package) +- Use real implementations, not mocks +- Leaf types: 100% coverage +- Orchestrating types: Integration tests + +**Common Pitfalls to Avoid:** +- ❌ Testing private methods +- ❌ Heavy mocking +- ❌ time.Sleep in tests +- ❌ Conditionals in test cases +- ❌ Unnamed struct fields in table tests + +--- + +# Example Files - Reusable Testing Patterns + +The following example files contain **transferable patterns** that apply to many scenarios, not just the specific technologies shown. Claude should read these files based on the **pattern needed**, not the specific technology mentioned. + +## Pattern 1: In-Memory Test Harness (Level 1) + +**File**: `examples/nats-in-memory.md` + +**Pattern**: Using official test harnesses from Go libraries + +**When to read:** +- Need to test with ANY service that provides an official Go test harness +- Testing message queues, databases, caches, or any service with in-memory test mode +- Want to avoid Docker but need realistic service behavior + +**Applies to:** +- **NATS** (shown in example) - Message queue with official test harness +- **Redis** - `github.com/alicebob/miniredis` pure Go in-memory Redis +- **MongoDB** - `github.com/tryvium-travels/memongo` in-memory MongoDB +- **PostgreSQL** - `github.com/jackc/pgx/v5` with pgx mock +- **Any Go library with test package** - Check if dependency has `/test` package + +**Key techniques to adapt:** +- Wrapping official harness with clean API +- Free port allocation for parallel tests +- Clean lifecycle management (Setup/Teardown) +- Thread-safe initialization + +--- + +## Pattern 2: Binary Dependency Management (Level 2) + +**File**: `examples/victoria-metrics.md` + +**Pattern**: Download, manage, and run ANY standalone binary for testing + +**When to read:** +- Need to test against ANY external binary executable +- No in-memory option available +- Want production-like testing without Docker + +**Applies to:** +- **Victoria Metrics** (shown in example) - Metrics database +- **Prometheus** - Metrics and alerting +- **Grafana** - Dashboards and visualization +- **Any database binaries** - PostgreSQL, MySQL, Redis, etc. +- **Any CLI tools** - Language servers, formatters, linters +- **Custom binaries** - Your own services or third-party tools + +**Key techniques to adapt:** +- OS/ARCH detection (`runtime.GOOS`, `runtime.GOARCH`) +- Thread-safe binary downloads with double-check locking +- Health check polling with retries +- Graceful shutdown with `sync.Once` +- Free port allocation +- Temp directory management +- Version management via environment variables + +--- + +## Pattern 3: Mock Server with Generic DSL (Level 1) + +**File**: `examples/jsonrpc-mock.md` + +**Pattern**: Building generic mock servers with configurable responses using `AddMockResponse()` + +**When to read:** +- Need to mock ANY request/response protocol +- Want readable test setup with DSL +- Testing clients that call external APIs + +**Applies to:** +- **JSON-RPC** (shown in example) - RPC over HTTP +- **REST APIs** - Use same pattern with route matching +- **GraphQL** - Configure response per query +- **gRPC** - Adapt for protobuf messages +- **WebSocket** - Mock message responses +- **Any HTTP-based protocol** - SOAP, XML-RPC, custom protocols + +**Key techniques to adapt:** +- Generic `AddMockResponse(identifier, response)` pattern +- Using `httptest.Server` as foundation +- Query/request tracking for assertions +- Configuration-based response mapping +- Thread-safe response storage + +--- + +## Pattern 4: Bidirectional Streaming with Rich DSL (Level 1) + +**File**: `examples/grpc-bufconn.md` + +**Pattern**: In-memory bidirectional communication with rich client/server mocks + +**When to read:** +- Testing ANY bidirectional streaming protocol +- Need full-duplex communication in tests +- Want to avoid network I/O + +**Applies to:** +- **gRPC** (shown in example) - Uses bufconn for in-memory +- **WebSockets** - Adapt bufconn pattern +- **TCP streams** - Custom protocols over TCP +- **Unix sockets** - Inter-process communication +- **Any streaming protocol** - Server-Sent Events, HTTP/2 streams + +**Key techniques to adapt:** +- `bufconn` for in-memory connections (gRPC-specific, but concept applies) +- Rich mock objects with helper methods +- Thread-safe state tracking with mutexes +- Assertion helpers (`ListenToStreamAndAssert()`) +- When testing **server** → mock the **clients** +- When testing **client** → mock the **server** + +--- + +## Pattern 5: HTTP DSL and Builder Pattern (Level 1) + +**File**: `examples/httptest-dsl.md` + +**Pattern**: Building readable test infrastructure with DSL wrappers over stdlib + +**When to read:** +- Want to wrap ANY test infrastructure with clean DSL +- Need fluent, readable test setup +- Building reusable test utilities + +**Applies to:** +- **HTTP mocking** (shown in example) - httptest.Server wrapper +- **Any test infrastructure** - Databases, queues, file systems +- **Test data builders** - Fluent APIs for creating test data +- **Custom test harnesses** - Wrapping complex setups + +**Key techniques to adapt:** +- Builder pattern with method chaining +- Fluent API design (`OnGET().RespondJSON()`) +- Separating configuration from execution +- Type-safe builders with Go generics +- Hiding complexity behind clean interfaces + +--- + +## Pattern 6: Test Organization and Structure + +**File**: `examples/test-organization.md` + +**When to read:** +- Setting up test structure for new projects +- Adding build tags for integration tests +- Configuring CI/CD for tests +- Creating testutils package structure + +**Universal patterns** (not technology-specific): +- File organization (`pkg_test` package naming) +- Build tags (`//go:build integration`) +- Makefile/Taskfile structure +- CI/CD configuration +- testutils package layout + +--- + +## Pattern 7: Integration Test Workflows + +**File**: `examples/integration-patterns.md` + +**When to read:** +- Testing component interactions across package boundaries +- Need patterns for Service + Repository testing +- Testing workflows that span multiple components + +**Universal patterns:** +- Pattern 1: Service + Repository with in-memory deps +- Pattern 2: Testing with real external services +- Pattern 3: Multi-component workflow with testify suites +- Dependency priority (in-memory > binary > test-containers) + +--- + +## Pattern 8: System Test (Black Box) + +**File**: `examples/system-patterns.md` + +**When to read:** +- Writing black-box end-to-end tests +- Testing via CLI or API +- Need tests that work without Docker + +**Universal patterns:** +- CLI testing with `exec.Command` +- API testing with HTTP client +- Dependency injection architecture +- Pure Go testing (no Docker) + +--- + +## How Claude Should Use These Files + +### Pattern-Based Reading Rules + +**When user needs to test with external dependencies:** + +1. **Has official Go test harness?** → Read `nats-in-memory.md` + - "Test with Redis/MongoDB/PostgreSQL/NATS" + - "Avoid Docker but need real service" + - Look for inspiration on wrapping official harnesses + +2. **Need to download/run binary?** → Read `victoria-metrics.md` + - "Test with Prometheus/Grafana/any binary" + - "Manage binary dependencies" + - Learn OS/ARCH detection, download patterns, health checks + +3. **Need to mock request/response?** → Read `jsonrpc-mock.md` + - "Mock REST/GraphQL/RPC/any HTTP API" + - "Build mock with DSL" + - Learn generic `AddMockResponse()` pattern + +4. **Need bidirectional streaming?** → Read `grpc-bufconn.md` + - "Test gRPC/WebSocket/streaming protocol" + - "In-memory bidirectional communication" + - Learn rich mock patterns, thread-safe state + +5. **Want readable test DSL?** → Read `httptest-dsl.md` + - "Build fluent test API" + - "Wrap test infrastructure" + - Learn builder pattern, method chaining + +**When user asks about test structure:** +- "How should I organize tests?" → Read `test-organization.md` +- "How do I write integration tests?" → Read `integration-patterns.md` +- "How do I write system tests?" → Read `system-patterns.md` + +### Key Principle + +**Examples show specific technologies (NATS, Victoria Metrics, JSON-RPC) but teach transferable patterns.** + +Claude should: +1. Identify the **pattern needed** (harness, binary, mock DSL, etc.) +2. Read the **example file** that demonstrates that pattern +3. **Adapt the techniques** to the user's specific technology +4. Use the example as a **template**, not a literal solution + +### Default Behavior (No Example Needed) + +For simple scenarios, use the core patterns in this file: +- Basic table-driven tests → Use patterns from this file +- Simple testify suites → Use patterns from this file +- Basic synchronization → Use patterns from this file +- Simple in-memory implementations → Use InMemoryRepository/TestEmailer from this file + +**Read example files when patterns/techniques are needed, not just for specific tech.** + +--- + +## Final Notes + +This reference provides core testing principles and patterns. For detailed implementations and complete examples, refer to the example files listed above. Each example file is self-contained and can be read independently based on your testing needs.