commit efb68b7cbeeac457c3c0f1ade56c60c87e6575ac Author: Zhongwei Li Date: Sun Nov 30 08:52:10 2025 +0800 Initial commit diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json new file mode 100644 index 0000000..7368c57 --- /dev/null +++ b/.claude-plugin/plugin.json @@ -0,0 +1,12 @@ +{ + "name": "stack-validator", + "description": "Validates GitLab stack projects before deployment, ensuring proper architecture patterns, directory structure, secrets management, .env configuration, and Docker best practices. Detects issues and provides actionable guidance", + "version": "1.0.0", + "author": { + "name": "rknall", + "email": "zhongweili@tubi.tv" + }, + "skills": [ + "./" + ] +} \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 0000000..e5ca0eb --- /dev/null +++ b/README.md @@ -0,0 +1,3 @@ +# stack-validator + +Validates GitLab stack projects before deployment, ensuring proper architecture patterns, directory structure, secrets management, .env configuration, and Docker best practices. Detects issues and provides actionable guidance diff --git a/SKILL.md b/SKILL.md new file mode 100644 index 0000000..a8c1bad --- /dev/null +++ b/SKILL.md @@ -0,0 +1,539 @@ +--- +name: "GitLab Stack Validator" +description: "Validates GitLab stack projects before deployment, ensuring proper architecture patterns, directory structure, secrets management, .env configuration, and Docker best practices. Use when users ask to validate a stack, check stack configuration, verify stack architecture, audit stack setup, or ensure stack deployment readiness." +--- + +# GitLab Stack Validator + +This skill validates GitLab stack projects to ensure they follow proper architecture patterns and are ready for deployment. It focuses on **detection and reporting** of issues, working alongside companion skills (stack-creator, secrets-manager) for remediation. + +## When to Use This Skill + +Activate this skill when the user requests: +- Validate a GitLab stack project +- Check stack configuration before deployment +- Verify stack architecture and structure +- Audit stack for best practices compliance +- Ensure stack follows proper patterns +- Pre-deployment validation checks +- Stack health check or readiness verification + +## Core Validation Principles + +This skill validates stacks that follow these architecture principles: + +1. **Configuration Management**: All configuration through docker-compose.yml and ./config directory +2. **Secrets Management**: Secrets stored in ./secrets and referenced via Docker secrets +3. **Environment Variables**: .env file with matching .env.example template +4. **Minimal Custom Scripts**: docker-entrypoint.sh only when containers don't support native secrets +5. **Proper Ownership**: No root-owned files (all files owned by Docker user) +6. **Temporary Files**: _temporary directory for transient files that are cleaned up after use +7. **Docker Best Practices**: Leverage docker-validation skill for Docker-specific checks + +## Validation Workflow + +When a user requests stack validation, follow this comprehensive workflow: + +### Phase 1: Pre-flight Checks + +**Step 1: Verify Stack Project** +1. Check if current directory appears to be a stack project +2. Look for key indicators: + - docker-compose.yml exists + - Presence of ./config, ./secrets, or ./_temporary directories + - .env file exists +3. If not a stack project, report findings and ask if user wants to initialize one + +**Step 2: Gather Context** +1. Ask user for validation scope (if needed): + - Full validation or targeted checks? + - Strict mode or permissive mode? + - Output format preference (text report vs JSON)? +2. Check if .stack-validator.yml exists for custom rules +3. Note the working directory for reporting + +### Phase 2: Directory Structure Validation + +**Step 1: Required Directories** +1. Check for required directories: + - `./config` - Configuration files directory + - `./secrets` - Secrets storage directory + - `./_temporary` - Temporary files directory +2. For each missing directory, record: + - Directory name + - Purpose and why it's required + - Impact on stack functionality + +**Step 2: Directory Permissions** +1. Check ./secrets has restricted permissions (700 or 600) +2. Verify other directories have appropriate permissions +3. Report any permission issues with security implications + +**Step 3: Directory Ownership** +1. Scan all project directories for ownership +2. Flag any root-owned directories: + - Directory path + - Current owner + - Expected owner (current user or docker user) +3. Flag any files with unexpected ownership + +**Step 4: .gitignore Validation** +1. Check if .gitignore exists +2. Verify it excludes: + - `./secrets` or `./secrets/*` + - `./_temporary` or `./_temporary/*` + - `.env` (should not be in git) +3. Report missing exclusions with security implications + +### Phase 3: Environment Variables Validation + +**Step 1: .env File Validation** +1. Check if .env file exists +2. Parse .env file for: + - Syntax errors + - Duplicate variable definitions + - Empty values that might be required +3. Record all environment variable names found + +**Step 2: .env.example Validation** +1. Check if .env.example exists +2. If missing, this is a **critical issue** - document why it's needed +3. Parse .env.example for variable names + +**Step 3: .env Synchronization Check** +1. Compare variables in .env with .env.example: + - Variables in .env but NOT in .env.example + - Variables in .env.example but NOT in .env +2. This is a **critical validation point** - they must match +3. Report any mismatches with specific variable names + +**Step 4: Environment Variable Security** +1. Scan .env for potential secrets: + - Variables with names containing: password, secret, key, token, api + - Base64-encoded looking values + - Long random strings +2. If secrets detected in .env, flag as security issue +3. Suggest moving to ./secrets and Docker secrets instead + +### Phase 4: Docker Configuration Validation + +**Step 1: Invoke docker-validation Skill** +1. Use the docker-validation skill to validate: + - docker-compose.yml syntax and best practices + - Dockerfile(s) if present + - Multi-stage builds + - Security configurations +2. Collect all findings from docker-validation +3. Integrate into overall stack validation report + +**Step 2: Stack-Specific Docker Checks** +1. Review docker-compose.yml for stack patterns: + - Secrets are defined in top-level `secrets:` section + - Services reference secrets via `secrets:` key (not environment variables) + - Volume mounts follow patterns (./config, ./secrets, ./_temporary) + - Networks are properly defined if multi-service + - Service dependencies use `depends_on` correctly + +**Step 3: Version Check** +1. Ensure docker-compose.yml does NOT have version field at top +2. This follows modern Docker Compose specification +3. If version field present, flag for removal + +### Phase 5: Secrets Management Validation + +**Step 1: Secrets Directory Check** +1. Verify ./secrets directory exists +2. Check permissions are restrictive (700) +3. Verify it's excluded from git + +**Step 2: Docker Secrets Validation** +1. Parse docker-compose.yml for secrets definitions +2. For each secret definition: + - Verify the secret file exists in ./secrets + - Check file permissions (600 or 400) + - Verify not tracked by git +3. Report missing secret files + +**Step 3: Secret References Validation** +1. Check each service's `secrets:` section +2. Verify referenced secrets are defined in top-level secrets +3. Flag any undefined secret references + +**Step 4: Environment Variables vs Secrets** +1. Scan service environment variables for potential secrets +2. Look for patterns like: + - *_PASSWORD + - *_SECRET + - *_KEY + - *_TOKEN + - API_* +3. If sensitive data in environment, suggest using secrets instead + +**Step 5: Secret Exposure Check** +1. Check for secrets in: + - docker-compose.yml (hardcoded values) + - .env file (should be in ./secrets instead) + - Configuration files in ./config + - Any shell scripts +2. Report any exposed secrets as **critical security issues** + +### Phase 6: Configuration Files Validation + +**Step 1: Config Directory Structure** +1. List all files in ./config directory +2. Organize by service (if applicable) +3. Check for proper organization + +**Step 2: Config File Validation** +1. For common config file types, validate syntax: + - YAML files (.yml, .yaml) + - JSON files (.json) + - INI files (.ini, .conf) + - TOML files (.toml) +2. Report any syntax errors + +**Step 3: Config vs Secrets Separation** +1. Scan config files for potential secrets +2. Look for hardcoded passwords, tokens, keys +3. Flag any secrets that should be in ./secrets instead + +**Step 4: Config Ownership** +1. Check all config files for ownership +2. Flag any root-owned config files +3. Report expected ownership (current user) + +### Phase 7: Script Validation + +**Step 1: docker-entrypoint.sh Detection** +1. Search for docker-entrypoint.sh files +2. For each found: + - Document location + - Check if truly necessary +3. Validate it's only used when container doesn't support native secrets + +**Step 2: Script Permissions** +1. Check docker-entrypoint.sh is executable (chmod +x) +2. Verify ownership (should not be root) +3. Report permission issues + +**Step 3: Script Content Validation** +1. Scan script for: + - Hardcoded secrets (critical issue) + - Proper secret handling from /run/secrets/ + - Error handling + - Syntax errors (if bash/sh) +2. Report any issues found + +**Step 4: Necessity Check** +1. Review if docker-entrypoint.sh is truly needed +2. Check if service supports native Docker secrets +3. Suggest removal if unnecessary + +### Phase 8: Temporary Directory Validation + +**Step 1: Directory Check** +1. Verify ./_temporary exists +2. Check it's in .gitignore +3. Verify permissions + +**Step 2: Content Check** +1. List contents of ./_temporary +2. Check for: + - Leftover files that should be cleaned + - Large files consuming space + - Old files (> 7 days old) +3. Report if not empty with details + +**Step 3: Usage Validation** +1. Check if ./_temporary is properly mounted in docker-compose.yml +2. Verify services use it for temporary files +3. Flag if not being utilized + +### Phase 9: File Ownership Audit + +**Step 1: Comprehensive Ownership Scan** +1. Use `find . -type f -user root 2>/dev/null` to find root-owned files +2. Exclude expected directories (.git, node_modules, vendor) +3. List all root-owned files found + +**Step 2: Categorize Issues** +1. Group by directory: + - ./config files owned by root + - ./secrets files owned by root + - Application files owned by root +2. Report with specific paths + +**Step 3: Impact Assessment** +1. For each root-owned file: + - Explain why this is problematic + - Impact on stack operations + - Potential errors that may occur + +### Phase 10: Report Generation + +**Step 1: Compile All Findings** +1. Organize findings by category: + - Directory Structure + - Environment Variables (.env sync) + - Docker Configuration + - Secrets Management + - Configuration Files + - Scripts (docker-entrypoint.sh) + - Temporary Directory + - File Ownership +2. Assign severity levels: + - ❌ **CRITICAL**: Security issues, missing required components, .env mismatch + - ⚠️ **WARNING**: Best practice violations, potential issues + - ✅ **PASS**: No issues found + +**Step 2: Generate Validation Report** + +Create a comprehensive report with this structure: + +``` +🔍 GitLab Stack Validation Report +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +Stack: [directory name] +Date: [timestamp] +Mode: [strict/permissive] + +📊 SUMMARY +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +✅ Passed: [count] +⚠️ Warnings: [count] +❌ Critical: [count] +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +📋 DETAILED FINDINGS +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +[For each validation category] + +[Category Icon] [Category Name]: [STATUS] +[If issues found:] + ❌ [Issue description] + Location: [file/directory path] + Impact: [what this affects] + Details: [specific information] + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +🎯 OVERALL STATUS: [PASSED/FAILED/WARNINGS] + +[If failures or warnings:] +🔧 RECOMMENDED ACTIONS +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +[Prioritized list of actions needed] +1. [Most critical action] +2. [Next action] +... + +💡 NEXT STEPS +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +- Use stack-creator skill to fix structural issues +- Use secrets-manager skill to properly configure secrets +- Re-run validation after fixes +``` + +**Step 3: Provide Actionable Guidance** +1. For each issue, provide: + - Clear description of the problem + - Why it matters + - Which companion skill can help fix it + - General guidance (but not actual fix commands) +2. Prioritize issues by severity +3. Group related issues together + +**Step 4: Export Options** +1. If user requested JSON output, export as: + ```json + { + "stack": "directory-name", + "timestamp": "ISO-8601", + "summary": { + "passed": 5, + "warnings": 2, + "critical": 1, + "status": "failed" + }, + "findings": [ + { + "category": "environment-variables", + "status": "critical", + "issues": [...] + } + ] + } + ``` + +## Validation Categories Reference + +### 1. Directory Structure +- Required directories exist +- Proper permissions +- .gitignore coverage +- Ownership correctness + +### 2. Environment Variables +- .env file exists and valid +- .env.example exists +- **CRITICAL**: .env and .env.example are synchronized +- No secrets in .env + +### 3. Docker Configuration +- docker-compose.yml valid (via docker-validation skill) +- No version field +- Secrets properly defined +- Volumes follow patterns +- Service dependencies correct + +### 4. Secrets Management +- ./secrets directory secure +- All referenced secrets exist +- Proper permissions +- No exposed secrets +- Environment variables don't contain secrets + +### 5. Configuration Files +- Proper organization +- Valid syntax +- No embedded secrets +- Correct ownership + +### 6. Scripts +- docker-entrypoint.sh only when necessary +- Executable permissions +- No hardcoded secrets +- Proper secret handling + +### 7. Temporary Directory +- Exists and in .gitignore +- Empty or contains only expected transient files +- Properly utilized in compose file + +### 8. File Ownership +- No root-owned files +- Consistent ownership +- Proper user/group + +## Integration with Companion Skills + +### stack-creator +- Recommend for: Creating missing directories, initializing structure +- Handles: Setting up new stacks, fixing structural issues + +### secrets-manager +- Recommend for: Secrets configuration, secret file management +- Handles: Creating/updating secrets, proper secret setup + +### docker-validation +- **Used directly during validation** for Docker-specific checks +- Provides: Docker Compose and Dockerfile validation +- Integrated into stack validation report + +## Validation Modes + +### Standard Mode (Default) +- Report all issues as warnings or errors +- Provide comprehensive findings +- Suggest improvements + +### Strict Mode +- Fail on warnings +- Require all best practices +- Zero tolerance for deviations + +### Permissive Mode +- Only fail on critical issues +- Allow warnings +- More lenient on best practices + +## Configuration Support + +If `.stack-validator.yml` exists in project root, respect these settings: + +```yaml +# Validation mode +strict_mode: false + +# Whether warnings should fail validation +fail_on_warnings: false + +# Paths to exclude from validation +exclude_paths: + - ./vendor + - ./node_modules + - ./_temporary/cache + +# Custom validation scripts to run +custom_checks: + - ./scripts/custom-validation.sh + +# Specific checks to skip +skip_checks: + - temporary-directory-empty +``` + +## Communication Style + +When reporting validation results: + +1. **Be Clear and Direct**: State issues plainly without hedging +2. **Be Specific**: Include exact file paths, line numbers, variable names +3. **Be Actionable**: Explain what needs to be done (without doing it) +4. **Be Helpful**: Point to companion skills that can fix issues +5. **Be Organized**: Group related issues, prioritize by severity +6. **Be Educational**: Explain why something is an issue +7. **Be Concise**: Don't overwhelm with details, but be thorough + +## Critical Validation Points + +These are **must-pass** criteria for production stacks: + +1. ✅ .env and .env.example are fully synchronized +2. ✅ No secrets in docker-compose.yml environment variables +3. ✅ ./secrets directory exists and has restrictive permissions +4. ✅ All referenced secrets exist +5. ✅ ./secrets and ./_temporary are in .gitignore +6. ✅ No root-owned files +7. ✅ docker-compose.yml passes docker-validation +8. ✅ No secrets exposed in git +9. ✅ .env file is not tracked by git + +## Important Notes + +- **Read-Only**: This skill NEVER modifies files - validation only +- **Detection-Focused**: Report issues, don't fix them +- **Comprehensive**: Check all aspects systematically +- **Fast**: Complete validation in seconds +- **Integrative**: Use docker-validation skill for Docker checks +- **Companion-Aware**: Direct users to appropriate skills for fixes + +## Example Validation Flow + +``` +User: "Validate my stack" + +1. Check if docker-compose.yml exists ✅ +2. Verify directory structure + - ./config ✅ + - ./secrets ✅ + - ./_temporary ⚠️ (not in .gitignore) +3. Validate .env configuration + - .env exists ✅ + - .env.example exists ❌ CRITICAL +4. Run docker-validation skill + - docker-compose.yml syntax ✅ + - Secrets configuration ⚠️ (using environment vars) +5. Check secrets management + - ./secrets permissions ✅ + - Secret files exist ✅ +6. Scan for ownership issues + - Found 3 root-owned files in ./config ❌ +7. Generate report with findings +8. Suggest: Use stack-creator to add .env.example and fix .gitignore +``` + +--- + +*This skill ensures stack quality, security, and deployment readiness through comprehensive validation.* diff --git a/common-issues.md b/common-issues.md new file mode 100644 index 0000000..da794ab --- /dev/null +++ b/common-issues.md @@ -0,0 +1,736 @@ +# Common Stack Validation Issues + +This guide documents common issues found during stack validation, their causes, impacts, and remediation guidance. + +## Table of Contents +1. [Critical Issues](#critical-issues) +2. [Security Issues](#security-issues) +3. [Configuration Issues](#configuration-issues) +4. [Structural Issues](#structural-issues) +5. [Docker Issues](#docker-issues) +6. [Ownership Issues](#ownership-issues) + +--- + +## Critical Issues + +### 1. Missing .env.example File + +**Issue**: .env file exists but .env.example is missing + +**Impact**: +- New developers don't know which environment variables are required +- No template for setting up new environments +- Difficult to understand configuration requirements +- Potential for missing critical variables in deployments + +**Detection**: +``` +❌ Environment Variables: CRITICAL + - .env.example file not found + - Required for documenting environment configuration +``` + +**Why This Matters**: +The .env.example file serves as documentation and a template for all required environment variables. Without it, setting up the stack in a new environment is error-prone. + +**Remediation**: +Use stack-creator skill to generate .env.example from current .env file: +- `claude "Use stack-creator to add .env.example"` +- Or manually create .env.example with all variables from .env + +--- + +### 2. .env and .env.example Out of Sync + +**Issue**: Variables in .env don't match .env.example + +**Impact**: +- Deployments may fail due to missing variables +- Documentation is inaccurate +- Team members may have different configurations +- Difficult to identify required vs optional variables + +**Detection**: +``` +❌ Environment Variables: CRITICAL + - Variables in .env but NOT in .env.example: + * API_TIMEOUT + * MAX_CONNECTIONS + - Variables in .env.example but NOT in .env: + * CACHE_TTL +``` + +**Why This Matters**: +The two files MUST be synchronized. .env.example is the source of truth for required variables. + +**Remediation**: +1. Manually sync the files - add missing variables +2. Use stack-creator to regenerate .env.example +3. Review if any variables should be removed + +--- + +### 3. Secrets in .env File + +**Issue**: Passwords, API keys, or tokens in .env + +**Impact**: +- Security risk if .env is accidentally committed +- Secrets not properly isolated +- Violates security best practices +- Difficult to rotate secrets + +**Detection**: +``` +❌ Environment Variables: CRITICAL + - Potential secrets detected in .env: + * DB_PASSWORD (contains password pattern) + * API_KEY (contains key pattern) + * JWT_SECRET (contains secret pattern) +``` + +**Why This Matters**: +Secrets should NEVER be in .env files. They should be in ./secrets and managed via Docker secrets. + +**Remediation**: +Use secrets-manager skill to properly configure secrets: +1. Move secrets to ./secrets directory +2. Update docker-compose.yml to use Docker secrets +3. Remove secrets from .env +4. Update applications to read from /run/secrets/ + +--- + +## Security Issues + +### 4. ./secrets Directory Not in .gitignore + +**Issue**: Secrets directory not excluded from git + +**Impact**: +- **CRITICAL SECURITY RISK**: Secrets may be committed to git +- Secret exposure in version control +- Compliance violations +- Potential data breach + +**Detection**: +``` +❌ Directory Structure: CRITICAL + - ./secrets not found in .gitignore + - Risk of committing secrets to version control +``` + +**Why This Matters**: +Once secrets are committed to git, they remain in history forever, even after removal. + +**Remediation**: +1. Add to .gitignore immediately: + ```gitignore + /secrets/ + /secrets/* + ``` +2. Check git status: `git status` - ensure no secrets staged +3. If secrets were committed, rotate all secrets immediately +4. Use BFG Repo-Cleaner to remove from git history (if needed) + +--- + +### 5. Secrets in docker-compose.yml Environment Variables + +**Issue**: Hardcoded secrets in environment variables + +**Impact**: +- Secrets exposed in docker-compose.yml (may be in git) +- Violates security best practices +- Difficult to rotate secrets +- Potential compliance issues + +**Detection**: +``` +❌ Secrets Management: CRITICAL + - Service 'app' has potential secrets in environment: + * DB_PASSWORD=supersecret123 (hardcoded password) + - Use Docker secrets instead +``` + +**Why This Matters**: +docker-compose.yml is often committed to git. Any secrets in it are exposed. + +**Remediation**: +Use secrets-manager skill: +1. Move secrets to ./secrets/ +2. Define in top-level secrets section +3. Reference via secrets key in services +4. Update app to read from /run/secrets/ + +--- + +### 6. Insecure ./secrets Directory Permissions + +**Issue**: ./secrets directory readable by others (e.g., 755 instead of 700) + +**Impact**: +- Other users on system can read secrets +- Security compliance violation +- Unnecessary exposure of sensitive data + +**Detection**: +``` +⚠️ Secrets Management: WARNING + - ./secrets directory permissions: 755 + - Should be 700 (owner only) +``` + +**Why This Matters**: +Secrets should only be readable by the user running the stack. + +**Remediation**: +```bash +chmod 700 ./secrets/ +chmod 600 ./secrets/* +``` + +--- + +### 7. .env File Tracked in Git + +**Issue**: .env file is committed to version control + +**Impact**: +- Configuration values exposed in git +- May contain secrets (critical if so) +- Environment-specific config in wrong place +- Difficult to manage per-environment settings + +**Detection**: +``` +❌ Directory Structure: CRITICAL + - .env is tracked by git (run: git ls-files .env) + - Must be added to .gitignore +``` + +**Why This Matters**: +.env files often contain environment-specific and sensitive data. + +**Remediation**: +1. Add .env to .gitignore +2. Remove from git: `git rm --cached .env` +3. Commit the removal +4. Review .env for any secrets - rotate if found + +--- + +## Configuration Issues + +### 8. Docker Compose Version Field Present + +**Issue**: docker-compose.yml includes version field + +**Impact**: +- Using deprecated syntax +- May cause warnings in modern Docker +- Not following current best practices + +**Detection**: +``` +⚠️ Docker Configuration: WARNING + - docker-compose.yml contains 'version' field + - Modern Compose files should omit version +``` + +**Why This Matters**: +Docker Compose Specification removed version field requirement. Modern files don't need it. + +**Remediation**: +Manually remove the version line: +```yaml +# ❌ Remove this +version: '3.8' + +# ✅ Start directly with services +services: + app: + ... +``` + +--- + +### 9. Secrets Defined but Files Missing + +**Issue**: docker-compose.yml references secrets that don't exist + +**Impact**: +- Stack will fail to start +- Docker will show "secret not found" errors +- Service initialization failures + +**Detection**: +``` +❌ Secrets Management: CRITICAL + - Secret 'db_password' defined but file not found: + Expected: ./secrets/db_password + - Secret 'api_key' defined but file not found: + Expected: ./secrets/api_key +``` + +**Why This Matters**: +Docker requires all referenced secret files to exist before starting services. + +**Remediation**: +Use secrets-manager skill to create missing secrets: +1. Create secret files: `echo -n "value" > ./secrets/db_password` +2. Set proper permissions: `chmod 600 ./secrets/db_password` +3. Verify all secrets exist before starting stack + +--- + +### 10. Undefined Secret References + +**Issue**: Service references secret not defined in top-level secrets section + +**Impact**: +- Docker Compose validation will fail +- Stack won't start +- Error messages: "service X references undefined secret" + +**Detection**: +``` +❌ Docker Configuration: CRITICAL + - Service 'app' references undefined secret: jwt_secret + - Secret must be defined in top-level 'secrets' section +``` + +**Why This Matters**: +All secrets used by services must be declared in the top-level secrets section. + +**Remediation**: +Add secret definition to docker-compose.yml: +```yaml +secrets: + jwt_secret: + file: ./secrets/jwt_secret +``` + +--- + +## Structural Issues + +### 11. Missing Required Directories + +**Issue**: Required directories (./config, ./secrets, ./_temporary) don't exist + +**Impact**: +- Stack may fail to start +- Volume mounts will fail +- Application errors +- Inconsistent structure + +**Detection**: +``` +❌ Directory Structure: CRITICAL + - Required directory './secrets' not found + - Required directory './_temporary' not found +``` + +**Why This Matters**: +Standard stack architecture requires these directories for proper organization. + +**Remediation**: +Use stack-creator skill to create missing directories: +```bash +mkdir -p config secrets _temporary +chmod 700 secrets +``` + +--- + +### 12. Incomplete .gitignore + +**Issue**: .gitignore doesn't exclude required paths + +**Impact**: +- Risk of committing secrets or temporary files +- Large repository size +- Cluttered git status +- Security risks + +**Detection**: +``` +⚠️ Directory Structure: WARNING + - .gitignore missing exclusion: /secrets/ + - .gitignore missing exclusion: /_temporary/ + - .gitignore missing exclusion: .env +``` + +**Why This Matters**: +Without proper exclusions, sensitive files may be accidentally committed. + +**Remediation**: +Add to .gitignore: +```gitignore +# Secrets +/secrets/ +/secrets/* + +# Environment +.env + +# Temporary +/_temporary/ +/_temporary/* +``` + +--- + +### 13. ./_temporary Directory Not Empty + +**Issue**: Temporary directory contains leftover files + +**Impact**: +- Disk space waste +- Stale cached data +- Potential application issues +- Poor housekeeping + +**Detection**: +``` +⚠️ Temporary Directory: WARNING + - ./_temporary is not empty: + * cache/sessions/old_session.dat (modified 15 days ago) + * uploads/temp_file.jpg (modified 3 days ago) + - Should be cleaned after use +``` + +**Why This Matters**: +Temporary directories should be cleaned regularly to avoid accumulating stale data. + +**Remediation**: +Manually clean the directory: +```bash +rm -rf ./_temporary/* +``` + +--- + +## Docker Issues + +### 14. Invalid docker-compose.yml Syntax + +**Issue**: YAML syntax errors in docker-compose.yml + +**Impact**: +- Stack cannot start +- Docker Compose validation fails +- Difficult to troubleshoot +- Deployment failures + +**Detection**: +``` +❌ Docker Configuration: CRITICAL + - docker-compose.yml syntax error at line 23 + - Error: mapping values are not allowed here +``` + +**Why This Matters**: +Invalid YAML prevents Docker Compose from parsing the file. + +**Remediation**: +1. Fix YAML syntax errors +2. Validate with: `docker compose config` +3. Use YAML linter for complex files + +--- + +### 15. Missing Service Dependencies + +**Issue**: Services don't declare depends_on for required services + +**Impact**: +- Services may start in wrong order +- Application errors on startup +- Race conditions +- Connection failures + +**Detection**: +``` +⚠️ Docker Configuration: WARNING + - Service 'app' uses DB but doesn't declare depends_on: postgres + - May cause startup ordering issues +``` + +**Why This Matters**: +Docker doesn't guarantee service start order without depends_on. + +**Remediation**: +Add depends_on to service: +```yaml +services: + app: + depends_on: + - postgres + - redis +``` + +--- + +### 16. No Restart Policy + +**Issue**: Services missing restart policy + +**Impact**: +- Services don't recover from crashes +- Manual intervention needed +- Poor reliability +- Downtime + +**Detection**: +``` +⚠️ Docker Configuration: WARNING + - Service 'app' has no restart policy + - Recommend: restart: unless-stopped +``` + +**Why This Matters**: +Production services should automatically restart on failure. + +**Remediation**: +Add restart policy: +```yaml +services: + app: + restart: unless-stopped +``` + +--- + +## Ownership Issues + +### 17. Root-Owned Configuration Files + +**Issue**: Files in ./config owned by root + +**Impact**: +- Cannot modify config without sudo +- Inconsistent permissions +- Potential deployment issues +- Bad practice + +**Detection**: +``` +❌ File Ownership: CRITICAL + - Root-owned files found: + * ./config/nginx/nginx.conf (owner: root) + * ./config/app/settings.yml (owner: root) +``` + +**Why This Matters**: +All stack files should be owned by the Docker user, not root. + +**Remediation**: +Fix ownership: +```bash +sudo chown -R $(id -u):$(id -g) ./config/ +``` + +To prevent: Ensure containers don't run as root, or don't mount config as writable. + +--- + +### 18. Root-Owned Files in _temporary + +**Issue**: Files in ./_temporary created by containers as root + +**Impact**: +- Cannot clean directory without sudo +- Disk space issues +- Permission problems + +**Detection**: +``` +⚠️ File Ownership: WARNING + - Root-owned files in ./_temporary: + * ./_temporary/cache/app_cache.db (owner: root) +``` + +**Why This Matters**: +Temporary files should be cleanable by the user. + +**Remediation**: +1. Fix current files: `sudo chown -R $(id -u):$(id -g) ./_temporary/` +2. Prevent future issues: Run containers as current user: + ```yaml + services: + app: + user: "${UID}:${GID}" + ``` + +--- + +### 19. Root-Owned Secret Files + +**Issue**: Files in ./secrets owned by root + +**Impact**: +- Cannot update secrets without sudo +- Security risk (root has access) +- Operational difficulties + +**Detection**: +``` +❌ File Ownership: CRITICAL + - Root-owned files in ./secrets: + * ./secrets/db_password (owner: root) +``` + +**Why This Matters**: +Secret files should be owned by the Docker user with restricted permissions. + +**Remediation**: +```bash +sudo chown $(id -u):$(id -g) ./secrets/* +chmod 600 ./secrets/* +``` + +--- + +## Script Issues + +### 20. Unnecessary docker-entrypoint.sh + +**Issue**: docker-entrypoint.sh exists but container supports native secrets + +**Impact**: +- Unnecessary complexity +- Maintenance overhead +- Potential bugs +- Not following best practices + +**Detection**: +``` +⚠️ Scripts: WARNING + - docker-entrypoint.sh found for service 'postgres' + - PostgreSQL supports native Docker secrets via *_FILE variables + - Consider removing custom entrypoint +``` + +**Why This Matters**: +Many modern containers support Docker secrets natively. Custom entrypoints add complexity. + +**Remediation**: +1. Check if container supports *_FILE environment variables +2. If yes, remove docker-entrypoint.sh +3. Use native secret support: + ```yaml + environment: + POSTGRES_PASSWORD_FILE: /run/secrets/db_password + ``` + +--- + +### 21. docker-entrypoint.sh Not Executable + +**Issue**: Script exists but doesn't have execute permissions + +**Impact**: +- Container will fail to start +- Permission denied errors +- Deployment failures + +**Detection**: +``` +❌ Scripts: CRITICAL + - docker-entrypoint.sh is not executable + - Current permissions: -rw-r--r-- + - Required: -rwxr-xr-x +``` + +**Why This Matters**: +Docker needs execute permission to run entrypoint scripts. + +**Remediation**: +```bash +chmod +x docker-entrypoint.sh +``` + +--- + +### 22. Hardcoded Secrets in Scripts + +**Issue**: Secrets hardcoded in docker-entrypoint.sh or other scripts + +**Impact**: +- **CRITICAL SECURITY RISK** +- Secrets exposed in git +- Difficult to rotate +- Compliance violations + +**Detection**: +``` +❌ Scripts: CRITICAL + - Hardcoded secret detected in docker-entrypoint.sh: + Line 15: export DB_PASSWORD="supersecret123" + - Secrets must be read from /run/secrets/ +``` + +**Why This Matters**: +Scripts are often committed to git. Hardcoded secrets = exposed secrets. + +**Remediation**: +Read secrets from Docker secrets: +```bash +# ❌ BAD +export DB_PASSWORD="supersecret123" + +# ✅ GOOD +export DB_PASSWORD=$(cat /run/secrets/db_password) +``` + +--- + +## Summary: Issue Severity Levels + +### Critical (Must Fix Before Deployment) +- Missing .env.example +- .env/.env.example out of sync +- Secrets in .env +- ./secrets not in .gitignore +- Secrets in docker-compose.yml +- Missing secret files +- Root-owned files +- Invalid docker-compose.yml syntax + +### High (Fix Soon) +- .env tracked in git +- Insecure secrets permissions +- Missing required directories +- Hardcoded secrets in scripts + +### Medium (Should Fix) +- Docker Compose version field +- Missing restart policies +- Missing dependencies +- Incomplete .gitignore + +### Low (Nice to Fix) +- ./_temporary not empty +- Unnecessary docker-entrypoint.sh +- Suboptimal configurations + +--- + +## Quick Reference: Issue to Skill Mapping + +| Issue Category | Recommended Skill | +|---------------|------------------| +| Missing .env.example | stack-creator | +| Directory structure | stack-creator | +| Secrets management | secrets-manager | +| Docker configuration | docker-validation (auto-used) | +| File ownership | Manual fix (chown) | +| .gitignore issues | stack-creator | +| .env sync | Manual + stack-creator | + +--- + +*This guide helps identify and understand common stack validation issues for faster remediation.* diff --git a/plugin.lock.json b/plugin.lock.json new file mode 100644 index 0000000..00da288 --- /dev/null +++ b/plugin.lock.json @@ -0,0 +1,53 @@ +{ + "$schema": "internal://schemas/plugin.lock.v1.json", + "pluginId": "gh:rknall/claude-skills:stack-validator", + "normalized": { + "repo": null, + "ref": "refs/tags/v20251128.0", + "commit": "2839340c159f875f7ac6ff4ef70c7de23a1a328e", + "treeHash": "89634cdb310c0d450b343b88d501964ddb4fcdcaef64289f22e9136d997d7310", + "generatedAt": "2025-11-28T10:27:59.086297Z", + "toolVersion": "publish_plugins.py@0.2.0" + }, + "origin": { + "remote": "git@github.com:zhongweili/42plugin-data.git", + "branch": "master", + "commit": "aa1497ed0949fd50e99e70d6324a29c5b34f9390", + "repoRoot": "/Users/zhongweili/projects/openmind/42plugin-data" + }, + "manifest": { + "name": "stack-validator", + "description": "Validates GitLab stack projects before deployment, ensuring proper architecture patterns, directory structure, secrets management, .env configuration, and Docker best practices. Detects issues and provides actionable guidance", + "version": "1.0.0" + }, + "content": { + "files": [ + { + "path": "common-issues.md", + "sha256": "db198b733f09488889ec113abea285a1145bfb49c2c775fff8f0e1884fb6ecc9" + }, + { + "path": "README.md", + "sha256": "190b6b8acadd8ccaf9af6ed5d5a630efa6dbe1901a1cb76724632bd27ad51a27" + }, + { + "path": "SKILL.md", + "sha256": "51c8ff822596d514164a9e996968bcfcd3d5ba474deea64312df7d3343617bd6" + }, + { + "path": "validation-patterns.md", + "sha256": "880d73867129d02a10d401f156f264a5b609f92a406a4a5fe60bb250839bb717" + }, + { + "path": ".claude-plugin/plugin.json", + "sha256": "89afe78eeae0a381fdc242e5cb40d7e19d7266d1283089005ce0e04b5c576c99" + } + ], + "dirSha256": "89634cdb310c0d450b343b88d501964ddb4fcdcaef64289f22e9136d997d7310" + }, + "security": { + "scannedAt": null, + "scannerVersion": null, + "flags": [] + } +} \ No newline at end of file diff --git a/validation-patterns.md b/validation-patterns.md new file mode 100644 index 0000000..606fd34 --- /dev/null +++ b/validation-patterns.md @@ -0,0 +1,564 @@ +# Stack Validation Patterns + +This document outlines the architecture patterns and validation criteria for GitLab stack projects. + +## Table of Contents +1. [Directory Structure Patterns](#directory-structure-patterns) +2. [Environment Variable Patterns](#environment-variable-patterns) +3. [Secrets Management Patterns](#secrets-management-patterns) +4. [Docker Compose Patterns](#docker-compose-patterns) +5. [Configuration Patterns](#configuration-patterns) +6. [File Ownership Patterns](#file-ownership-patterns) + +--- + +## Directory Structure Patterns + +### Standard Stack Directory Layout + +``` +my-stack/ +├── docker-compose.yml # Main compose file (NO version field) +├── .env # Environment variables (NOT in git) +├── .env.example # Environment template (IN git) +├── .gitignore # Must exclude secrets, .env, _temporary +├── .stack-validator.yml # Optional: Custom validation rules +├── config/ # Configuration files +│ ├── nginx/ +│ │ └── nginx.conf +│ ├── app/ +│ │ └── settings.yml +│ └── db/ +│ └── init.sql +├── secrets/ # Secret files (NOT in git) +│ ├── db_password +│ ├── api_key +│ └── jwt_secret +└── _temporary/ # Transient files (NOT in git) + └── (cleaned after use) +``` + +### Required Directories + +| Directory | Purpose | Git Status | Permissions | +|-----------|---------|------------|-------------| +| `./config` | Configuration files | Tracked | 755 | +| `./secrets` | Secret files | **NOT tracked** | 700 | +| `./_temporary` | Temporary/cache files | **NOT tracked** | 755 | + +### .gitignore Requirements + +**MUST contain:** +```gitignore +# Secrets - never commit +/secrets/ +/secrets/* + +# Environment variables - never commit +.env + +# Temporary files +/_temporary/ +/_temporary/* + +# Common exclusions +*.log +.DS_Store +``` + +--- + +## Environment Variable Patterns + +### .env File Structure + +**Purpose**: Define environment-specific variables for stack deployment + +**Example .env:** +```bash +# Application +APP_NAME=my-application +APP_ENV=production +APP_DEBUG=false +APP_URL=https://example.com + +# Database +DB_HOST=postgres +DB_PORT=5432 +DB_NAME=app_database +DB_USER=app_user +# NOTE: DB_PASSWORD should be in ./secrets/db_password, not here! + +# Redis +REDIS_HOST=redis +REDIS_PORT=6379 + +# Ports +WEB_PORT=80 +API_PORT=8080 + +# Docker +COMPOSE_PROJECT_NAME=my-stack +``` + +### .env.example File Structure + +**Purpose**: Template for required environment variables + +**CRITICAL RULE**: .env.example MUST contain ALL variables from .env and vice versa + +**Example .env.example:** +```bash +# Application +APP_NAME=my-application +APP_ENV=development +APP_DEBUG=true +APP_URL=http://localhost + +# Database +DB_HOST=postgres +DB_PORT=5432 +DB_NAME=app_database +DB_USER=app_user +# NOTE: DB_PASSWORD is managed via Docker secrets in ./secrets/ + +# Redis +REDIS_HOST=redis +REDIS_PORT=6379 + +# Ports - Customize for your environment +WEB_PORT=80 +API_PORT=8080 + +# Docker +COMPOSE_PROJECT_NAME=my-stack +``` + +### Environment Variable Validation Rules + +1. **Synchronization**: Every variable in .env MUST be in .env.example +2. **Documentation**: .env.example should have comments explaining each variable +3. **No Secrets**: .env should NOT contain passwords, API keys, tokens, or secrets +4. **Default Values**: .env.example should have safe defaults for development +5. **Required Variables**: Both files must define all required variables + +### Variables That Should NOT Be in .env + +Move these to ./secrets and use Docker secrets: + +```bash +# ❌ BAD - Don't put these in .env +DB_PASSWORD=supersecret123 +API_KEY=sk_live_abc123xyz +JWT_SECRET=my-jwt-secret-key +STRIPE_SECRET_KEY=sk_test_123 +OAUTH_CLIENT_SECRET=abc123xyz + +# ✅ GOOD - Reference via Docker secrets instead +# See docker-compose.yml secrets section +# Secrets are in ./secrets/ directory +``` + +--- + +## Secrets Management Patterns + +### Secrets Directory Structure + +``` +secrets/ +├── db_password # PostgreSQL password +├── db_root_password # Root password (if needed) +├── api_key # External API key +├── jwt_secret # JWT signing secret +└── oauth_client_secret # OAuth secret +``` + +### Secret File Format + +**Single-line, no trailing newline:** +```bash +# Create secret (no newline) +echo -n "my-secret-value" > ./secrets/db_password + +# ✅ Correct: 16 bytes +# ❌ Wrong: 17 bytes (includes newline) +``` + +### Secret File Permissions + +```bash +# Directory +chmod 700 ./secrets/ + +# Individual files +chmod 600 ./secrets/db_password +chmod 600 ./secrets/api_key +``` + +### Docker Compose Secrets Pattern + +**Top-level secrets definition:** +```yaml +secrets: + db_password: + file: ./secrets/db_password + api_key: + file: ./secrets/api_key + jwt_secret: + file: ./secrets/jwt_secret +``` + +**Service secrets reference:** +```yaml +services: + app: + image: myapp:latest + secrets: + - db_password + - api_key + - jwt_secret + environment: + # ✅ GOOD - Reference location, not value + DB_PASSWORD_FILE: /run/secrets/db_password + API_KEY_FILE: /run/secrets/api_key + + # ❌ BAD - Don't put actual secrets here + # DB_PASSWORD: supersecret123 +``` + +### Application Secret Usage + +**In application code:** +```python +# Read secret from Docker secrets mount +def get_secret(secret_name): + secret_path = f'/run/secrets/{secret_name}' + with open(secret_path, 'r') as f: + return f.read().strip() + +# Usage +db_password = get_secret('db_password') +api_key = get_secret('api_key') +``` + +--- + +## Docker Compose Patterns + +### Modern Docker Compose Format + +**❌ DON'T use version field:** +```yaml +# ❌ OLD - Don't include version +version: '3.8' +``` + +**✅ DO use modern format:** +```yaml +# ✅ MODERN - No version field +services: + app: + image: myapp:latest +``` + +### Complete Stack Example + +```yaml +services: + app: + image: myapp:latest + container_name: my-app + restart: unless-stopped + depends_on: + - postgres + - redis + secrets: + - db_password + - api_key + environment: + APP_ENV: ${APP_ENV} + DB_HOST: ${DB_HOST} + DB_PORT: ${DB_PORT} + DB_NAME: ${DB_NAME} + DB_USER: ${DB_USER} + DB_PASSWORD_FILE: /run/secrets/db_password + API_KEY_FILE: /run/secrets/api_key + volumes: + - ./config/app:/app/config:ro + - app-data:/app/data + networks: + - app-network + ports: + - "${WEB_PORT}:80" + + postgres: + image: postgres:16-alpine + container_name: my-postgres + restart: unless-stopped + secrets: + - db_password + environment: + POSTGRES_DB: ${DB_NAME} + POSTGRES_USER: ${DB_USER} + POSTGRES_PASSWORD_FILE: /run/secrets/db_password + volumes: + - ./config/db/init.sql:/docker-entrypoint-initdb.d/init.sql:ro + - postgres-data:/var/lib/postgresql/data + networks: + - app-network + + redis: + image: redis:7-alpine + container_name: my-redis + restart: unless-stopped + volumes: + - redis-data:/data + networks: + - app-network + +volumes: + app-data: + postgres-data: + redis-data: + +networks: + app-network: + driver: bridge + +secrets: + db_password: + file: ./secrets/db_password + api_key: + file: ./secrets/api_key +``` + +### Volume Mount Patterns + +**Configuration files (read-only):** +```yaml +volumes: + - ./config/nginx/nginx.conf:/etc/nginx/nginx.conf:ro + - ./config/app/settings.yml:/app/config/settings.yml:ro +``` + +**Secrets (via Docker secrets - automatic mount):** +```yaml +secrets: + - db_password # Mounted at /run/secrets/db_password +``` + +**Persistent data (named volumes):** +```yaml +volumes: + - postgres-data:/var/lib/postgresql/data + - redis-data:/data +``` + +**Temporary files (local directory):** +```yaml +volumes: + - ./_temporary/cache:/app/cache + - ./_temporary/uploads:/app/uploads +``` + +--- + +## Configuration Patterns + +### Configuration File Organization + +**By service:** +``` +config/ +├── nginx/ +│ ├── nginx.conf +│ └── ssl/ +│ ├── cert.pem +│ └── key.pem +├── app/ +│ ├── settings.yml +│ └── logging.conf +└── db/ + └── init.sql +``` + +### Configuration vs Secrets Separation + +**✅ Configuration (in ./config):** +- Server hostnames +- Port numbers +- Feature flags +- Logging levels +- Public certificates +- Database names +- Cache settings + +**❌ NOT Configuration (in ./secrets):** +- Passwords +- API keys +- Tokens +- Private keys +- OAuth secrets +- JWT secrets +- Encryption keys + +### Example Configuration File + +**config/app/settings.yml:** +```yaml +# Application Settings +app: + name: ${APP_NAME} + environment: ${APP_ENV} + debug: ${APP_DEBUG} + url: ${APP_URL} + +# Database (connection info, NOT credentials) +database: + host: ${DB_HOST} + port: ${DB_PORT} + name: ${DB_NAME} + user: ${DB_USER} + # Password loaded from /run/secrets/db_password + +# Redis +cache: + driver: redis + host: ${REDIS_HOST} + port: ${REDIS_PORT} + +# Logging +logging: + level: info + output: stdout + format: json +``` + +--- + +## File Ownership Patterns + +### Correct Ownership + +All stack files should be owned by the Docker user (current user), NOT root. + +**Check ownership:** +```bash +# List all files with ownership +eza -la --tree + +# Find root-owned files (should return nothing) +find . -type f -user root 2>/dev/null +``` + +### Common Ownership Issues + +**Problem**: Files created by Docker containers as root + +**Example scenario:** +```yaml +# Container runs as root, creates files in mounted volume +services: + app: + image: nginx:latest # Runs as root by default + volumes: + - ./config/nginx.conf:/etc/nginx/nginx.conf + - ./_temporary/cache:/var/cache/nginx # ⚠️ Creates files as root! +``` + +**Fix**: Ensure containers run as non-root user + +```yaml +services: + app: + image: nginx:latest + user: "${UID}:${GID}" # Run as current user + volumes: + - ./config/nginx.conf:/etc/nginx/nginx.conf + - ./_temporary/cache:/var/cache/nginx +``` + +### Fixing Ownership + +**Stack-validator detects ownership issues but doesn't fix them.** + +**Manual fix (user action):** +```bash +# Fix ownership of specific file +sudo chown $(id -u):$(id -g) ./config/nginx.conf + +# Fix ownership of entire directory +sudo chown -R $(id -u):$(id -g) ./config/ + +# Fix ownership of all project files +sudo chown -R $(id -u):$(id -g) . +``` + +**Prevention**: Use stack-creator skill to properly initialize stacks with correct ownership from the start. + +--- + +## Validation Checklist + +### Pre-Deployment Validation + +Use this checklist to ensure stack readiness: + +- [ ] **Directory Structure** + - [ ] ./config directory exists + - [ ] ./secrets directory exists with 700 permissions + - [ ] ./_temporary directory exists + - [ ] .gitignore excludes secrets, .env, _temporary + +- [ ] **Environment Variables** + - [ ] .env file exists and is valid + - [ ] .env.example exists and is valid + - [ ] .env and .env.example are synchronized + - [ ] No secrets in .env file + - [ ] .env is in .gitignore + +- [ ] **Docker Configuration** + - [ ] docker-compose.yml has no version field + - [ ] docker-compose.yml passes docker-validation + - [ ] Secrets defined in top-level secrets section + - [ ] Services reference secrets via secrets key + - [ ] Volume mounts follow patterns + +- [ ] **Secrets Management** + - [ ] All secret files exist in ./secrets + - [ ] Secret files have 600 permissions + - [ ] ./secrets directory has 700 permissions + - [ ] No secrets in docker-compose.yml environment + - [ ] No secrets in git + +- [ ] **File Ownership** + - [ ] No root-owned files in project + - [ ] All files owned by Docker user + - [ ] Config files have correct ownership + +- [ ] **Configuration** + - [ ] Config files properly organized + - [ ] No secrets in config files + - [ ] Config file syntax valid + +--- + +## Reference: Common Validation Failures + +| Issue | Category | Severity | Fix With | +|-------|----------|----------|----------| +| Missing .env.example | Environment | Critical | stack-creator | +| .env/.env.example mismatch | Environment | Critical | Manual sync + stack-creator | +| Secrets in .env | Security | Critical | secrets-manager | +| ./secrets not in .gitignore | Security | Critical | stack-creator | +| Root-owned files | Ownership | High | Manual chown | +| Missing ./secrets directory | Secrets | High | stack-creator | +| docker-compose.yml has version | Docker | Medium | Manual edit | +| Secrets in environment vars | Security | Critical | secrets-manager | +| Missing required directory | Structure | High | stack-creator | +| ./_temporary not empty | Cleanup | Low | Manual cleanup | + +--- + +*These patterns ensure consistent, secure, and maintainable GitLab stack projects.*