Initial commit

skills/scope/SKILL.md

@@ -0,0 +1,643 @@
+---
+name: scope
+description: Guide for working with Scope, a developer environment management tool that automates environment checks, detects known errors, and provides automated fixes. Use when creating Scope configurations (ScopeKnownError, ScopeDoctorGroup, ScopeReportLocation), debugging environment issues, or writing rules for error detection and remediation.
+---
+
+# Scope - Developer Environment Management
+
+Scope is a DevEx tool that helps maintain consistent development environments through automated checks, error detection, and fixes.
+
+## Core Commands
+
+### scope doctor
+
+Run automated environment checks and fixes.
+
+```bash
+scope doctor run                   # Run all checks
+scope doctor run --only group-name # Run specific group
+scope doctor run --fix=false       # Check only, no fixes
+scope doctor run --no-cache        # Disable caching
+scope doctor list                  # List available checks
+```
+
+### scope analyze
+
+Detect known errors in logs or command output.
+
+```bash
+scope analyze logs file.log                # Analyze log file
+scope analyze command -- cmd args          # Analyze command output
+scope analyze --extra-config path file.log # Use additional configs
+```
+
+### scope report
+
+Generate bug reports with automatic secret redaction.
+
+```bash
+scope report ./script.sh     # Run and report on script
+scope report -- command args # Run and report on command
+```
+
+### scope-intercept
+
+Monitor script execution in real-time (used as shebang).
+
+```bash
+#!/usr/bin/env scope-intercept bash
+# Script content here
+```
+
+## Configuration Resources
+
+All Scope configurations use Kubernetes-inspired YAML format:
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: <ResourceType>
+metadata:
+  name: unique-identifier
+  description: Human-readable description
+spec:
+  # Resource-specific configuration
+```
+
+## ScopeKnownError
+
+Define error patterns and provide automated help/fixes for common issues.
+
+### Structure
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeKnownError
+metadata:
+  name: descriptive-error-name
+  description: Brief description of what this error means
+spec:
+  pattern: 'regex pattern to match the error'
+  help: |
+    Clear explanation of the issue.
+
+    Steps to resolve:
+    1. First step
+    2. Second step
+    3. Where to get help if needed
+  fix:
+    prompt:
+      text: User-friendly prompt asking permission
+    commands:
+      - first-fix-command
+      - second-fix-command
+```
+
+### Key Fields
+
+- **pattern**: Regex pattern using ripgrep syntax to match error text
+- **help**: Multi-line markdown explanation with resolution steps
+- **fix** (optional): Automated remediation configuration
+  - **commands**: List of commands to run to fix the error (required)
+  - **helpText**: Descriptive text shown if the fix fails (optional)
+  - **helpUrl**: Documentation link for manual resolution (optional)
+  - **prompt**: User approval before running fix (optional)
+    - **text**: Question asking for user permission
+    - **extraContext**: Additional context about what the fix does and why approval is needed
+
+**Note:** When a fix is defined, it only runs when the error pattern is detected. The fix is optional - if not provided, only the help text is shown.
+
+### File Organization
+
+Place files in categorized directories:
+
+- Location: `{config-root}/known-errors/{category}/{error-name}.yaml`
+- Common categories: `docker/`, `ruby/`, `git/`, `github/`, `mysql/`, `postgres/`, `dns/`, `aws/`
+- Include matching `.txt` file with example error for testing
+
+### Example
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeKnownError
+metadata:
+  name: gem-missing-file
+  description: A gem source file is missing and fails to be loaded
+spec:
+  pattern: "/lib/ruby/.* `require': cannot load such file --.*/lib/ruby/gems/.*(LoadError)"
+  help: |
+    A gem source file is missing. The solution is to reinstall the gems:
+    1. Run `bundle pristine`
+  fix:
+    commands:
+      - bundle pristine
+    helpText: |
+      Bundle pristine failed. Try these steps:
+      1. Check your bundle config: bundle config list
+      2. Reinstall bundler: gem install bundler
+      3. Contact #help-ruby if issues persist
+    helpUrl: https://bundler.io/man/bundle-pristine.1.html
+    prompt:
+      text: |-
+        This will reinstall all gems from your Gemfile.
+        Do you wish to continue?
+      extraContext: >-
+        bundle pristine reinstalls gems without changing versions,
+        which resolves missing file errors but preserves your lock file
+```
+
+### Validation
+
+Validate schema structure:
+
+```bash
+brew install sourcemeta/apps/jsonschema
+jsonschema validate schema.json known-error.yaml
+```
+
+Test pattern matching:
+
+```bash
+scope analyze logs --extra-config config-dir/ test-error.txt
+```
+
+### Additional Resources
+
+- [Complete example with fix and prompt](https://github.com/oscope-dev/scope/blob/main/examples/.scope/known-error-with-fix.yaml)
+- [ScopeKnownError model documentation](https://oscope-dev.github.io/scope/docs/models/ScopeKnownError)
+
+### Pattern Writing Tips
+
+- Use ripgrep regex syntax (similar to PCRE)
+- Escape special characters: `\\.`, `\\[`, `\\(`
+- Use character classes: `[[:digit:]]`, `[[:alpha:]]`
+- Test patterns: `echo "error text" | rg "pattern"`
+- Balance specificity vs flexibility to avoid false positives/negatives
+
+## ScopeDoctorGroup
+
+Define environment setup and maintenance checks with automated fixes.
+
+### Structure
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: tool-name
+  description: Brief description of what this group does
+spec:
+  include: when-required # or "by-default"
+  needs:
+    - dependency-1
+    - dependency-2
+  actions:
+    - name: action-name
+      description: What this action checks/fixes
+      required: true # default true; false for optional checks
+      check:
+        paths:
+          - 'file-to-watch.txt'
+          - '**/*.config'
+        commands:
+          - ./bin/check-script.sh
+      fix:
+        commands:
+          - ./bin/fix-script.sh
+        helpText: |
+          What to do if the fix fails.
+          Where to get help.
+        helpUrl: https://docs.example.com/help
+```
+
+### Key Components
+
+#### Include Modes
+
+- **by-default**: Always runs with `scope doctor run`
+- **when-required**: Only runs when explicitly specified or as dependency
+
+#### Dependencies
+
+List other groups that must run first in `needs` array. Creates execution graph.
+
+#### Actions
+
+Each action is an atomic check/fix operation that runs in order.
+
+#### Check Logic
+
+Determines if fix should run. Fix runs when:
+
+- **No check defined**: Fix always runs
+- **paths specified**: Any file changed or no files match globs
+- **commands specified**: Any command exits non-zero
+- Both can be combined (OR logic)
+
+#### Fix Section
+
+- **commands**: List of commands to run in order
+- **helpText**: Shown if fix fails (markdown supported)
+- **helpUrl**: Optional link to documentation
+
+### File Organization
+
+- Application setup: `{config-root}/application/`
+- Environment setup: `{config-root}/environment/`
+- Project-specific: `.scope/` in project root
+
+### Example
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: ruby-version
+  description: Set up Ruby with correct version
+spec:
+  include: when-required
+  needs:
+    - ruby-manager
+    - homebrew
+  actions:
+    - name: install
+      description: Ensures correct version of ruby is installed
+      check:
+        paths:
+          - '{{ working_dir }}/.ruby-version'
+        commands:
+          - ./bin/ruby-version.sh verify
+      fix:
+        commands:
+          - ./bin/ruby-version.sh install
+        helpText: |
+          Ruby installation failed.
+          Contact: #help-channel
+```
+
+### Script Conventions
+
+Create helper scripts following check/fix pattern:
+
+```bash
+#!/usr/bin/env bash
+set -euo pipefail
+
+ACTION="${1:-check}"
+
+check() {
+  # Return 0 if check passes, non-zero if fix needed
+  if [[ condition ]]; then
+    echo "Check passed"
+    return 0
+  else
+    echo "Check failed" >&2
+    return 1
+  fi
+}
+
+fix() {
+  # Perform fix, return 0 on success
+  echo "Running fix..."
+  # Fix commands here
+  return 0
+}
+
+case "$ACTION" in
+  check) check ;;
+  fix) fix ;;
+  *)
+    echo "Usage: $0 [check|fix]" >&2
+    exit 1
+    ;;
+esac
+```
+
+Script best practices:
+
+- Use relative paths with `./` prefix (relative to YAML file)
+- Make scripts idempotent (safe to run multiple times)
+- Exit 0 for success, non-zero for failure
+- Write errors to stderr
+- Include helpful error messages
+
+### Caching
+
+File-based checks use content hashing:
+
+- Cache stores file path and content hash
+- Only re-runs if file contents change
+- Persists between runs
+- Disable with `--no-cache`
+
+## ScopeReportLocation
+
+Configure where and how bug reports are uploaded.
+
+### GitHub Issues
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeReportLocation
+metadata:
+  name: github
+spec:
+  destination:
+    githubIssue:
+      owner: org-name
+      repo: repo-name
+  additionalData:
+    nodeVersion: node -v
+    rubyPath: which ruby
+```
+
+Authentication via environment variables:
+
+- GitHub App: `SCOPE_GH_APP_ID` + `SCOPE_GH_APP_KEY`
+- Personal Access Token: `SCOPE_GH_TOKEN`
+
+### Local File System
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeReportLocation
+metadata:
+  name: local
+spec:
+  destination:
+    local:
+      directory: /tmp/scope-reports
+  additionalData:
+    pwd: pwd
+    username: id -un
+    scopeVersion: scope version
+```
+
+### RustyPaste
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeReportLocation
+metadata:
+  name: rustypaste
+spec:
+  destination:
+    rustyPaste:
+      url: http://localhost:8000
+```
+
+### Additional Data
+
+Capture environment context with arbitrary commands:
+
+```yaml
+additionalData:
+  diskSpace: df -h
+  gitBranch: git branch --show-current
+  envVars: env | sort
+```
+
+Commands execute at report generation time. Scope automatically redacts secrets.
+
+## Common Patterns
+
+### Version Check Pattern
+
+```yaml
+actions:
+  - name: minimum-version
+    description: Ensure minimum version installed
+    check:
+      commands:
+        - ./bin/check-version.sh check tool 1.2.3
+    fix:
+      helpText: |
+        You don't have the minimum version installed.
+        Check Managed Software Center for updates.
+```
+
+### File-Based Check Pattern
+
+```yaml
+actions:
+  - name: config-file
+    description: Verify config file exists and is valid
+    check:
+      paths:
+        - .config-file
+      commands:
+        - test -s .config-file
+    fix:
+      commands:
+        - ./bin/setup-config.sh
+```
+
+### Service Health Pattern
+
+```yaml
+actions:
+  - name: service-running
+    description: Ensure service is running
+    check:
+      commands:
+        - ./bin/check-service.sh
+    fix:
+      commands:
+        - ./bin/start-service.sh
+      helpText: |
+        Service failed to start.
+        Check logs: tail "$(brew --prefix)/var/log/service.log"
+```
+
+### Dependency Install Pattern
+
+```yaml
+actions:
+  - name: install-packages
+    description: Install project dependencies
+    check:
+      paths:
+        - package.json
+        - yarn.lock
+    fix:
+      commands:
+        - yarn install
+```
+
+### Orchestration Pattern
+
+```yaml
+# Main group that just coordinates dependencies
+spec:
+  needs:
+    - environment-setup
+    - database-setup
+    - language-runtime
+  actions: [] # No actions, just orchestration
+```
+
+## Workflow: Creating Known Error
+
+1. **Identify error pattern**
+
+   - Capture actual error output
+   - Find unique identifying text
+   - Create regex pattern
+
+2. **Create files**
+
+   ```bash
+   mkdir -p {config-root}/known-errors/category
+   touch {config-root}/known-errors/category/error-name.yaml
+   touch {config-root}/known-errors/category/error-name.txt
+   ```
+
+3. **Write YAML definition**
+
+   - Use template structure
+   - Write clear help text with numbered steps
+   - Add fix commands if automation is possible
+
+4. **Create test file**
+
+   - Put actual error output in `.txt` file
+   - Include enough context for pattern matching
+
+5. **Test pattern**
+
+   ```bash
+   scope analyze logs \
+     --extra-config {config-root} \
+     {config-root}/known-errors/category/error-name.txt
+   ```
+
+6. **Validate schema**
+   ```bash
+   jsonschema validate schema.json error-name.yaml
+   ```
+
+## Workflow: Creating Doctor Group
+
+1. **Define problem domain**
+
+   - What needs checking/fixing?
+   - What are the dependencies?
+   - Application or environment level?
+
+2. **Create group file**
+
+   ```bash
+   # Application-level
+   touch {config-root}/application/tool.yaml
+   
+   # Environment-level
+   touch {config-root}/environment/tool.yaml
+   
+   # Project-specific
+   touch .scope/tool.yaml
+   ```
+
+3. **Create helper scripts**
+
+   ```bash
+   mkdir -p .scope/bin
+   touch .scope/bin/tool.sh
+   chmod +x .scope/bin/tool.sh
+   ```
+
+4. **Write group definition**
+
+   - Define metadata (name, description)
+   - List dependencies in `needs`
+   - Create actions for each discrete check
+   - Reference scripts with relative paths
+
+5. **Test the group**
+
+   ```bash
+   scope doctor list                       # Verify detected
+   scope doctor run --only tool            # Test execution
+   scope doctor run --only tool --no-cache # Test without cache
+   ```
+
+6. **Add to parent group**
+   Update parent group's `needs` list if this is a new dependency
+
+## Debugging
+
+### Known Error Not Matching
+
+- Test regex: `rg "pattern" test-file.txt`
+- Check special character escaping
+- Verify pattern exists in test file
+- Use `scope analyze logs --extra-config` with test file
+
+### Doctor Check Always Running
+
+- Verify `paths` globs match files
+- Check commands exit 0 on success
+- Try `--no-cache` to rule out caching
+- Verify script permissions (executable)
+
+### Dependency Issues
+
+- Run `scope doctor list` to see execution order
+- Verify all `needs` items exist
+- Check for circular dependencies
+- Use `--only` to test individual groups
+
+### Script Path Issues
+
+- Use `./` prefix for relative paths (relative to YAML)
+- Ensure scripts have execute permissions
+- Use absolute paths or PATH for system commands
+- Verify working directory
+
+## Security Features
+
+### Automatic Redaction
+
+Scope automatically redacts sensitive information:
+
+- GitHub API tokens
+- AWS credentials
+- SSH keys
+- Environment variable values containing secrets
+- Uses patterns from [ripsecrets](https://github.com/sirwart/ripsecrets)
+
+This makes it safe to capture full environment in reports and share debug output publicly.
+
+## Reference
+
+### Configuration Structure
+
+```
+config-repository/
+├── {config-root}/
+│   ├── application/          # App-level (ruby, node, postgres)
+│   ├── environment/          # System-level (homebrew, docker, github)
+│   ├── known-errors/         # Error definitions by category
+│   │   ├── docker/
+│   │   ├── ruby/
+│   │   ├── git/
+│   │   └── ...
+│   └── reports/              # Report locations
+```
+
+### At Scale
+
+Real-world implementations include:
+
+- 70+ known error definitions
+- 30+ doctor groups
+- Categories: Docker, Ruby, Git, GitHub, MySQL, Postgres, DNS, AWS
+- Reduces time-to-resolution for common issues
+- Self-service troubleshooting
+- Consistent environments across teams
+
+## Additional Resources
+
+See references directory for:
+
+- `quick-reference.md` - Command cheat sheet and pattern examples
+- `real-world-examples.md` - Production-tested configurations
+- `testing-guide.md` - Comprehensive validation workflows

skills/scope/references/quick-reference.md

@@ -0,0 +1,375 @@
+# Scope Quick Reference
+
+Condensed reference for common Scope operations and patterns.
+
+## Command Cheat Sheet
+
+```bash
+# Doctor
+scope doctor run                   # Run all checks
+scope doctor run --only group-name # Run specific group
+scope doctor run --fix=false       # Check only
+scope doctor run --no-cache        # Force re-check
+scope doctor list                  # List all checks
+
+# Analyze
+scope analyze logs file.log            # Check log for errors
+scope analyze command -- cmd args      # Check command output
+scope analyze --extra-config dir/ file # Use extra configs
+
+# Report
+scope report ./script.sh     # Run & report script
+scope report -- command args # Run & report command
+
+# Version
+scope version # Show version
+```
+
+## YAML Templates
+
+### Known Error Template
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeKnownError
+metadata:
+  name: error-name
+  description: What this error means
+spec:
+  pattern: 'regex pattern'
+  help: |
+    Explanation and steps to fix:
+    1. First step
+    2. Second step
+  fix:
+    prompt:
+      text: Permission prompt
+    commands:
+      - fix-command
+```
+
+### Doctor Group Template
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: group-name
+  description: What this group does
+spec:
+  include: when-required
+  needs:
+    - dependency-1
+  actions:
+    - name: action-name
+      description: What this checks
+      required: true
+      check:
+        paths:
+          - 'file.txt'
+        commands:
+          - ./bin/check.sh
+      fix:
+        commands:
+          - ./bin/fix.sh
+        helpText: |
+          Help if fix fails
+```
+
+### Report Location Template
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeReportLocation
+metadata:
+  name: location-name
+spec:
+  destination:
+    local:
+      directory: /tmp/reports
+    # OR githubIssue:
+    #   owner: org
+    #   repo: repo
+    # OR rustyPaste:
+    #   url: http://localhost:8000
+  additionalData:
+    command1: command-to-run
+```
+
+## Regex Pattern Examples
+
+```yaml
+# Match specific version error
+pattern: "ruby [0-9]+\\.[0-9]+\\.[0-9]+ is not installed"
+
+# Match file not found
+pattern: "cannot load such file -- .*/([^/]+)\\.(rb|so)"
+
+# Match Docker daemon not running
+pattern: "\\.colima/[^/]+/docker\\.sock.*Is the docker daemon running\\?"
+
+# Match Git lock error
+pattern: "Unable to create '.*\\.git/refs/heads/.*\\.lock'"
+
+# Match DNS resolution failure
+pattern: "Could not resolve host: ([a-zA-Z0-9.-]+)"
+
+# Match permission denied
+pattern: "Permission denied.*(/[^:]+)"
+
+# Character classes
+[[:digit:]]   # 0-9
+[[:alpha:]]   # a-z, A-Z
+[[:alnum:]]   # alphanumeric
+[[:space:]]   # whitespace
+
+# Quantifiers
+*             # 0 or more
++             # 1 or more
+?             # 0 or 1
+{n,m}         # between n and m
+
+# Escaping
+\\.           # literal dot
+\\[           # literal [
+\\(           # literal (
+```
+
+## Check/Fix Script Template
+
+```bash
+#!/usr/bin/env bash
+set -euo pipefail
+
+ACTION="${1:-check}"
+
+check() {
+  # Exit 0 if OK, non-zero if needs fix
+  if [[ condition ]]; then
+    return 0
+  else
+    echo "Check failed: reason" >&2
+    return 1
+  fi
+}
+
+fix() {
+  # Perform fix, exit 0 on success
+  echo "Fixing..."
+  # commands
+  return 0
+}
+
+case "$ACTION" in
+  check) check ;;
+  fix) fix ;;
+  *)
+    echo "Usage: $0 [check|fix]" >&2
+    exit 1
+    ;;
+esac
+```
+
+## Common Patterns
+
+### Version Check
+
+```yaml
+- name: min-version
+  description: Check minimum version
+  check:
+    commands:
+      - test "$(tool --version | cut -d' ' -f2)" = "1.2.3"
+  fix:
+    helpText: Update tool via Managed Software Center
+```
+
+### File Exists
+
+```yaml
+- name: config-exists
+  description: Config file exists
+  check:
+    commands:
+      - test -f .config
+  fix:
+    commands:
+      - ./bin/create-config.sh
+```
+
+### Service Running
+
+```yaml
+- name: service-up
+  description: Service is running
+  check:
+    commands:
+      - pgrep -x service-name
+  fix:
+    commands:
+      - brew services restart service-name
+```
+
+### Dependencies Installed
+
+```yaml
+- name: deps
+  description: Dependencies installed
+  check:
+    paths:
+      - package.json
+      - yarn.lock
+  fix:
+    commands:
+      - yarn install
+```
+
+### Path-Based Auto-Run
+
+```yaml
+# Runs fix when file changes
+- name: auto-update
+  check:
+    paths:
+      - config.yaml
+      - '**/*.conf'
+  fix:
+    commands:
+      - ./bin/reload.sh
+```
+
+## Validation Workflow
+
+```bash
+# 1. Create error file
+mkdir -p {config-root}/known-errors/category
+cat > {config-root}/known-errors/category/error.yaml << 'EOF'
+apiVersion: scope.github.com/v1alpha
+kind: ScopeKnownError
+metadata:
+  name: my-error
+spec:
+  pattern: "error pattern"
+  help: How to fix
+EOF
+
+# 2. Create test file with actual error
+cat > {config-root}/known-errors/category/error.txt << 'EOF'
+Actual error output goes here
+EOF
+
+# 3. Test pattern
+scope analyze logs \
+  --extra-config {config-root} \
+  {config-root}/known-errors/category/error.txt
+
+# 4. Validate schema (if available)
+jsonschema validate schema.json error.yaml
+```
+
+## File Organization
+
+```
+# Gusto shared configs
+{config-root}/
+├── application/              # App-level (ruby, node, db)
+├── environment/              # System-level (homebrew, git)
+├── known-errors/
+│   ├── docker/
+│   ├── ruby/
+│   ├── git/
+│   └── {category}/
+│       ├── error-name.yaml
+│       └── error-name.txt   # Test file
+└── reports/
+
+# Project-specific
+.scope/
+├── project-name.yaml         # Main orchestrator
+├── db.yaml                  # Database setup
+├── ruby.yaml                # Language setup
+└── bin/                     # Helper scripts
+    ├── check-*.sh
+    └── fix-*.sh
+```
+
+## Debugging Checklist
+
+### Known Error Not Matching
+
+- [ ] Test regex: `echo "error" | rg "pattern"`
+- [ ] Check escaping of special chars
+- [ ] Verify test file has actual error
+- [ ] Try broader pattern first
+
+### Doctor Always Runs
+
+- [ ] Check path globs match: `ls -la path/pattern`
+- [ ] Verify check command exits 0: `./bin/check.sh; echo $?`
+- [ ] Try `--no-cache`
+- [ ] Check script is executable: `ls -l script.sh`
+
+### Dependencies Not Working
+
+- [ ] Run `scope doctor list` - see order
+- [ ] Verify `needs` names match exactly
+- [ ] Check for circular deps
+- [ ] Test with `--only group-name`
+
+### Script Issues
+
+- [ ] Add `set -euo pipefail` to scripts
+- [ ] Check relative path has `./` prefix
+- [ ] Make executable: `chmod +x script.sh`
+- [ ] Test standalone: `./bin/script.sh check`
+
+## Testing Tips
+
+```bash
+# Test regex patterns
+echo "error text here" | rg "pattern"
+
+# Test check command
+./bin/check.sh check
+echo "Exit code: $?"
+
+# Test doctor group in isolation
+scope doctor run --only group-name --no-cache
+
+# See what would run
+scope doctor list | grep group-name
+
+# Test with extra config
+scope analyze --extra-config /path/to/config file.log
+
+# Validate YAML syntax
+yamllint file.yaml
+
+# Check file matching
+ls -la path/to/files/**/*.pattern
+```
+
+## Environment Variables
+
+```bash
+# Report authentication
+SCOPE_GH_TOKEN=ghp_xxx        # GitHub PAT
+SCOPE_GH_APP_ID=123           # GitHub App
+SCOPE_GH_APP_KEY=/path/to/key # GitHub App key
+
+# Template variables (in YAML)
+{{ working_dir }} # Current working directory
+```
+
+## Common Gotchas
+
+1. **Regex escaping**: Use `\\.` for literal dot, not `.`
+2. **Relative paths**: Must start with `./` (relative to YAML file)
+3. **Check exit codes**: 0 = pass, non-zero = needs fix
+4. **Cache persistence**: Use `--no-cache` when testing
+5. **Pattern specificity**: Too broad = false positives, too narrow = misses errors
+6. **Script permissions**: Must be executable (`chmod +x`)
+7. **YAML indentation**: Use 2 spaces, not tabs
+8. **Action order**: Actions run in order defined
+9. **Dependency order**: Use `scope doctor list` to verify
+10. **Help text**: Use `|` for multi-line strings in YAML

skills/scope/references/real-world-examples.md

@@ -0,0 +1,579 @@
+# Real-World Scope Examples
+
+Curated examples from production Scope configurations showing battle-tested patterns.
+
+## Known Errors
+
+### Docker: Colima Not Running
+
+**File**: `known-errors/docker/default-colima-not-running.yaml`
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeKnownError
+metadata:
+  name: default-colima-not-running
+  description: The default instance of Colima is not running
+spec:
+  pattern: '.colima/default/docker.sock. Is the docker daemon running?'
+  help: |
+    Colima is not running. Start it by:
+    1. `scope doctor run --only company/docker@v1`
+
+    If that doesn't resolve the issue, reach out to us at @team
+    in the #help-channel channel in Slack for help.
+  fix:
+    prompt:
+      text: Run scope doctor?
+    commands:
+      - scope doctor run --only company/docker@v1
+```
+
+**Key Patterns**:
+
+- Uses escaped dot in path pattern: `\\.colima/`
+- Provides clear escalation path
+- Fix delegates to doctor group for complex multi-step resolution
+- Includes Slack channel for human help
+
+### Ruby: Gem Missing File
+
+**File**: `known-errors/ruby/gem-missing-file.yaml`
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeKnownError
+metadata:
+  name: gem-missing-file
+  description: A gem source file is missing, and fails to be loaded
+spec:
+  pattern: "/lib/ruby/([[:digit:]]\\.[[:digit:]]\\.[[:digit:]]|gems)/.* `(require|require_relative)': cannot load such file --.*/lib/ruby/gems/.*(LoadError)"
+  help: |
+    A gem source file is missing and fails to be loaded. The cause of this is
+    unknown and still being investigated (TICKET-123).
+
+    The solution is to reinstall the gems to fix the missing file:
+    1. Run `bundle pristine`
+  fix:
+    prompt:
+      text: Run bundle pristine?
+    commands:
+      - bundle pristine
+```
+
+**Key Patterns**:
+
+- Complex regex with alternation: `([[:digit:]]\\.[[:digit:]]\\.[[:digit:]]|gems)`
+- Uses character classes: `[[:digit:]]`
+- Multiple escaped characters in paths
+- References tracking issue in help text
+- Simple, safe fix command
+
+### Git: Cannot Lock Ref
+
+**File**: `known-errors/git/cannot-lock-ref.yaml`
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeKnownError
+metadata:
+  name: cannot-lock-ref
+  description: Git cannot create lock file for ref
+spec:
+  pattern: "error: cannot lock ref '[^']+': Unable to create '[^']+\\.lock': File exists"
+  help: |
+    Another git process is running or crashed leaving a lock file.
+
+    To resolve:
+    1. Check for running git processes: `ps aux | grep git`
+    2. If none running, remove the lock file mentioned in the error
+    3. Example: `rm .git/refs/heads/branch-name.lock`
+  fix:
+    prompt:
+      text: This requires manual intervention. Proceed with caution?
+    commands:
+      - echo "Check for git processes: ps aux | grep git"
+      - echo "If safe, manually remove the .lock file mentioned above"
+```
+
+**Key Patterns**:
+
+- Uses character class negation: `[^']+` (anything except single quote)
+- Escaped special characters: `\\.lock`
+- Fix provides diagnostic commands rather than automated fix
+- Warns user about manual intervention
+
+### MySQL: Connection Refused
+
+**File**: `known-errors/mysql/trilogy-connection-refused.yaml`
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeKnownError
+metadata:
+  name: trilogy-connection-refused
+  description: MySQL connection refused, service may not be running
+spec:
+  pattern: "Trilogy::ConnectionRefusedError.*Connection refused - connect\\(2\\)"
+  help: |
+    MySQL/MariaDB is not running or not accepting connections.
+
+    To fix:
+    1. Check if service is running: `brew services list | grep mysql`
+    2. Start the service: `brew services start mysql@8.0`
+    3. Or run: `scope doctor run --only database`
+  fix:
+    prompt:
+      text: Attempt to start MySQL service?
+    commands:
+      - brew services restart mysql@8.0
+```
+
+**Key Patterns**:
+
+- Escaped parentheses in regex: `\\(2\\)`
+- Provides multiple resolution paths
+- Delegates to doctor group for comprehensive fix
+- Uses `restart` instead of `start` (idempotent)
+
+## Doctor Groups
+
+### Ruby Version Management
+
+**File**: `application/ruby-version.yaml`
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: ruby-version
+  description: Set up Ruby with accurate version
+spec:
+  include: when-required
+  needs:
+    - ruby-manager
+  actions:
+    - name: .ruby-version
+      description: Verify a valid .ruby-version file is present.
+      check:
+        commands:
+          - test -s .ruby-version
+      fix:
+        helpText: |
+          The .ruby-version file must exist and not be blank.
+    - name: install
+      description: Ensures correct version of ruby is installed
+      check:
+        paths:
+          - '{{ working_dir }}/.ruby-version'
+      fix:
+        commands:
+          - ./bin/ruby-version.sh install
+    - name: verify
+      description: Verify the desired ruby version and current version are the same
+      check:
+        commands:
+          - ./bin/ruby-version.sh verify
+      fix:
+        helpText: |
+          Something went wrong.
+          The ruby version was installed, but is not the version available in your current shell.
+          See error messages above for additional details and possible solutions.
+```
+
+**Key Patterns**:
+
+- Multiple sequential actions building on each other
+- First action has no fix commands, only helpText (manual intervention)
+- Second action watches file changes with `paths`
+- Third action validates end state
+- Uses template variable: `{{ working_dir }}`
+- Delegates complex logic to external script
+
+### Colima (Docker) Setup
+
+**File**: `environment/colima.yaml`
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: company/docker@v1
+  description: Colima
+spec:
+  include: when-required
+  needs:
+    - homebrew
+  actions:
+    - name: install
+      description: company-docker is installed
+      check:
+        commands:
+          - ./bin/colima.sh check install
+      fix:
+        commands:
+          - ./bin/colima.sh fix install
+    - name: profile
+      description: The gusto profile exists
+      check:
+        commands:
+          - ./bin/colima.sh check profile
+      fix:
+        commands:
+          - ./bin/colima.sh fix profile
+        helpText: |-
+          The ~/.colima/company.yaml file still doesn't exist after running `sudo config-management`.
+          Please contact #ops-channel in slack.
+    - name: running
+      description: service and vm are running
+      check:
+        commands:
+          - ./bin/colima.sh check running
+      fix:
+        commands:
+          - ./bin/colima.sh fix running
+        helpText: |-
+          We were unable to start the company-docker service and/or the colima vm.
+          Please review the logs.
+
+          tail "$(brew --prefix)/var/log/service.log"
+
+          If you are not able to resolve the issue,
+          please contact #help-channel in slack.
+    - name: docker-context
+      description: docker context is set to gusto
+      check:
+        commands:
+          - ./bin/colima.sh check context
+      fix:
+        commands:
+          - ./bin/colima.sh fix context
+    - name: default-service
+      description: The default colima brew service is stopped
+      required: false
+      check:
+        commands:
+          - ./bin/colima.sh check default-service
+      fix:
+        commands:
+          - ./bin/colima.sh fix default-service
+    - name: default-profile
+      description: The default colima profile is stopped
+      required: false
+      check:
+        commands:
+          - ./bin/colima.sh check default-profile
+      fix:
+        commands:
+          - ./bin/colima.sh fix default-profile
+```
+
+**Key Patterns**:
+
+- Versioned name: `company/docker@v1` (allows breaking changes)
+- All actions delegate to same script with subcommands
+- Mix of required and optional actions
+- Complex multi-step setup
+- Detailed helpText with log locations
+- Shell command expansion in helpText: `$(brew --prefix)`
+- Last two actions are optional cleanup (`required: false`)
+
+### Brewfile Package Management
+
+**File**: `application/brewfile.yaml`
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: brewfile
+  description: Homebrew managed packages
+spec:
+  include: when-required
+  needs:
+    - github-cli
+    - homebrew
+  actions:
+    - name: brew-bundle
+      description: Install Homebrew packages from Brewfile
+      check:
+        commands:
+          - ./bin/brew-bundle.sh check
+      fix:
+        commands:
+          - ./bin/brew-bundle.sh fix
+        helpText: |
+          brew dependencies cannot be satisfied
+
+          Please review the output above for errors and possible solutions.
+          If you need assistance, please contact #help-channel in slack.
+```
+
+**Key Patterns**:
+
+- Multiple dependencies ensure prerequisites installed first
+- Single action with simple check/fix delegation
+- Generic helpText directs to previous output
+- Minimal but effective
+
+### Version Requirements Check
+
+**File**: `.scope/scope.yaml` (project-specific)
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: scope
+spec:
+  include: when-required
+  needs: []
+  actions:
+    - name: minimum-scope-version
+      description: Ensures we have at least the minimum version of scope installed
+      check:
+        commands:
+          - ./bin/check-scope-version.sh check scope 2024.2.68
+      fix:
+        helpText: |
+          You don't have the minimum version of scope installed.
+          Check the Managed Software Center for updates.
+
+          If that doesn't work, please contact #help-channel in slack.
+    - name: minimum-gusto-scope-version
+      description: Ensures we have at least the minimum version of scope installed
+      check:
+        commands:
+          - ./bin/check-scope-version.sh check gusto 2025.05.15.0001
+      fix:
+        helpText: |
+          You don't have the minimum version of scope_config installed.
+          Check the Managed Software Center for updates.
+
+          If that doesn't work, please contact #help-channel in slack.
+```
+
+**Key Patterns**:
+
+- No dependencies (runs first)
+- No automated fix (requires external tool)
+- Passes version as argument to script
+- Consistent helpText pattern across actions
+
+### Orchestrator Pattern
+
+**File**: `.scope/project.yaml` (project-specific)
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: project
+  description: Application setup
+spec:
+  needs:
+    - scope
+    - company/environment@v1
+    - brewfile
+    - company/ruby@v1
+    - company/javascript@v1
+    - gitconfig
+    - lefthook
+    - db
+    - rubymine
+    - ruby-next
+    - kafka
+  actions: []
+```
+
+**Key Patterns**:
+
+- No actions, only dependencies
+- Orchestrates entire setup in correct order
+- Acts as entrypoint for `scope doctor run`
+- Clear dependency chain
+
+## Report Location
+
+**File**: `reports/report-location.yaml`
+
+```yaml
+apiVersion: scope.github.com/v1alpha
+kind: ScopeReportLocation
+metadata:
+  name: local
+spec:
+  destination:
+    local:
+      directory: /tmp/scope-reports
+  additionalData:
+    pwd: pwd
+    username: id -un
+    ruby: which ruby
+    node: which node
+    nodeVersion: node -v
+    scopeVersion: scope version
+    configVersion: config-tool --version
+```
+
+**Key Patterns**:
+
+- Local filesystem destination (no auth required)
+- Captures environment context
+- Uses simple shell commands
+- Platform-specific command: `pkgutil` (macOS)
+- Mix of path commands (`which`) and version commands
+
+## Helper Script Examples
+
+### Check/Fix Pattern
+
+**Example**: `bin/ruby-version.sh`
+
+```bash
+#!/usr/bin/env bash
+set -euo pipefail
+
+ACTION="${1:-}"
+COMMAND="${2:-}"
+
+check_file_exists() {
+  test -s .ruby-version
+}
+
+install_version() {
+  local desired_version
+  desired_version=$(cat .ruby-version)
+
+  if mise which ruby &> /dev/null; then
+    echo "Ruby is already available"
+    return 0
+  fi
+
+  mise install ruby@"${desired_version}"
+}
+
+verify_version() {
+  local desired_version current_version
+  desired_version=$(cat .ruby-version)
+  current_version=$(ruby --version | awk '{print $2}')
+
+  if [[ "$desired_version" == "$current_version" ]]; then
+    return 0
+  else
+    echo "Desired: $desired_version, Current: $current_version" >&2
+    return 1
+  fi
+}
+
+case "$ACTION" in
+  check)
+    check_file_exists
+    ;;
+  install)
+    install_version
+    ;;
+  verify)
+    verify_version
+    ;;
+  *)
+    echo "Usage: $0 [check|install|verify]" >&2
+    exit 1
+    ;;
+esac
+```
+
+**Key Patterns**:
+
+- Supports multiple subcommands
+- Extracts values from files
+- Uses command substitution
+- Provides clear error messages to stderr
+- Returns appropriate exit codes
+
+### Version Comparison
+
+**Example**: `bin/check-scope-version.sh`
+
+```bash
+#!/usr/bin/env bash
+set -euo pipefail
+
+ACTION="${1:-}"
+PACKAGE="${2:-}"
+MIN_VERSION="${3:-}"
+
+check_version() {
+  local current_version
+
+  case "$PACKAGE" in
+    scope)
+      current_version=$(scope version 2>&1 | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1)
+      ;;
+    gusto)
+      current_version=$(config-tool --version 2>&1 | awk '/version:/ {print $2}')
+      ;;
+    *)
+      echo "Unknown package: $PACKAGE" >&2
+      exit 1
+      ;;
+  esac
+
+  if [[ "$(printf '%s\n' "$MIN_VERSION" "$current_version" | sort -V | head -1)" == "$MIN_VERSION" ]]; then
+    echo "Version $current_version meets minimum $MIN_VERSION"
+    return 0
+  else
+    echo "Version $current_version does not meet minimum $MIN_VERSION" >&2
+    return 1
+  fi
+}
+
+case "$ACTION" in
+  check)
+    check_version
+    ;;
+  *)
+    echo "Usage: $0 check <package> <min-version>" >&2
+    exit 1
+    ;;
+esac
+```
+
+**Key Patterns**:
+
+- Semantic version comparison using `sort -V`
+- Multiple package sources
+- Regex extraction of version numbers
+- Parameter validation
+
+## Lessons from Production
+
+### What Works Well
+
+1. **Versioned group names** (`company/docker@v1`) allow non-breaking changes
+2. **Orchestrator groups** with no actions simplify complex setups
+3. **Optional actions** (`required: false`) for nice-to-haves
+4. **Delegating to scripts** keeps YAML simple, logic testable
+5. **Consistent naming** (category/tool pattern) aids discovery
+6. **Rich helpText** with log locations and Slack channels
+7. **Multiple fix strategies** in help text (auto, manual, escalate)
+
+### Common Pitfalls
+
+1. **Overly broad patterns** catch unrelated errors
+2. **Missing escaping** in regex patterns
+3. **Hardcoded paths** instead of variables
+4. **Complex logic in YAML** instead of scripts
+5. **Missing error messages** when checks fail
+6. **No test files** make pattern validation harder
+7. **Circular dependencies** between groups
+
+### Scale Insights
+
+At 70+ known errors and 30+ doctor groups:
+
+- Categorization prevents overwhelming users
+- Consistent patterns make contribution easier
+- Test files are essential for maintenance
+- Versioning enables evolution without breaking changes
+- Clear ownership (Slack channels) reduces support burden

skills/scope/references/testing-guide.md

@@ -0,0 +1,661 @@
+# Scope Testing Guide
+
+Comprehensive guide for testing Scope configurations before deployment.
+
+## Testing Philosophy
+
+1. **Test patterns in isolation** before adding to YAML
+2. **Validate schema** before testing functionality
+3. **Test with real errors** using `.txt` files
+4. **Test incrementally** (pattern → YAML → integration)
+5. **Automate regression tests** for known errors
+
+## Known Error Testing
+
+### 1. Test Regex Pattern in Isolation
+
+```bash
+# Test pattern matches expected text
+echo "error: cannot lock ref 'refs/heads/main'" | rg "cannot lock ref"
+# Output: error: cannot lock ref 'refs/heads/main'
+
+# Test pattern doesn't match unrelated text
+echo "error: something else" | rg "cannot lock ref"
+# Output: (nothing - no match)
+
+# Test with actual error file
+rg "pattern" path/to/error-output.log
+```
+
+**Common Issues**:
+
+- Pattern too broad: matches too many things
+- Pattern too specific: misses variations
+- Missing escapes: special chars break regex
+- Wrong quantifiers: `*` vs `+` vs `?`
+
+### 2. Create Test File
+
+```bash
+# Create directory
+mkdir -p {config-root}/known-errors/category
+
+# Create test file with ACTUAL error output
+cat > {config-root}/known-errors/category/error-name.txt << 'EOF'
+[Paste actual error output here]
+This should be the real error text that users see
+Including all the context around it
+EOF
+```
+
+**Best Practices**:
+
+- Use real error output, not synthetic examples
+- Include surrounding context (lines before/after)
+- Test multiple variations if error has variants
+- Keep file size reasonable (< 100 lines)
+
+### 3. Create YAML Definition
+
+```bash
+cat > {config-root}/known-errors/category/error-name.yaml << 'EOF'
+apiVersion: scope.github.com/v1alpha
+kind: ScopeKnownError
+metadata:
+  name: error-name
+  description: Brief description
+spec:
+  pattern: "your pattern here"
+  help: |
+    How to fix this error
+EOF
+```
+
+### 4. Test Pattern Matching
+
+```bash
+# Test that pattern matches the test file
+scope analyze logs \
+  --extra-config {config-root} \
+  {config-root}/known-errors/category/error-name.txt
+```
+
+**Expected Output** (pattern matches):
+
+```
+Known Error: error-name
+Brief description
+
+How to fix this error
+```
+
+**Expected Output** (pattern doesn't match):
+
+```
+No known errors found
+```
+
+### 5. Validate Schema
+
+```bash
+# Install validator
+brew install sourcemeta/apps/jsonschema
+
+# Get schema (one-time setup)
+curl -o /tmp/ScopeKnownError.json \
+  https://github.com/oscope-dev/scope/raw/main/scope/schema/v1alpha.com.github.scope.ScopeKnownError.json
+
+# Validate
+jsonschema validate \
+  /tmp/ScopeKnownError.json \
+  {config-root}/known-errors/category/error-name.yaml
+```
+
+**Expected Output** (valid):
+
+```
+ok: {config-root}/known-errors/category/error-name.yaml
+```
+
+### 6. Test in Real Scenario
+
+```bash
+# Run the actual failing command and pipe to scope
+failing-command 2>&1 | scope analyze command
+
+# Or analyze existing log file
+scope analyze logs /path/to/error.log
+```
+
+## Doctor Group Testing
+
+### 1. Test Check Script Standalone
+
+```bash
+# Create test script
+cat > .scope/bin/test-check.sh << 'EOF'
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Your check logic here
+if [[ -f .required-file ]]; then
+  echo "Check passed"
+  exit 0
+else
+  echo "Check failed: .required-file missing" >&2
+  exit 1
+fi
+EOF
+
+chmod +x .scope/bin/test-check.sh
+
+# Test success case
+touch .required-file
+./.scope/bin/test-check.sh
+echo "Exit code: $?"
+# Expected: Check passed, Exit code: 0
+
+# Test failure case
+rm .required-file
+./.scope/bin/test-check.sh
+echo "Exit code: $?"
+# Expected: Check failed: .required-file missing, Exit code: 1
+```
+
+### 2. Test Fix Script Standalone
+
+```bash
+# Create fix script
+cat > .scope/bin/test-fix.sh << 'EOF'
+#!/usr/bin/env bash
+set -euo pipefail
+
+echo "Creating .required-file..."
+touch .required-file
+echo "Done"
+EOF
+
+chmod +x .scope/bin/test-fix.sh
+
+# Test fix
+rm -f .required-file
+./.scope/bin/test-fix.sh
+ls -la .required-file
+# Expected: File created
+```
+
+### 3. Create Doctor Group YAML
+
+```bash
+cat > .scope/test-group.yaml << 'EOF'
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: test-group
+  description: Test group for validation
+spec:
+  include: when-required
+  needs: []
+  actions:
+    - name: test-action
+      description: Test action
+      check:
+        commands:
+          - ./bin/test-check.sh
+      fix:
+        commands:
+          - ./bin/test-fix.sh
+EOF
+```
+
+### 4. Verify Group is Detected
+
+```bash
+# List all groups
+scope doctor list | grep test-group
+
+# Expected output includes:
+# ScopeDoctorGroup/test-group    Test group for validation    .scope/test-group.yaml
+```
+
+### 5. Test Check-Only Mode
+
+```bash
+# Remove file to trigger check failure
+rm -f .required-file
+
+# Run in check-only mode
+scope doctor run --only test-group --fix=false
+
+# Expected: Shows check failed, but doesn't run fix
+```
+
+### 6. Test With Fix
+
+```bash
+# Remove file to trigger check failure
+rm -f .required-file
+
+# Run with fix enabled
+scope doctor run --only test-group
+
+# Expected: Check fails, fix runs, file created
+ls -la .required-file
+```
+
+### 7. Test Caching
+
+```bash
+# First run (should execute check)
+scope doctor run --only test-group
+# Expected: Runs check and fix if needed
+
+# Second run (should use cache if file-based)
+scope doctor run --only test-group
+# Expected: Skips if using path-based checks
+
+# Force re-run
+scope doctor run --only test-group --no-cache
+# Expected: Runs check again
+```
+
+### 8. Test Dependencies
+
+```bash
+# Create dependency group
+cat > .scope/dependency.yaml << 'EOF'
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: dependency
+spec:
+  include: when-required
+  needs: []
+  actions:
+    - name: setup
+      check:
+        commands:
+          - test -f .dependency-marker
+      fix:
+        commands:
+          - touch .dependency-marker
+EOF
+
+# Update test group to depend on it
+# Add to test-group.yaml:
+# needs:
+#   - dependency
+
+# Test dependency resolution
+rm -f .dependency-marker .required-file
+scope doctor run --only test-group
+
+# Expected: Runs dependency first, then test-group
+ls -la .dependency-marker .required-file
+```
+
+### 9. Validate Schema
+
+```bash
+# Get schema
+curl -o /tmp/ScopeDoctorGroup.json \
+  https://github.com/oscope-dev/scope/raw/main/scope/schema/v1alpha.com.github.scope.ScopeDoctorGroup.json
+
+# Validate
+jsonschema validate \
+  /tmp/ScopeDoctorGroup.json \
+  .scope/test-group.yaml
+```
+
+## Integration Testing
+
+### Test Complete Workflow
+
+```bash
+# 1. Clean slate
+rm -rf /tmp/scope-test
+mkdir -p /tmp/scope-test/.scope/bin
+
+# 2. Create complete setup
+cd /tmp/scope-test
+
+# Create check script
+cat > .scope/bin/check.sh << 'EOF'
+#!/usr/bin/env bash
+test -f .setup-complete
+EOF
+chmod +x .scope/bin/check.sh
+
+# Create fix script
+cat > .scope/bin/fix.sh << 'EOF'
+#!/usr/bin/env bash
+echo "Setting up..."
+sleep 1
+touch .setup-complete
+echo "Done"
+EOF
+chmod +x .scope/bin/fix.sh
+
+# Create doctor group
+cat > .scope/setup.yaml << 'EOF'
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: setup
+spec:
+  include: by-default
+  needs: []
+  actions:
+    - name: initialize
+      description: Initialize project
+      check:
+        commands:
+          - ./.scope/bin/check.sh
+      fix:
+        commands:
+          - ./.scope/bin/fix.sh
+        helpText: |
+          Setup failed. Try running manually:
+          ./.scope/bin/fix.sh
+EOF
+
+# 3. Test end-to-end
+scope doctor run
+
+# 4. Verify result
+test -f .setup-complete && echo "SUCCESS" || echo "FAILED"
+```
+
+## Regression Testing
+
+### Create Test Suite
+
+```bash
+# Create test runner
+cat > test-scope.sh << 'EOF'
+#!/usr/bin/env bash
+set -euo pipefail
+
+PASSED=0
+FAILED=0
+
+test_known_error() {
+  local name=$1
+  local yaml=$2
+  local test_file=$3
+
+  echo "Testing: $name"
+
+  if scope analyze logs --extra-config {config-root} "$test_file" | grep -q "$name"; then
+    echo "  ✓ Pattern matches"
+    ((PASSED++))
+  else
+    echo "  ✗ Pattern does not match"
+    ((FAILED++))
+  fi
+}
+
+# Test all known errors
+for yaml in {config-root}/known-errors/*/*.yaml; do
+  name=$(basename "$yaml" .yaml)
+  txt="${yaml%.yaml}.txt"
+
+  if [[ -f "$txt" ]]; then
+    test_known_error "$name" "$yaml" "$txt"
+  fi
+done
+
+echo ""
+echo "Results: $PASSED passed, $FAILED failed"
+
+exit $FAILED
+EOF
+
+chmod +x test-scope.sh
+
+# Run test suite
+./test-scope.sh
+```
+
+### Continuous Testing
+
+```bash
+# Watch for changes and re-test
+while true; do
+  inotifywait -r -e modify {config-root}/
+  ./test-scope.sh
+done
+```
+
+## Performance Testing
+
+### Measure Doctor Run Time
+
+```bash
+# Time complete run
+time scope doctor run
+
+# Time specific group
+time scope doctor run --only group-name
+
+# Compare cached vs uncached
+time scope doctor run            # With cache
+time scope doctor run --no-cache # Without cache
+```
+
+### Profile Cache Effectiveness
+
+```bash
+# First run (cold cache)
+scope doctor run > /tmp/run1.log 2>&1
+
+# Second run (warm cache)
+scope doctor run > /tmp/run2.log 2>&1
+
+# Compare
+diff /tmp/run1.log /tmp/run2.log
+```
+
+## Debugging Tests
+
+### Enable Verbose Output
+
+```bash
+# Scope doesn't have --verbose, but you can debug scripts
+# Add to scripts:
+set -x # Print commands before execution
+
+# Example:
+#!/usr/bin/env bash
+set -euxo pipefail # Added 'x' for debug output
+```
+
+### Capture Full Output
+
+```bash
+# Capture stdout and stderr
+scope doctor run --only group-name > /tmp/stdout.log 2> /tmp/stderr.log
+
+# Capture combined
+scope doctor run --only group-name &> /tmp/combined.log
+
+# Capture and display
+scope doctor run --only group-name 2>&1 | tee /tmp/output.log
+```
+
+### Test Specific Action
+
+```bash
+# Doctor groups run all actions in sequence
+# To test just one action, you can run the script directly:
+./.scope/bin/script.sh check
+./.scope/bin/script.sh fix
+```
+
+## Common Test Scenarios
+
+### Test: Pattern with Special Characters
+
+```bash
+# Pattern: "error: cannot lock ref 'refs/heads/main'"
+# Test: Does it escape properly?
+
+echo "error: cannot lock ref 'refs/heads/main'" > /tmp/test.txt
+
+# This should match:
+rg "cannot lock ref '[^']+'" /tmp/test.txt
+
+# This should NOT match (missing escape):
+rg "cannot lock ref 'refs/heads/main'" /tmp/test.txt # Literal string
+```
+
+### Test: Path-Based Check Triggers on Change
+
+```bash
+# Setup group with path check
+cat > .scope/path-test.yaml << 'EOF'
+spec:
+  actions:
+    - name: test
+      check:
+        paths:
+          - config.yaml
+      fix:
+        commands:
+          - echo "Config changed"
+EOF
+
+# First run - config doesn't exist, should trigger
+scope doctor run --only path-test
+
+# Create config
+echo "version: 1" > config.yaml
+
+# Second run - config exists, should trigger (first time seeing it)
+scope doctor run --only path-test
+
+# Third run - config unchanged, should NOT trigger
+scope doctor run --only path-test
+
+# Change config
+echo "version: 2" > config.yaml
+
+# Fourth run - config changed, should trigger
+scope doctor run --only path-test
+```
+
+### Test: Dependency Order
+
+```bash
+# Create test that verifies order
+cat > .scope/dep-order-test.yaml << 'EOF'
+spec:
+  needs:
+    - first
+    - second
+  actions:
+    - name: verify
+      check:
+        commands:
+          - test -f .first-ran
+          - test -f .second-ran
+EOF
+
+# Run and check log order
+scope doctor run --only dep-order-test 2>&1 | grep -E "(first|second|verify)"
+```
+
+## Best Practices
+
+### DO
+
+✓ Test regex patterns with `rg` before adding to YAML
+✓ Create `.txt` test files with real error output
+✓ Validate schema before testing functionality
+✓ Test scripts standalone before integrating
+✓ Use `--no-cache` when testing changes
+✓ Test both success and failure paths
+✓ Test dependency resolution
+✓ Keep test files small and focused
+✓ Use version control to track test changes
+
+### DON'T
+
+✗ Skip testing regex patterns in isolation
+✗ Use synthetic error examples
+✗ Assume patterns work without testing
+✗ Test only the happy path
+✗ Forget to make scripts executable
+✗ Hard-code paths in tests
+✗ Test in production first
+✗ Commit without validation
+
+## Troubleshooting Tests
+
+### Pattern doesn't match test file
+
+```bash
+# Debug steps:
+# 1. Check file encoding
+file -I test-file.txt
+
+# 2. Check for hidden characters
+cat -A test-file.txt
+
+# 3. Test pattern piece by piece
+rg "simple" test-file.txt
+rg "simple.*pattern" test-file.txt
+rg "full.*complex.*pattern" test-file.txt
+
+# 4. Check escaping
+rg "pattern with \. dot" test-file.txt
+```
+
+### Check always fails
+
+```bash
+# Debug steps:
+# 1. Run script manually
+./.scope/bin/check.sh
+echo "Exit code: $?"
+
+# 2. Check for syntax errors
+bash -n ./.scope/bin/check.sh
+
+# 3. Add debug output
+set -x in script
+
+# 4. Check permissions
+ls -l ./.scope/bin/check.sh
+```
+
+### Fix runs but doesn't work
+
+```bash
+# Debug steps:
+# 1. Check fix script exit code
+./.scope/bin/fix.sh
+echo "Exit code: $?"
+
+# 2. Check what fix actually does
+./.scope/bin/fix.sh
+ls -la # Check if files created
+
+# 3. Run check after fix
+./.scope/bin/fix.sh && ./.scope/bin/check.sh
+echo "Check exit code: $?"
+```
+
+### Cache causes issues
+
+```bash
+# Solutions:
+# 1. Always use --no-cache when testing
+scope doctor run --only test --no-cache
+
+# 2. Clear cache manually (implementation-specific)
+# Check scope docs for cache location
+
+# 3. Use command-based checks instead of path-based
+# Commands don't use cache
+```

skills/scope/scripts/create-doctor-group.sh

@@ -0,0 +1,205 @@
+#!/usr/bin/env bash
+# Create a new Scope doctor group with helper scripts
+
+set -euo pipefail
+
+usage() {
+  cat << EOF
+Usage: $0 <type> <group-name>
+
+Create a new Scope doctor group with helper scripts.
+
+Arguments:
+  type        Group type: 'application' or 'environment' or 'project'
+  group-name  Descriptive name (e.g., ruby-version, colima)
+
+Examples:
+  $0 application ruby-version
+  $0 environment colima
+  $0 project database-setup
+
+The script will create:
+  For application/environment:
+    - {config-root}/{type}/{group-name}.yaml
+    - {config-root}/{type}/bin/{group-name}.sh
+
+  For project:
+    - .scope/{group-name}.yaml
+    - .scope/bin/{group-name}.sh
+EOF
+  exit 1
+}
+
+# Check arguments
+if [[ $# -ne 2 ]]; then
+  usage
+fi
+
+TYPE="$1"
+GROUP_NAME="$2"
+
+# Validate type and determine base directory
+case "$TYPE" in
+  application | environment)
+    # Try common config directory structures
+    if [[ -d "config/${TYPE}" ]]; then
+      BASE_DIR="config/${TYPE}"
+    elif [[ -d "scope-config/${TYPE}" ]]; then
+      BASE_DIR="scope-config/${TYPE}"
+    elif [[ -d "${TYPE}" ]]; then
+      BASE_DIR="${TYPE}"
+    else
+      echo "Error: Cannot find ${TYPE} directory" >&2
+      echo "Run this from config root directory" >&2
+      exit 1
+    fi
+    BIN_DIR="${BASE_DIR}/bin"
+    ;;
+  project)
+    BASE_DIR=".scope"
+    BIN_DIR="${BASE_DIR}/bin"
+    ;;
+  *)
+    echo "Error: type must be 'application', 'environment', or 'project'" >&2
+    usage
+    ;;
+esac
+
+# Create directories
+mkdir -p "$BASE_DIR" "$BIN_DIR"
+
+# Create YAML file
+YAML_FILE="${BASE_DIR}/${GROUP_NAME}.yaml"
+if [[ -f "$YAML_FILE" ]]; then
+  echo "Error: $YAML_FILE already exists" >&2
+  exit 1
+fi
+
+cat > "$YAML_FILE" << EOF
+apiVersion: scope.github.com/v1alpha
+kind: ScopeDoctorGroup
+metadata:
+  name: ${GROUP_NAME}
+  description: TODO: Brief description of what this group does
+spec:
+  include: when-required  # or 'by-default'
+  needs:
+    # TODO: Add dependencies here
+    # - dependency-1
+    # - dependency-2
+  actions:
+    - name: check-and-fix
+      description: TODO: What this action checks/fixes
+      required: true
+      check:
+        # Option 1: Command-based check
+        commands:
+          - ./bin/${GROUP_NAME}.sh check
+        # Option 2: Path-based check (watches files for changes)
+        # paths:
+        #   - "file-to-watch.txt"
+        #   - "**/*.config"
+      fix:
+        commands:
+          - ./bin/${GROUP_NAME}.sh fix
+        helpText: |
+          TODO: What to do if the fix fails.
+
+          Common issues:
+          1. Issue 1 and resolution
+          2. Issue 2 and resolution
+
+          If you need assistance, contact #help-channel in Slack.
+        # Optional: helpUrl: https://docs.example.com/troubleshooting
+EOF
+
+# Create helper script
+SCRIPT_FILE="${BIN_DIR}/${GROUP_NAME}.sh"
+if [[ -f "$SCRIPT_FILE" ]]; then
+  echo "Error: $SCRIPT_FILE already exists" >&2
+  exit 1
+fi
+
+cat > "$SCRIPT_FILE" << 'SCRIPTEOF'
+#!/usr/bin/env bash
+# Helper script for GROUP_NAME doctor group
+
+set -euo pipefail
+
+ACTION="${1:-}"
+
+check() {
+  # TODO: Implement check logic
+  # Return 0 if check passes (nothing to fix)
+  # Return non-zero if check fails (fix needed)
+
+  echo "Checking..." >&2
+
+  # Example: Check if a file exists
+  # if [[ -f .required-file ]]; then
+  #   echo "Check passed" >&2
+  #   return 0
+  # else
+  #   echo "Check failed: .required-file missing" >&2
+  #   return 1
+  # fi
+
+  echo "TODO: Implement check logic" >&2
+  return 1
+}
+
+fix() {
+  # TODO: Implement fix logic
+  # Return 0 on success
+  # Return non-zero on failure
+
+  echo "Fixing..." >&2
+
+  # Example: Create required file
+  # touch .required-file
+
+  echo "TODO: Implement fix logic" >&2
+  return 1
+}
+
+case "$ACTION" in
+  check)
+    check
+    ;;
+  fix)
+    fix
+    ;;
+  *)
+    echo "Usage: $0 [check|fix]" >&2
+    exit 1
+    ;;
+esac
+SCRIPTEOF
+
+# Replace GROUP_NAME placeholder
+sed -i.bak "s/GROUP_NAME/${GROUP_NAME}/g" "$SCRIPT_FILE" && rm "${SCRIPT_FILE}.bak"
+
+# Make script executable
+chmod +x "$SCRIPT_FILE"
+
+echo "Created files:"
+echo "  - $YAML_FILE"
+echo "  - $SCRIPT_FILE"
+echo ""
+echo "Next steps:"
+echo "1. Edit $SCRIPT_FILE:"
+echo "   - Implement check() logic"
+echo "   - Implement fix() logic"
+echo "2. Test script standalone:"
+echo "   - $SCRIPT_FILE check"
+echo "   - $SCRIPT_FILE fix"
+echo "3. Edit $YAML_FILE:"
+echo "   - Update description"
+echo "   - Add dependencies in 'needs'"
+echo "   - Adjust include mode (when-required vs by-default)"
+echo "   - Update helpText"
+echo "4. Test group:"
+echo "   - scope doctor list | grep ${GROUP_NAME}"
+echo "   - scope doctor run --only ${GROUP_NAME} --no-cache"
+echo "5. Validate schema:"
+echo "   - jsonschema validate schema.json $YAML_FILE"

skills/scope/scripts/create-known-error.sh

@@ -0,0 +1,117 @@
+#!/usr/bin/env bash
+# Create a new Scope known error with proper structure
+
+set -euo pipefail
+
+usage() {
+  cat << EOF
+Usage: $0 <category> <error-name>
+
+Create a new Scope known error definition with test file.
+
+Arguments:
+  category    Error category (e.g., docker, ruby, git)
+  error-name  Descriptive error name (e.g., colima-not-running)
+
+Examples:
+  $0 docker colima-not-running
+  $0 ruby gem-missing-file
+  $0 git cannot-lock-ref
+
+The script will create:
+  - {config-root}/known-errors/{category}/{error-name}.yaml
+  - {config-root}/known-errors/{category}/{error-name}.txt
+
+You will need to edit both files with actual content.
+EOF
+  exit 1
+}
+
+# Check arguments
+if [[ $# -ne 2 ]]; then
+  usage
+fi
+
+CATEGORY="$1"
+ERROR_NAME="$2"
+
+# Determine base directory
+if [[ -d "known-errors" ]]; then
+  BASE_DIR="known-errors"
+elif [[ -d "config/known-errors" ]]; then
+  BASE_DIR="config/known-errors"
+elif [[ -d "scope-config/known-errors" ]]; then
+  BASE_DIR="scope-config/known-errors"
+else
+  echo "Error: Cannot find known-errors directory" >&2
+  echo "Run this from config root or known-errors parent directory" >&2
+  exit 1
+fi
+
+# Create directory
+DIR="${BASE_DIR}/${CATEGORY}"
+mkdir -p "$DIR"
+
+# Create YAML file
+YAML_FILE="${DIR}/${ERROR_NAME}.yaml"
+if [[ -f "$YAML_FILE" ]]; then
+  echo "Error: $YAML_FILE already exists" >&2
+  exit 1
+fi
+
+cat > "$YAML_FILE" << EOF
+apiVersion: scope.github.com/v1alpha
+kind: ScopeKnownError
+metadata:
+  name: ${ERROR_NAME}
+  description: TODO: Brief description of what this error means
+spec:
+  pattern: "TODO: regex pattern to match the error"
+  help: |
+    TODO: Clear explanation of the issue.
+
+    Steps to resolve:
+    1. First step
+    2. Second step
+    3. Where to get help if needed
+
+    If you need assistance, contact #help-channel in Slack.
+  # Uncomment to add automated fix:
+  # fix:
+  #   prompt:
+  #     text: TODO: User-friendly prompt asking permission
+  #   commands:
+  #     - TODO: command-to-fix
+EOF
+
+# Create test file
+TXT_FILE="${DIR}/${ERROR_NAME}.txt"
+cat > "$TXT_FILE" << EOF
+TODO: Paste actual error output here.
+
+This should be the real error text that users see, including:
+- The error message itself
+- Surrounding context (lines before/after)
+- Stack traces if applicable
+- Command that failed
+
+Example:
+  $ some-command that failed
+  Error: something went wrong
+  Details about the error here
+EOF
+
+echo "Created files:"
+echo "  - $YAML_FILE"
+echo "  - $TXT_FILE"
+echo ""
+echo "Next steps:"
+echo "1. Edit $TXT_FILE with actual error output"
+echo "2. Test pattern: rg 'your-pattern' $TXT_FILE"
+echo "3. Edit $YAML_FILE with:"
+echo "   - Proper description"
+echo "   - Correct pattern"
+echo "   - Clear help text"
+echo "   - Optional fix commands"
+echo "4. Test: scope analyze logs --extra-config ${BASE_DIR%/known-errors*} $TXT_FILE"
+echo "5. Validate: jsonschema validate schema.json $YAML_FILE"