Initial commit

skills/yq/reference/yq-guide.md

@@ -0,0 +1,432 @@
+# yq: YAML Query and Extraction
+
+**Goal: Extract specific data from YAML without reading entire file.**
+
+**Note:** This guide assumes `mikefarah/yq` (the most common version). Syntax is similar to jq.
+
+## The Essential Pattern
+
+```bash
+yq '.field' file.yml
+```
+
+yq defaults to YAML output. Use `-r` for raw output or `-o json` for JSON:
+```bash
+yq -r '.field' file.yml          # Raw string output
+yq -o json file.yml              # Convert to JSON
+```
+
+---
+
+# Core Patterns (80% of Use Cases)
+
+## 1. Extract Top-Level Field
+```bash
+yq '.version' config.yml
+yq '.name' config.yml
+```
+
+## 2. Extract Nested Field
+```bash
+yq '.services.web.image' docker-compose.yml
+yq '.jobs.build.steps' .github/workflows/ci.yml
+```
+
+## 3. Extract from Array by Index
+```bash
+yq '.items[0]' file.yml              # First element
+yq '.jobs.build.steps[2]' ci.yml     # Third step
+```
+
+## 4. Extract All Array Elements
+```bash
+yq '.items[]' file.yml               # All elements
+yq '.services.*.ports' docker-compose.yml    # All ports from all services
+```
+
+## 5. Extract Field from Each Array Element
+```bash
+yq '.services.*.image' docker-compose.yml          # All service images
+yq '.jobs.*.runs-on' .github/workflows/ci.yml      # All job runners
+```
+
+## 6. Get Object Keys
+```bash
+yq 'keys' object.yml
+yq '.services | keys' docker-compose.yml
+```
+
+## 7. Filter Array by Condition
+```bash
+yq '.items[] | select(.active == true)' file.yml
+yq '.services.* | select(.ports)' docker-compose.yml
+```
+
+## 8. Extract Specific Array Elements
+```bash
+yq '.services.web' docker-compose.yml
+yq '.jobs.build' .github/workflows/ci.yml
+```
+
+## 9. Handle Missing Fields
+```bash
+yq '.field // "default"' file.yml
+```
+
+## 10. Convert YAML to JSON
+```bash
+yq -o json file.yml
+```
+
+---
+
+# Common Real-World Workflows
+
+## Docker Compose
+
+### "List all services"
+```bash
+yq '.services | keys' docker-compose.yml
+```
+
+### "Get image for service"
+```bash
+yq '.services.web.image' docker-compose.yml
+```
+
+### "Get all ports"
+```bash
+yq '.services.*.ports' docker-compose.yml
+```
+
+### "Get environment variables for service"
+```bash
+yq '.services.api.environment' docker-compose.yml
+```
+
+### "Get depends_on for service"
+```bash
+yq '.services.web.depends_on' docker-compose.yml
+```
+
+## GitHub Actions
+
+### "List all jobs"
+```bash
+yq '.jobs | keys' .github/workflows/ci.yml
+```
+
+### "Get build steps"
+```bash
+yq '.jobs.build.steps' .github/workflows/ci.yml
+```
+
+### "Get trigger events"
+```bash
+yq '.on' .github/workflows/ci.yml
+```
+
+### "Get runner for job"
+```bash
+yq '.jobs.build.runs-on' .github/workflows/ci.yml
+```
+
+## Kubernetes
+
+### "Get container image"
+```bash
+yq '.spec.template.spec.containers[0].image' deployment.yml
+```
+
+### "Get replicas"
+```bash
+yq '.spec.replicas' deployment.yml
+```
+
+### "List all container names"
+```bash
+yq '.spec.template.spec.containers[].name' deployment.yml
+```
+
+## Configuration Files
+
+### "Get database host"
+```bash
+yq '.database.host' config.yml
+```
+
+### "Get API key"
+```bash
+yq '.api.key' config.yml
+```
+
+---
+
+# Advanced Patterns (20% Use Cases)
+
+## Combine Multiple Queries
+```bash
+yq '{version: .version, services: (.services | keys)}' docker-compose.yml
+```
+
+## Count Array Length
+```bash
+yq '.items | length' file.yml
+yq '.services | length' docker-compose.yml
+```
+
+## Filter and Extract
+```bash
+yq '.services.* | select(.ports) | .image' docker-compose.yml
+```
+
+## Map Array
+```bash
+yq '[.items[].name]' file.yml
+```
+
+## Multi-Document YAML (Multiple --- separated docs)
+```bash
+yq 'select(document_index == 0)' multi.yml     # First document
+yq 'select(document_index == 1)' multi.yml     # Second document
+```
+
+---
+
+# Output Formats
+
+```bash
+yq file.yml                # YAML output (default)
+yq -o json file.yml        # JSON output
+yq -o yaml file.yml        # Explicit YAML output
+yq -r '.field' file.yml    # Raw output (strings without quotes)
+```
+
+**For string fields, use `-r` for raw output (like jq).**
+
+---
+
+# Pipe Composition
+
+yq uses `|` for piping within queries (like jq):
+```bash
+yq '.services | keys | .[]' docker-compose.yml
+```
+
+Can also pipe to shell commands:
+```bash
+yq '.services | keys' docker-compose.yml | wc -l        # Count services
+yq '.services.*.image' docker-compose.yml | sort | uniq  # Unique images
+```
+
+---
+
+# Common Flags
+
+- `-r` - Raw output (strings without quotes)
+- `-o FORMAT` - Output format (yaml, json, props, xml, etc.)
+- `-i` - In-place edit (DANGEROUS - use carefully)
+- `-P` - Pretty print
+- `-I INDENT` - Indentation level
+
+**Default to `-r` for string extraction, `-o json` for JSON output.**
+
+---
+
+# YAML-Specific Features
+
+## Anchors and Aliases
+YAML supports anchors (&) and aliases (*):
+```yaml
+default: &default
+  timeout: 30
+
+production:
+  <<: *default
+  host: prod.example.com
+```
+
+yq resolves these automatically:
+```bash
+yq '.production.timeout' file.yml    # Returns 30 (from anchor)
+```
+
+## Multi-Document YAML
+Many YAML files contain multiple documents separated by `---`:
+```bash
+yq 'select(document_index == 0)' file.yml    # First document
+yq 'select(document_index == 1)' file.yml    # Second document
+```
+
+---
+
+# Handling Edge Cases
+
+## If Field Might Not Exist
+```bash
+yq '.field // "not found"' file.yml
+```
+
+## If Array Might Be Empty
+```bash
+yq '.items[]? // empty' file.yml
+```
+
+## Multiple Possible Paths
+```bash
+yq '.field1 // .field2 // "default"' file.yml
+```
+
+---
+
+# Comparison with jq
+
+**Similarities:**
+- Similar query syntax (`.field`, `.nested.field`, `.array[]`)
+- Pipe operator `|`
+- Filter with `select()`
+- Default values with `//`
+
+**Differences:**
+- yq handles YAML (jq handles JSON)
+- yq can output multiple formats (`-o json`, `-o yaml`)
+- yq handles YAML features (anchors, multi-doc)
+- Slightly different handling of wildcards (`.*` in yq, similar in jq)
+
+**Converting between formats:**
+```bash
+yq -o json file.yml | jq '.field'    # YAML → JSON → query with jq
+jq '.' file.json | yq -P             # JSON → pretty YAML
+```
+
+---
+
+# Integration with Other Tools
+
+## With ast-grep
+```bash
+# Get dependencies from YAML, search code for usage
+yq '.dependencies | keys' config.yml | while read dep; do
+  rg -l "$dep"
+done
+```
+
+## With jq (via JSON conversion)
+```bash
+yq -o json file.yml | jq '.complex.query'
+```
+If yq syntax doesn't work, convert to JSON and use jq.
+
+## With Docker Compose
+```bash
+# Get all service images, then pull them
+yq '.services.*.image' docker-compose.yml | xargs -n1 docker pull
+```
+
+---
+
+# Best Practices
+
+## 1. Use -r for String Fields
+```bash
+# BAD:  yq '.version' file.yml  → may include YAML formatting
+# GOOD: yq -r '.version' file.yml  → raw string
+```
+
+## 2. Use -o json for Complex Queries
+If yq query is complex, convert to JSON and use jq:
+```bash
+yq -o json file.yml | jq '.complex.query'
+```
+
+## 3. Test Queries on Small Examples
+```bash
+echo 'key: value' | yq '.key'
+```
+
+## 4. Use // for Defaults
+```bash
+yq -r '.field // "default"' file.yml
+```
+
+## 5. Be Aware of Multi-Document Files
+Check if file has multiple documents (look for `---`).
+
+---
+
+# Quick Reference
+
+## Most Common Commands
+
+```bash
+# Single field
+yq '.field' file.yml
+
+# Nested field
+yq '.parent.child' file.yml
+
+# Array element
+yq '.array[0]' file.yml
+
+# All array elements
+yq '.array[]' file.yml
+
+# Object keys
+yq 'keys' file.yml
+
+# Filter array
+yq '.array[] | select(.field == "value")' file.yml
+
+# All services (docker-compose)
+yq '.services | keys' docker-compose.yml
+
+# Convert to JSON
+yq -o json file.yml
+
+# Raw string output
+yq -r '.field' file.yml
+```
+
+---
+
+# When to Use Read Instead
+
+Use Read tool when:
+- File is < 50 lines
+- Need to see overall structure
+- Making edits (need full context)
+- Exploring unknown YAML structure
+
+Use yq when:
+- File is large (docker-compose, k8s configs often are)
+- You know exactly what field(s) you need
+- Want to save context tokens
+
+---
+
+# Summary
+
+**Your default pattern:**
+```bash
+yq '.field' file.yml
+```
+
+**For strings:**
+```bash
+yq -r '.field' file.yml
+```
+
+**For JSON output:**
+```bash
+yq -o json file.yml
+```
+
+**Key principles:**
+1. Use `-r` for raw string output
+2. Use `.` notation for nested fields
+3. Use `[]` for arrays, `[n]` for specific index
+4. Use `| keys` for object keys
+5. Use `//` for defaults
+6. Use `-o json` to convert to JSON if needed
+
+**Massive context savings: Extract only what you need instead of reading entire YAML files.**