# Turborepo Task Pipelines Task orchestration, dependencies, and parallel execution strategies. ## Pipeline Configuration Define tasks in `turbo.json`: ```json { "$schema": "https://turbo.build/schema.json", "pipeline": { "build": { "dependsOn": ["^build"], "outputs": ["dist/**", ".next/**"] }, "test": { "dependsOn": ["build"], "outputs": ["coverage/**"] }, "lint": {}, "dev": { "cache": false, "persistent": true } } } ``` ## Task Dependencies ### Topological Dependencies (^) `^` means "run this task in dependencies first": ```json { "pipeline": { "build": { "dependsOn": ["^build"] } } } ``` Example flow: ``` packages/ui (dependency) ↓ builds first apps/web (depends on @repo/ui) ↓ builds second ``` ### Internal Dependencies Run tasks in same package first: ```json { "pipeline": { "deploy": { "dependsOn": ["build", "test"] } } } ``` Execution order in same package: 1. Run `build` 2. Run `test` 3. Run `deploy` ### Combined Dependencies Mix topological and internal: ```json { "pipeline": { "test": { "dependsOn": ["^build", "lint"] } } } ``` Execution order: 1. Build all dependencies (`^build`) 2. Lint current package (`lint`) 3. Run tests (`test`) ## Task Configuration Options ### outputs Define what gets cached: ```json { "build": { "outputs": [ "dist/**", // All files in dist ".next/**", // Next.js build "!.next/cache/**", // Exclude Next.js cache "build/**", // Build directory "public/dist/**" // Public assets ] } } ``` ### cache Enable/disable caching: ```json { "dev": { "cache": false // Don't cache dev server }, "build": { "cache": true // Cache build (default) } } ``` ### persistent Keep task running (for dev servers): ```json { "dev": { "cache": false, "persistent": true // Don't kill after completion } } ``` ### env Environment variables affecting output: ```json { "build": { "env": [ "NODE_ENV", "NEXT_PUBLIC_API_URL", "DATABASE_URL" ] } } ``` ### passThroughEnv Pass env vars without affecting cache: ```json { "build": { "passThroughEnv": [ "DEBUG", // Pass through but don't invalidate cache "LOG_LEVEL" ] } } ``` ### inputs Override default input detection: ```json { "build": { "inputs": [ "src/**/*.ts", "!src/**/*.test.ts", // Exclude test files "package.json" ] } } ``` ### outputMode Control output display: ```json { "build": { "outputMode": "full" // Show all output }, "dev": { "outputMode": "hash-only" // Show cache hash only }, "test": { "outputMode": "new-only" // Show new output only }, "lint": { "outputMode": "errors-only" // Show errors only } } ``` ## Running Tasks ### Basic Execution ```bash # Run build in all packages turbo run build # Run multiple tasks turbo run build test lint # Run with specific package manager pnpm turbo run build ``` ### Filtering Run tasks in specific packages: ```bash # Single package turbo run build --filter=web turbo run build --filter=@repo/ui # Multiple packages turbo run build --filter=web --filter=api # All apps turbo run build --filter='./apps/*' # Pattern matching turbo run test --filter='*-api' ``` ### Dependency Filtering ```bash # Package and its dependencies turbo run build --filter='...web' # Package's dependencies only (exclude package itself) turbo run build --filter='...^web' # Package and its dependents turbo run test --filter='ui...' # Package's dependents only turbo run test --filter='^ui...' ``` ### Git-Based Filtering Run only on changed packages: ```bash # Changed since main branch turbo run build --filter='[main]' # Changed since HEAD~1 turbo run build --filter='[HEAD~1]' # Changed in working directory turbo run test --filter='...[HEAD]' # Package and dependencies, only if changed turbo run build --filter='...[origin/main]' ``` ## Concurrency Control ### Parallel Execution (Default) Turborepo runs tasks in parallel when safe: ```bash # Run with default parallelism turbo run build ``` ### Limit Concurrency ```bash # Max 3 tasks at once turbo run build --concurrency=3 # 50% of CPU cores turbo run build --concurrency=50% # No parallelism (sequential) turbo run build --concurrency=1 ``` ### Continue on Error ```bash # Don't stop on first error turbo run test --continue ``` ## Task Execution Order Example monorepo: ``` apps/ ├── web (depends on @repo/ui, @repo/utils) └── docs (depends on @repo/ui) packages/ ├── ui (depends on @repo/utils) └── utils (no dependencies) ``` With config: ```json { "pipeline": { "build": { "dependsOn": ["^build"] } } } ``` Execution order for `turbo run build`: 1. **Wave 1** (parallel): `@repo/utils` (no dependencies) 2. **Wave 2** (parallel): `@repo/ui` (depends on utils) 3. **Wave 3** (parallel): `web` and `docs` (both depend on ui) ## Complex Pipeline Examples ### Full-Stack Application ```json { "pipeline": { "build": { "dependsOn": ["^build"], "outputs": [".next/**", "dist/**"] }, "test": { "dependsOn": ["^build"], "outputs": ["coverage/**"] }, "lint": { "dependsOn": ["^build"] }, "typecheck": { "dependsOn": ["^build"] }, "dev": { "cache": false, "persistent": true }, "deploy": { "dependsOn": ["build", "test", "lint", "typecheck"] } } } ``` ### Monorepo with Code Generation ```json { "pipeline": { "generate": { "cache": false, "outputs": ["src/generated/**"] }, "build": { "dependsOn": ["^build", "generate"], "outputs": ["dist/**"] }, "test": { "dependsOn": ["generate"], "outputs": ["coverage/**"] } } } ``` ### Database-Dependent Pipeline ```json { "pipeline": { "db:generate": { "cache": false }, "db:migrate": { "cache": false }, "build": { "dependsOn": ["^build", "db:generate"], "outputs": ["dist/**"] }, "test:unit": { "dependsOn": ["build"] }, "test:integration": { "dependsOn": ["db:migrate"], "cache": false } } } ``` ## Dry Run Preview execution without running: ```bash # See what would run turbo run build --dry-run # JSON output for scripts turbo run build --dry-run=json # Show full task graph turbo run build --graph ``` ## Force Execution Ignore cache and run tasks: ```bash # Force rebuild everything turbo run build --force # Force specific package turbo run build --filter=web --force ``` ## Output Control ```bash # Show only errors turbo run build --output-logs=errors-only # Show new logs only turbo run build --output-logs=new-only # Show cache hash only turbo run build --output-logs=hash-only # Show full output turbo run build --output-logs=full ``` ## Best Practices 1. **Use topological dependencies** - `^build` ensures correct build order 2. **Cache build outputs** - Define `outputs` for faster rebuilds 3. **Disable cache for dev** - Set `cache: false` for dev servers 4. **Mark persistent tasks** - Use `persistent: true` for long-running tasks 5. **Filter strategically** - Use filters to run only affected tasks 6. **Control concurrency** - Limit parallelism for resource-intensive tasks 7. **Configure env vars** - Include vars that affect output in `env` 8. **Use dry-run** - Preview execution plan before running 9. **Continue on error in CI** - Use `--continue` to see all errors 10. **Leverage git filtering** - Run only on changed packages in CI ## Common Patterns ### CI/CD Pipeline ```yaml # .github/workflows/ci.yml jobs: build: steps: - run: turbo run build test lint --filter='...[origin/main]' ``` Only build/test/lint changed packages and their dependents. ### Development Workflow ```bash # Start all dev servers turbo run dev # Start specific app with dependencies turbo run dev --filter=web... ``` ### Pre-commit Hook ```json // package.json { "scripts": { "pre-commit": "turbo run lint test --filter='...[HEAD]'" } } ``` Only lint/test changed packages. ### Deployment ```bash # Build and test specific app turbo run build test --filter=web... # Deploy if successful turbo run deploy --filter=web ``` Build app and its dependencies, then deploy.