Initial commit

This commit is contained in:
Zhongwei Li
2025-11-29 18:48:10 +08:00
commit b317d592e2
8 changed files with 837 additions and 0 deletions

View File

@@ -0,0 +1,21 @@
---
description: Deploy tested Dataform table to production
---
You are deploying a Dataform table to production using best practices from the dataform-engineering-fundamentals skill.
**Workflow:**
1. Invoke the dataform-engineering-fundamentals skill
2. Ask the user which table they want to deploy
3. **Pre-deployment verification:**
- Confirm the table has been tested in dev environment
- Verify all tests are passing
- Check that documentation (columns: {}) is complete
4. **Deployment:**
- Run `dataform run --dry-run --actions <table_name>` (production dry-run)
- If successful, run `dataform run --actions <table_name>` (production execution)
- Verify deployment with validation queries
5. Report deployment results
**Critical**: Never deploy without dev testing first. Wrong results delivered quickly are worse than correct results delivered with a small delay.

24
commands/dataform-etl.md Normal file
View File

@@ -0,0 +1,24 @@
---
description: Launch ETL agent for BigQuery Dataform development
---
You are launching the ETL Dataform engineer agent to handle data transformation pipeline work.
**Purpose**: The ETL agent specializes in BigQuery Dataform projects, SQLX files, data quality, and pipeline development. Use this for:
- Complex Dataform transformations
- Pipeline troubleshooting
- Data quality implementations
- ELT workflow coordination
**Task**: Use the Task tool with `subagent_type="etl"` to launch the ETL agent. Pass the user's request as the prompt, including:
- What they need to accomplish
- Any relevant context about tables, datasets, or business logic
- Whether this is new development, modification, or troubleshooting
The ETL agent has access to the dataform-engineering-fundamentals skill and will follow best practices for BigQuery Dataform development.
**Example**:
```
User asks: "Help me create a customer metrics table"
You launch: Task tool with subagent_type="etl" and prompt="Create a customer metrics table in Dataform following TDD workflow. Ask user about required metrics and data sources."
```

View File

@@ -0,0 +1,33 @@
---
description: Create new Dataform table using TDD workflow
---
You are creating a new Dataform table following the Test-Driven Development (TDD) workflow from the dataform-engineering-fundamentals skill.
**Workflow:**
1. Invoke the dataform-engineering-fundamentals skill
2. Ask the user about the table requirements:
- Table name and purpose
- Expected columns and their descriptions
- Data sources (for creating source declarations if needed)
- Business logic and transformations
3. **RED Phase - Write tests first:**
- Create assertion file in `definitions/assertions/`
- Write data quality tests (duplicates, nulls, invalid values)
- Run tests - they should FAIL (table doesn't exist yet)
4. **GREEN Phase - Write minimal implementation:**
- Create source declarations if needed
- Create table SQLX file with:
- Proper config block with type, schema
- Complete columns: {} documentation
- SQL transformation
- Run table creation: `dataform run --schema-suffix dev --actions <table_name>`
- Run tests - they should PASS
5. **REFACTOR Phase - Improve while keeping tests passing:**
- Optimize query performance if needed
- Add partitioning/clustering if appropriate
- Improve documentation clarity
6. Report completion with file locations and next steps
**Critical**: Always write tests FIRST, then implementation. Tests-after means you're checking what it does, not defining what it should do.

18
commands/dataform-test.md Normal file
View File

@@ -0,0 +1,18 @@
---
description: Test Dataform table in dev environment with safety checks
---
You are testing a Dataform table using best practices from the dataform-engineering-fundamentals skill.
**Workflow:**
1. Invoke the dataform-engineering-fundamentals skill
2. Ask the user which table they want to test
3. Follow the safety checklist:
- Run `dataform compile` to check syntax
- Run `dataform run --schema-suffix dev --dry-run --actions <table_name>` to validate SQL
- Run `dataform run --schema-suffix dev --actions <table_name>` to execute in dev
- Run basic validation queries to verify results
4. Report results and any issues found
**Critical**: Always use `--schema-suffix dev` for testing. Never test directly in production.