310 lines
8.0 KiB
Markdown
310 lines
8.0 KiB
Markdown
---
|
||
name: Workflow Compose
|
||
description: Executes multi-step workflows by chaining Betty Framework skills.
|
||
---
|
||
|
||
# workflow.compose
|
||
|
||
## Purpose
|
||
|
||
Allows declarative execution of Betty Framework workflows by reading a YAML definition and chaining skills like `skill.create`, `skill.define`, and `registry.update`.
|
||
|
||
Enables complex multi-step processes to be defined once and executed reliably with proper error handling and audit logging.
|
||
|
||
## Usage
|
||
|
||
### Basic Usage
|
||
|
||
```bash
|
||
python skills/workflow.compose/workflow_compose.py <path_to_workflow.yaml>
|
||
```
|
||
|
||
### Arguments
|
||
|
||
| Argument | Type | Required | Description |
|
||
|----------|------|----------|-------------|
|
||
| workflow_path | string | Yes | Path to the workflow YAML file to execute |
|
||
|
||
## Workflow YAML Structure
|
||
|
||
```yaml
|
||
# workflows/create_and_register.yaml
|
||
name: "Create and Register Skill"
|
||
description: "Complete lifecycle: create, validate, and register a new skill"
|
||
|
||
steps:
|
||
- skill: skill.create
|
||
args: ["workflow.validate", "Validates workflow definitions"]
|
||
required: true
|
||
|
||
- skill: skill.define
|
||
args: ["skills/workflow.validate/skill.yaml"]
|
||
required: true
|
||
|
||
- skill: registry.update
|
||
args: ["skills/workflow.validate/skill.yaml"]
|
||
required: false # Continue even if this fails
|
||
```
|
||
|
||
### Workflow Fields
|
||
|
||
| Field | Required | Description | Example |
|
||
|-------|----------|-------------|---------|
|
||
| `name` | No | Workflow name | `"API Design Workflow"` |
|
||
| `description` | No | What the workflow does | `"Complete API lifecycle"` |
|
||
| `steps` | Yes | Array of steps to execute | See below |
|
||
|
||
### Step Fields
|
||
|
||
| Field | Required | Description | Example |
|
||
|-------|----------|-------------|---------|
|
||
| `skill` | Yes | Skill name to execute | `api.validate` |
|
||
| `args` | No | Arguments to pass to skill | `["specs/api.yaml", "zalando"]` |
|
||
| `required` | No | Stop workflow if step fails | `true` (default: `false`) |
|
||
|
||
## Behavior
|
||
|
||
1. **Load Workflow**: Parses the workflow YAML file
|
||
2. **Sequential Execution**: Runs each step in order
|
||
3. **Error Handling**:
|
||
- If `required: true`, workflow stops on failure
|
||
- If `required: false`, workflow continues and logs error
|
||
4. **Audit Logging**: Calls `audit.log` skill (if available) for each step
|
||
5. **History Tracking**: Records execution history in `/registry/workflow_history.json`
|
||
|
||
## Outputs
|
||
|
||
### Success Response
|
||
|
||
```json
|
||
{
|
||
"ok": true,
|
||
"status": "success",
|
||
"errors": [],
|
||
"path": "workflows/create_and_register.yaml",
|
||
"details": {
|
||
"workflow_name": "Create and Register Skill",
|
||
"steps_executed": 3,
|
||
"steps_succeeded": 3,
|
||
"steps_failed": 0,
|
||
"duration_ms": 1234,
|
||
"history_file": "/registry/workflow_history.json"
|
||
}
|
||
}
|
||
```
|
||
|
||
### Partial Failure Response
|
||
|
||
```json
|
||
{
|
||
"ok": false,
|
||
"status": "failed",
|
||
"errors": [
|
||
"Step 2 (skill.define) failed: Missing required fields: version"
|
||
],
|
||
"path": "workflows/create_and_register.yaml",
|
||
"details": {
|
||
"workflow_name": "Create and Register Skill",
|
||
"steps_executed": 2,
|
||
"steps_succeeded": 1,
|
||
"steps_failed": 1,
|
||
"failed_step": "skill.define",
|
||
"failed_step_index": 1
|
||
}
|
||
}
|
||
```
|
||
|
||
## Example Workflow Files
|
||
|
||
### Example 1: Complete Skill Lifecycle
|
||
|
||
```yaml
|
||
# workflows/create_and_register.yaml
|
||
name: "Create and Register Skill"
|
||
description: "Scaffold, validate, and register a new skill"
|
||
|
||
steps:
|
||
- skill: skill.create
|
||
args: ["workflow.validate", "Validates workflow definitions"]
|
||
required: true
|
||
|
||
- skill: skill.define
|
||
args: ["skills/workflow.validate/skill.yaml"]
|
||
required: true
|
||
|
||
- skill: registry.update
|
||
args: ["skills/workflow.validate/skill.yaml"]
|
||
required: true
|
||
```
|
||
|
||
**Execution**:
|
||
|
||
```bash
|
||
$ python skills/workflow.compose/workflow_compose.py workflows/create_and_register.yaml
|
||
{
|
||
"ok": true,
|
||
"status": "success",
|
||
"details": {
|
||
"steps_executed": 3,
|
||
"steps_succeeded": 3
|
||
}
|
||
}
|
||
```
|
||
|
||
### Example 2: API Design Workflow
|
||
|
||
```yaml
|
||
# workflows/api_design.yaml
|
||
name: "API Design Workflow"
|
||
description: "Design, validate, and generate models for new API"
|
||
|
||
steps:
|
||
- skill: api.define
|
||
args: ["user-service", "openapi", "zalando", "specs", "1.0.0"]
|
||
required: true
|
||
|
||
- skill: api.validate
|
||
args: ["specs/user-service.openapi.yaml", "zalando", "true"]
|
||
required: true
|
||
|
||
- skill: api.generate-models
|
||
args: ["specs/user-service.openapi.yaml", "typescript", "src/models"]
|
||
required: false # Continue even if model generation fails
|
||
```
|
||
|
||
### Example 3: Multi-Spec Validation
|
||
|
||
```yaml
|
||
# workflows/validate_all_specs.yaml
|
||
name: "Validate All API Specs"
|
||
description: "Validate all OpenAPI specifications in specs directory"
|
||
|
||
steps:
|
||
- skill: api.validate
|
||
args: ["specs/users.openapi.yaml", "zalando"]
|
||
required: false
|
||
|
||
- skill: api.validate
|
||
args: ["specs/orders.openapi.yaml", "zalando"]
|
||
required: false
|
||
|
||
- skill: api.validate
|
||
args: ["specs/payments.openapi.yaml", "zalando"]
|
||
required: false
|
||
```
|
||
|
||
## Workflow History
|
||
|
||
Execution history is logged to `/registry/workflow_history.json`:
|
||
|
||
```json
|
||
{
|
||
"executions": [
|
||
{
|
||
"workflow_path": "workflows/create_and_register.yaml",
|
||
"workflow_name": "Create and Register Skill",
|
||
"timestamp": "2025-10-23T12:34:56Z",
|
||
"status": "success",
|
||
"steps_executed": 3,
|
||
"steps_succeeded": 3,
|
||
"duration_ms": 1234
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
## Audit Integration
|
||
|
||
If `audit.log` skill is available, each step execution is logged:
|
||
|
||
```python
|
||
log_audit_entry(
|
||
skill_name="api.validate",
|
||
status="success",
|
||
duration_ms=456,
|
||
metadata={"workflow": "api_design.yaml", "step": 1}
|
||
)
|
||
```
|
||
|
||
## Integration
|
||
|
||
### With workflow.validate
|
||
|
||
Validate workflow syntax before execution:
|
||
|
||
```bash
|
||
# Validate first
|
||
python skills/workflow.validate/workflow_validate.py workflows/my-workflow.yaml
|
||
|
||
# Then execute
|
||
python skills/workflow.compose/workflow_compose.py workflows/my-workflow.yaml
|
||
```
|
||
|
||
### With Hooks
|
||
|
||
Auto-validate workflows when saved:
|
||
|
||
```bash
|
||
python skills/hook.define/hook_define.py \
|
||
--event on_file_save \
|
||
--pattern "workflows/*.yaml" \
|
||
--command "python skills/workflow.validate/workflow_validate.py {file_path}" \
|
||
--blocking true
|
||
```
|
||
|
||
### In CI/CD
|
||
|
||
```yaml
|
||
# .github/workflows/test.yml
|
||
- name: Run workflow tests
|
||
run: |
|
||
python skills/workflow.compose/workflow_compose.py workflows/test_suite.yaml
|
||
```
|
||
|
||
## Common Errors
|
||
|
||
| Error | Cause | Solution |
|
||
|-------|-------|----------|
|
||
| "Workflow file not found" | Path incorrect | Check workflow file path |
|
||
| "Invalid YAML in workflow" | Malformed YAML | Fix YAML syntax errors |
|
||
| "Skill handler not found" | Referenced skill doesn't exist | Ensure skill is registered or path is correct |
|
||
| "Step X failed" | Skill execution failed | Check skill's error output, fix issues |
|
||
| "Skill execution timed out" | Skill took >5 minutes | Optimize skill or increase timeout in code |
|
||
|
||
## Best Practices
|
||
|
||
1. **Validate First**: Run `workflow.validate` before executing workflows
|
||
2. **Use Required Judiciously**: Only mark critical steps as `required: true`
|
||
3. **Small Workflows**: Keep workflows focused on single logical task
|
||
4. **Error Handling**: Plan for partial failures in non-required steps
|
||
5. **Test Workflows**: Test workflows in development before using in production
|
||
6. **Version Control**: Keep workflow files in git
|
||
|
||
## Files Modified
|
||
|
||
- **History**: `/registry/workflow_history.json` – Execution history
|
||
- **Logs**: Step execution logged to Betty's logging system
|
||
|
||
## Exit Codes
|
||
|
||
- **0**: Success (all required steps succeeded)
|
||
- **1**: Failure (at least one required step failed)
|
||
|
||
## Timeout
|
||
|
||
Each skill has a 5-minute (300 second) timeout by default. If a skill exceeds this, the workflow fails.
|
||
|
||
## See Also
|
||
|
||
- **workflow.validate** – Validate workflow syntax ([workflow.validate SKILL.md](../workflow.validate/SKILL.md))
|
||
- **Betty Architecture** – [Five-Layer Model](../../docs/betty-architecture.md) for understanding workflows
|
||
- **API-Driven Development** – [Example workflows](../../docs/api-driven-development.md)
|
||
|
||
## Status
|
||
|
||
**Active** – Production-ready, core orchestration skill
|
||
|
||
## Version History
|
||
|
||
- **0.1.0** (Oct 2025) – Initial implementation with sequential execution, error handling, and audit logging
|