zhongwei/gh-epieczko-betty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 18:26:08 +08:00
commit 8f22ddf339
295 changed files with 59710 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

304
skills/generate.docs/SKILL.md Normal file
View File

@@ -0,0 +1,304 @@
# generate.docs
## Overview
**generate.docs** automatically generates or updates SKILL.md documentation from skill.yaml manifest files, ensuring consistent and comprehensive documentation across all Betty skills.
## Purpose
Eliminate manual documentation drift by:
- **Before**: Developers manually write and update SKILL.md files, leading to inconsistency and outdated docs
- **After**: Documentation is automatically generated from the authoritative skill.yaml manifest
This skill helps maintain high-quality documentation by:
- Reading skill.yaml manifest files
- Extracting inputs, outputs, and metadata
- Creating standardized SKILL.md documentation
- Ensuring consistency across all Betty skills
- Supporting dry-run previews and safe overwrites
## Usage
### Basic Usage
```bash
python skills/generate.docs/generate_docs.py <manifest_path> [options]
```
### Parameters
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `manifest_path` | string | Yes | - | Path to skill.yaml manifest file to generate documentation from |
| `--overwrite` | boolean | No | `false` | Overwrite existing SKILL.md file if it exists |
| `--dry-run` | boolean | No | `false` | Preview the generated documentation without writing to disk |
| `--output-path` | string | No | - | Custom output path for SKILL.md (defaults to same directory as manifest) |
## Outputs
| Output | Type | Description |
|--------|------|-------------|
| `doc_path` | string | Path to generated or updated SKILL.md file |
| `doc_content` | string | Generated documentation content |
| `dry_run_preview` | string | Preview of documentation (when dry_run=true) |
## Examples
### Example 1: Generate Documentation for a New Skill
```bash
python skills/generate.docs/generate_docs.py skills/api.define/skill.yaml
```
**Output**: Creates `skills/api.define/SKILL.md` with comprehensive documentation
**Result**:
```json
{
"status": "success",
"data": {
"doc_path": "skills/api.define/SKILL.md",
"skill_name": "api.define",
"dry_run": false
}
}
```
### Example 2: Preview Documentation Without Writing
```bash
python skills/generate.docs/generate_docs.py \
skills/hook.define/skill.yaml \
--dry-run=true
```
**Result**: Prints formatted documentation preview to console without creating any files
```
================================================================================
DRY RUN PREVIEW
================================================================================
# hook.define
## Overview
...
================================================================================
```
### Example 3: Overwrite Existing Documentation
```bash
python skills/generate.docs/generate_docs.py \
skills/api.validate/skill.yaml \
--overwrite=true
```
**Result**: Replaces existing `skills/api.validate/SKILL.md` with newly generated version
### Example 4: Custom Output Path
```bash
python skills/generate.docs/generate_docs.py \
skills/workflow.compose/skill.yaml \
--output-path=docs/skills/workflow-compose.md
```
**Result**: Generates documentation at `docs/skills/workflow-compose.md` instead of default location
### Example 5: Batch Documentation Generation
```bash
# Generate docs for all skills in the repository
for manifest in skills/*/skill.yaml; do
echo "Generating docs for $manifest..."
python skills/generate.docs/generate_docs.py "$manifest" --overwrite=true
done
```
**Result**: Updates documentation for all skills, ensuring consistency across the entire framework
## Generated Documentation Structure
The generated SKILL.md includes the following sections:
1. **Overview** - Skill name and brief description from manifest
2. **Purpose** - Detailed explanation of what the skill does
3. **Usage** - Command-line usage examples with proper syntax
4. **Parameters** - Detailed table of all inputs with types, requirements, and defaults
5. **Outputs** - Description of all skill outputs
6. **Usage Template** - Practical examples showing common use cases
7. **Integration Notes** - How to use with workflows, other skills, and batch operations
8. **Dependencies** - Required dependencies from manifest
9. **Tags** - Skill tags for categorization
10. **Version** - Skill version from manifest
## Integration Notes
### Use in Workflows
```yaml
# workflows/maintain_docs.yaml
name: Documentation Maintenance
description: Keep all skill documentation up to date
steps:
- name: Update skill docs
skill: generate.docs
args:
- "${skill_manifest_path}"
- "--overwrite=true"
- name: Commit changes
command: git add ${doc_path} && git commit -m "docs: update skill documentation"
```
### Use with skill.create
When creating a new skill, automatically generate its documentation:
```bash
# Create new skill
python skills/skill.create/skill_create.py my.new.skill
# Generate documentation
python skills/generate.docs/generate_docs.py \
skills/my.new.skill/skill.yaml
```
### Integration with CI/CD
Add to your CI pipeline to ensure documentation stays in sync:
```yaml
# .github/workflows/docs.yml
- name: Check documentation is up to date
run: |
for manifest in skills/*/skill.yaml; do
python skills/generate.docs/generate_docs.py "$manifest" --dry-run=true > /tmp/preview.md
skill_dir=$(dirname "$manifest")
if ! diff -q "$skill_dir/SKILL.md" /tmp/preview.md; then
echo "Documentation out of sync for $manifest"
exit 1
fi
done
```
### Use with Hooks
Automatically regenerate docs when skill.yaml is modified:
```bash
python skills/hook.define/hook_define.py \
on_file_save \
"python skills/generate.docs/generate_docs.py {file_path} --overwrite=true" \
--pattern="*/skill.yaml" \
--blocking=false \
--description="Auto-regenerate SKILL.md when skill.yaml changes"
```
## Benefits
### For Developers
- No manual documentation writing
- Consistent format across all skills
- Preview changes before committing
- Safe overwrite protection
### For Teams
- Single source of truth (skill.yaml)
- Automated documentation updates
- Standardized skill documentation
- Easy onboarding with clear docs
### For Maintenance
- Detect documentation drift
- Batch regeneration for all skills
- CI/CD integration for validation
- Version-controlled documentation
## Error Handling
### Manifest Not Found
```bash
$ python skills/generate.docs/generate_docs.py nonexistent/skill.yaml
```
**Error**:
```json
{
"status": "error",
"error": "Manifest file not found: nonexistent/skill.yaml"
}
```
### File Already Exists (Without Overwrite)
```bash
$ python skills/generate.docs/generate_docs.py skills/api.define/skill.yaml
```
**Error**:
```json
{
"status": "error",
"error": "SKILL.md already exists at skills/api.define/SKILL.md. Use --overwrite=true to replace it or --dry-run=true to preview."
}
```
### Invalid YAML Manifest
```bash
$ python skills/generate.docs/generate_docs.py broken-skill.yaml
```
**Error**:
```json
{
"status": "error",
"error": "Failed to parse YAML manifest: ..."
}
```
## Customization
After generation, you can manually enhance the documentation:
1. **Add detailed examples** - The generated docs include basic examples; add more complex ones
2. **Include diagrams** - Add architecture or flow diagrams
3. **Expand integration notes** - Add specific team or project integration details
4. **Add troubleshooting** - Document common issues and solutions
**Note**: Manual changes will be overwritten if you regenerate with `--overwrite=true`. Consider adding custom sections to the generator or maintaining separate docs for detailed content.
## Best Practices
1. **Run in dry-run mode first** - Preview changes before writing
2. **Use version control** - Track documentation changes via git
3. **Regenerate after manifest changes** - Keep docs in sync with manifest
4. **Include in PR reviews** - Ensure manifest and docs are updated together
5. **Automate in CI** - Validate docs match manifests in automated checks
## Dependencies
- **PyYAML**: Required for YAML manifest parsing
```bash
pip install pyyaml
```
- **context.schema**: For validation rule definitions
## Files Created
- `SKILL.md` - Generated skill documentation (in same directory as skill.yaml by default)
## See Also
- [skill.create](../skill.create/SKILL.md) - Create new skills
- [skill.define](../skill.define/SKILL.md) - Define skill manifests
- [Betty Architecture](../../docs/betty-architecture.md) - Five-layer model
- [Skill Development Guide](../../docs/skill-development.md) - Creating new skills
## Version
**0.1.0** - Initial implementation with manifest-to-markdown generation

1
skills/generate.docs/__init__.py Normal file
View File

@@ -0,0 +1 @@
# Auto-generated package initializer for skills.

540
skills/generate.docs/generate_docs.py Executable file
View File

@@ -0,0 +1,540 @@
#!/usr/bin/env python3
"""
Generate or update SKILL.md documentation from skill.yaml manifest files.
This skill automatically creates comprehensive documentation for Betty skills
based on their manifest definitions, ensuring consistency and completeness.
"""
import sys
import json
import argparse
import os
from pathlib import Path
from typing import Dict, Any, List, Optional
# Add betty module to path
from betty.logging_utils import setup_logger
from betty.errors import format_error_response, BettyError
from betty.validation import validate_path
logger = setup_logger(__name__)
def load_skill_manifest(manifest_path: str) -> Dict[str, Any]:
"""
Load and parse a skill.yaml manifest file.
Args:
manifest_path: Path to skill.yaml file
Returns:
Parsed manifest data
Raises:
BettyError: If manifest file is invalid or not found
"""
manifest_file = Path(manifest_path)
if not manifest_file.exists():
raise BettyError(f"Manifest file not found: {manifest_path}")
if not manifest_file.is_file():
raise BettyError(f"Manifest path is not a file: {manifest_path}")
try:
import yaml
with open(manifest_file, 'r') as f:
manifest = yaml.safe_load(f)
if not isinstance(manifest, dict):
raise BettyError("Manifest must be a YAML object/dictionary")
logger.info(f"Loaded skill manifest from {manifest_path}")
return manifest
except yaml.YAMLError as e:
raise BettyError(f"Failed to parse YAML manifest: {e}")
except Exception as e:
raise BettyError(f"Failed to load manifest: {e}")
def normalize_input(inp: Any) -> Dict[str, Any]:
"""
Normalize input definition to standard format.
Args:
inp: Input definition (string or dict)
Returns:
Normalized input dictionary
"""
if isinstance(inp, str):
# Simple string format: "workflow_path"
return {
'name': inp,
'type': 'any',
'required': True,
'description': 'No description'
}
elif isinstance(inp, dict):
# Already in object format
return inp
else:
return {
'name': 'unknown',
'type': 'any',
'required': False,
'description': 'Invalid input format'
}
def format_inputs_section(inputs: List[Any]) -> str:
"""
Format the inputs section for the documentation.
Args:
inputs: List of input definitions from manifest (strings or dicts)
Returns:
Formatted markdown table
"""
if not inputs:
return "_No inputs defined_\n"
lines = [
"| Parameter | Type | Required | Default | Description |",
"|-----------|------|----------|---------|-------------|"
]
for inp in inputs:
normalized = normalize_input(inp)
name = normalized.get('name', 'unknown')
type_val = normalized.get('type', 'any')
required = 'Yes' if normalized.get('required', False) else 'No'
default = normalized.get('default', '-')
if default is True:
default = 'true'
elif default is False:
default = 'false'
elif default != '-':
default = f'`{default}`'
description = normalized.get('description', 'No description')
lines.append(f"| `{name}` | {type_val} | {required} | {default} | {description} |")
return '\n'.join(lines) + '\n'
def normalize_output(out: Any) -> Dict[str, Any]:
"""
Normalize output definition to standard format.
Args:
out: Output definition (string or dict)
Returns:
Normalized output dictionary
"""
if isinstance(out, str):
# Simple string format: "validation_result.json"
return {
'name': out,
'type': 'any',
'description': 'No description'
}
elif isinstance(out, dict):
# Already in object format
return out
else:
return {
'name': 'unknown',
'type': 'any',
'description': 'Invalid output format'
}
def format_outputs_section(outputs: List[Any]) -> str:
"""
Format the outputs section for the documentation.
Args:
outputs: List of output definitions from manifest (strings or dicts)
Returns:
Formatted markdown table
"""
if not outputs:
return "_No outputs defined_\n"
lines = [
"| Output | Type | Description |",
"|--------|------|-------------|"
]
for out in outputs:
normalized = normalize_output(out)
name = normalized.get('name', 'unknown')
type_val = normalized.get('type', 'any')
description = normalized.get('description', 'No description')
lines.append(f"| `{name}` | {type_val} | {description} |")
return '\n'.join(lines) + '\n'
def generate_usage_template(manifest: Dict[str, Any]) -> str:
"""
Generate a usage template based on the skill manifest.
Args:
manifest: Parsed skill manifest
Returns:
Usage template string
"""
skill_name = manifest.get('name', 'skill')
inputs = manifest.get('inputs', [])
# Get the handler/script name
entrypoints = manifest.get('entrypoints', [])
if entrypoints:
handler = entrypoints[0].get('handler', f'{skill_name.replace(".", "_")}.py')
else:
handler = f'{skill_name.replace(".", "_")}.py'
# Build basic usage
skill_dir = f"skills/{skill_name}"
usage = f"```bash\npython {skill_dir}/{handler}"
# Normalize inputs
normalized_inputs = [normalize_input(inp) for inp in inputs]
# Add required positional arguments
required_inputs = [inp for inp in normalized_inputs if inp.get('required', False)]
for inp in required_inputs:
usage += f" <{inp['name']}>"
# Add optional arguments hint
if any(not inp.get('required', False) for inp in normalized_inputs):
usage += " [options]"
usage += "\n```"
return usage
def generate_parameters_detail(inputs: List[Any]) -> str:
"""
Generate detailed parameter documentation.
Args:
inputs: List of input definitions (strings or dicts)
Returns:
Formatted parameter details
"""
if not inputs:
return ""
lines = []
for inp in inputs:
normalized = normalize_input(inp)
name = normalized.get('name', 'unknown')
description = normalized.get('description', 'No description')
type_val = normalized.get('type', 'any')
required = normalized.get('required', False)
default = normalized.get('default')
detail = f"- `--{name}` ({type_val})"
if required:
detail += " **[Required]**"
detail += f": {description}"
if default is not None:
detail += f" (default: `{default}`)"
lines.append(detail)
return '\n'.join(lines) + '\n'
def generate_skill_documentation(manifest: Dict[str, Any]) -> str:
"""
Generate complete SKILL.md documentation from manifest.
Args:
manifest: Parsed skill manifest
Returns:
Generated markdown documentation
"""
name = manifest.get('name', 'Unknown Skill')
description = manifest.get('description', 'No description available')
version = manifest.get('version', '0.1.0')
inputs = manifest.get('inputs', [])
outputs = manifest.get('outputs', [])
tags = manifest.get('tags', [])
dependencies = manifest.get('dependencies', [])
# Build documentation
doc = f"""# {name}
## Overview
**{name}** {description}
## Purpose
{description}
This skill automatically generates documentation by:
- Reading skill.yaml manifest files
- Extracting inputs, outputs, and metadata
- Creating standardized SKILL.md documentation
- Ensuring consistency across all Betty skills
## Usage
### Basic Usage
{generate_usage_template(manifest)}
### Parameters
{format_inputs_section(inputs)}
## Outputs
{format_outputs_section(outputs)}
## Usage Template
### Example: Generate Documentation for a Skill
```bash
python skills/{name}/{'generate_docs.py' if '.' in name else name + '.py'} path/to/skill.yaml
```
### Example: Preview Without Writing
```bash
python skills/{name}/{'generate_docs.py' if '.' in name else name + '.py'} \\
path/to/skill.yaml \\
--dry-run=true
```
### Example: Overwrite Existing Documentation
```bash
python skills/{name}/{'generate_docs.py' if '.' in name else name + '.py'} \\
path/to/skill.yaml \\
--overwrite=true
```
## Integration Notes
### Use in Workflows
```yaml
# workflows/documentation.yaml
steps:
- skill: {name}
args:
- "skills/my.skill/skill.yaml"
- "--overwrite=true"
```
### Use with Other Skills
```bash
# Generate documentation for a newly created skill
python skills/skill.create/skill_create.py my.new.skill
python skills/{name}/{'generate_docs.py' if '.' in name else name + '.py'} skills/my.new.skill/skill.yaml
```
### Batch Documentation Generation
```bash
# Generate docs for all skills
for manifest in skills/*/skill.yaml; do
python skills/{name}/{'generate_docs.py' if '.' in name else name + '.py'} "$manifest" --overwrite=true
done
```
## Output Structure
The generated SKILL.md includes:
1. **Overview** - Skill name and brief description
2. **Purpose** - Detailed explanation of what the skill does
3. **Usage** - Command-line usage examples
4. **Parameters** - Detailed input parameter documentation
5. **Outputs** - Description of skill outputs
6. **Usage Template** - Practical examples
7. **Integration Notes** - How to use with workflows and other skills
## Dependencies
"""
if dependencies:
for dep in dependencies:
doc += f"- **{dep}**: Required dependency\n"
else:
doc += "_No external dependencies_\n"
doc += "\n## Tags\n\n"
if tags:
doc += ', '.join(f'`{tag}`' for tag in tags) + '\n'
else:
doc += "_No tags defined_\n"
doc += f"""
## See Also
- [Betty Architecture](../../docs/betty-architecture.md) - Five-layer model
- [Skill Development Guide](../../docs/skill-development.md) - Creating new skills
## Version
**{version}** - Generated documentation from skill manifest
"""
return doc
def generate_docs(
manifest_path: str,
overwrite: bool = False,
dry_run: bool = False,
output_path: Optional[str] = None
) -> Dict[str, Any]:
"""
Generate SKILL.md documentation from a skill manifest.
Args:
manifest_path: Path to skill.yaml file
overwrite: Whether to overwrite existing SKILL.md
dry_run: Preview without writing
output_path: Custom output path (optional)
Returns:
Result dictionary with doc path and content
Raises:
BettyError: If manifest is invalid or file operations fail
"""
# Load manifest
manifest = load_skill_manifest(manifest_path)
# Generate documentation
doc_content = generate_skill_documentation(manifest)
# Determine output path
if output_path:
doc_path = Path(output_path)
else:
# Default to same directory as manifest
manifest_file = Path(manifest_path)
doc_path = manifest_file.parent / "SKILL.md"
# Check if file exists and overwrite is False
if doc_path.exists() and not overwrite and not dry_run:
raise BettyError(
f"SKILL.md already exists at {doc_path}. "
f"Use --overwrite=true to replace it or --dry-run=true to preview."
)
result = {
"doc_path": str(doc_path),
"doc_content": doc_content,
"skill_name": manifest.get('name', 'unknown'),
"dry_run": dry_run
}
if dry_run:
result["dry_run_preview"] = doc_content
logger.info(f"DRY RUN: Would write documentation to {doc_path}")
print("\n" + "="*80)
print("DRY RUN PREVIEW")
print("="*80)
print(doc_content)
print("="*80)
return result
# Write documentation to file
try:
doc_path.parent.mkdir(parents=True, exist_ok=True)
with open(doc_path, 'w') as f:
f.write(doc_content)
logger.info(f"Generated SKILL.md at {doc_path}")
except Exception as e:
raise BettyError(f"Failed to write documentation: {e}")
return result
def main():
parser = argparse.ArgumentParser(
description="Generate or update SKILL.md documentation from skill.yaml manifest"
)
parser.add_argument(
"manifest_path",
type=str,
help="Path to skill.yaml manifest file"
)
parser.add_argument(
"--overwrite",
type=lambda x: x.lower() in ['true', '1', 'yes'],
default=False,
help="Overwrite existing SKILL.md file (default: false)"
)
parser.add_argument(
"--dry-run",
type=lambda x: x.lower() in ['true', '1', 'yes'],
default=False,
help="Preview without writing to disk (default: false)"
)
parser.add_argument(
"--output-path",
type=str,
help="Custom output path for SKILL.md (optional)"
)
args = parser.parse_args()
try:
# Check if PyYAML is installed
try:
import yaml
except ImportError:
raise BettyError(
"PyYAML is required for generate.docs. Install with: pip install pyyaml"
)
# Generate documentation
logger.info(f"Generating documentation from {args.manifest_path}")
result = generate_docs(
manifest_path=args.manifest_path,
overwrite=args.overwrite,
dry_run=args.dry_run,
output_path=args.output_path
)
# Return structured result
output = {
"status": "success",
"data": result
}
if not args.dry_run:
print(json.dumps(output, indent=2))
except Exception as e:
logger.error(f"Failed to generate documentation: {e}")
print(json.dumps(format_error_response(e), indent=2))
sys.exit(1)
if __name__ == "__main__":
main()

54
skills/generate.docs/skill.yaml Normal file
View File

@@ -0,0 +1,54 @@
name: generate.docs
version: 0.1.0
description: Automatically generate or update SKILL.md documentation from skill.yaml manifests
inputs:
- name: manifest_path
type: string
required: true
description: Path to skill.yaml manifest file to generate documentation from
- name: overwrite
type: boolean
required: false
default: false
description: Overwrite existing SKILL.md file if it exists
- name: dry_run
type: boolean
required: false
default: false
description: Preview the generated documentation without writing to disk
- name: output_path
type: string
required: false
description: Custom output path for SKILL.md (defaults to same directory as manifest)
outputs:
- name: doc_path
type: string
description: Path to generated or updated SKILL.md file
- name: doc_content
type: string
description: Generated documentation content
- name: dry_run_preview
type: string
description: Preview of documentation (when dry_run=true)
dependencies:
- context.schema
entrypoints:
- command: /skill/generate/docs
handler: generate_docs.py
runtime: python
permissions:
- filesystem:read
- filesystem:write
status: active
tags: [documentation, automation, scaffolding, skill-management]