zhongwei/gh-epieczko-betty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 18:26:08 +08:00
commit 8f22ddf339
295 changed files with 59710 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

490
skills/docs.sync.readme/SKILL.md Normal file
View File

@@ -0,0 +1,490 @@
---
name: Documentation README Sync
description: Automatically regenerate README.md from Betty Framework registries
---
# docs.sync.readme
## Overview
**docs.sync.readme** is the documentation synchronization tool that regenerates the top-level `README.md` to reflect all current registered skills and agents. It ensures that the README stays in sync with the actual state of the Betty Framework by pulling from registry files.
## Purpose
Automates the maintenance of `README.md` to keep documentation accurate and up-to-date with:
- **Skill Registry** (`registry/skills.json`) – All registered skills
- **Agent Registry** (`registry/agents.json`) – All registered agents
This eliminates manual editing of the README and prevents documentation drift as skills and agents are added, modified, or removed.
## What It Does
1. **Reads Registries**: Loads `skills.json` and `agents.json`
2. **Categorizes Skills**: Groups skills by tag/category:
- Foundation (skill.*, registry.*, workflow.*)
- API Development (api.*)
- Infrastructure (agents, commands, hooks, policy)
- Governance (policy, audit)
3. **Updates Sections**:
- Current Core Skills table with categorized skills
- Agents documentation links
- Skills documentation references
4. **Maintains Style**: Preserves README tone, formatting, and structure
5. **Generates Report**: Creates sync report with statistics
## Usage
### Basic Usage
```bash
python skills/docs.sync.readme/readme_sync.py
```
No arguments required - reads from standard registry locations.
### Via Betty CLI
```bash
/docs/sync/readme
```
### Expected Registry Structure
```
betty/
├── registry/
│ ├── skills.json # Skills registry
│ └── agents.json # Agents registry
└── README.md # File to update
```
## Behavior
### 1. Registry Loading
Reads JSON files from:
- `registry/skills.json` – Skills registry
- `registry/agents.json` – Agents registry
If a registry file is missing, logs a warning and continues with empty data.
### 2. Skill Categorization
**Foundation Skills**:
- Matches: `skill.*`, `registry.*`, `workflow.*`
- Examples: `skill.create`, `workflow.compose`
**API Development Skills**:
- Matches: `api.*` or tags: `api`, `openapi`, `asyncapi`
- Examples: `api.define`, `api.validate`
**Infrastructure Skills**:
- Matches tags: `agents`, `command`, `hook`, `policy`, `plugin`
- Examples: `agent.define`, `hook.register`, `plugin.sync`
**Governance Skills**:
- Matches tags: `governance`, `policy`, `audit`
- Examples: `policy.enforce`, `audit.log`
Only **active** skills are included. Test skills (starting with `test.`) are filtered out.
### 3. Skills Section Update
Replaces the "## 🧩 Current Core Skills" section with:
```markdown
## 🧩 Current Core Skills
Betty's self-referential "kernel" of skills bootstraps the rest of the system:
### Foundation Skills
| Skill | Purpose |
|--------|----------|
| **skill.create** | Generates a new Betty Framework Skill directory and manifest. |
| **skill.define** | Validates and registers skill manifests (.skill.yaml) for the Betty Framework. |
| **registry.update** | Updates the Betty Framework Skill Registry by adding or modifying entries. |
### API Development Skills
| Skill | Purpose |
|--------|----------|
| **api.define** | Create OpenAPI and AsyncAPI specifications from templates |
| **api.validate** | Validate OpenAPI and AsyncAPI specifications against enterprise guidelines |
### Infrastructure Skills
| Skill | Purpose |
|--------|----------|
| **agent.define** | Validates and registers agent manifests for the Betty Framework. |
| **hook.define** | Create and register validation hooks for Claude Code |
These skills form the baseline for an **AI-native SDLC** where creation, validation, registration, and orchestration are themselves skills.
```
### 4. Agents Section Update
Updates the "### Agents Documentation" subsection with current agents:
```markdown
### Agents Documentation
Each agent has a `README.md` in its directory:
* [api.designer](agents/api.designer/README.md) — Design RESTful APIs following enterprise guidelines with iterative refinement
* [api.analyzer](agents/api.analyzer/README.md) — Analyze API specifications for backward compatibility and breaking changes
```
Includes both `active` and `draft` agents.
### 5. Report Generation
Creates `sync_report.json` with statistics:
```json
{
"skills_by_category": {
"foundation": 5,
"api": 4,
"infrastructure": 9,
"governance": 1
},
"total_skills": 19,
"agents_count": 2,
"timestamp": "2025-10-23T20:30:00.123456+00:00"
}
```
## Outputs
### Success Response
```json
{
"ok": true,
"status": "success",
"readme_path": "/home/user/betty/README.md",
"report": {
"skills_by_category": {
"foundation": 5,
"api": 4,
"infrastructure": 9,
"governance": 1
},
"total_skills": 19,
"agents_count": 2,
"timestamp": "2025-10-23T20:30:00.123456+00:00"
}
}
```
### Failure Response
```json
{
"ok": false,
"status": "failed",
"error": "README.md not found at /home/user/betty/README.md"
}
```
## What Gets Updated
### ✅ Updated Sections
- **Current Core Skills** (categorized tables)
- **Agents Documentation** (agent links list)
- Skills documentation references
### ❌ Not Modified
- Mission and inspiration
- Purpose and scope
- Repository structure
- Design principles
- Roadmap
- Contributing guidelines
- Requirements
The skill only updates specific documentation sections while preserving all other README content.
## Examples
### Example 1: Sync After Adding New Skills
**Scenario**: You've added several new skills and want to update the README
```bash
# Create and register new skills
/skill/create data.transform "Transform data between formats"
/skill/define skills/data.transform/skill.yaml
/skill/create telemetry.report "Generate telemetry reports"
/skill/define skills/telemetry.report/skill.yaml
# Sync README to include new skills
/docs/sync/readme
```
**Output**:
```
INFO: Starting README.md sync from registries...
INFO: Loading registry files...
INFO: Generating updated README content...
INFO: ✅ Updated README.md
INFO: - Foundation skills: 5
INFO: - API skills: 4
INFO: - Infrastructure skills: 11
INFO: - Governance skills: 1
INFO: - Total active skills: 21
INFO: - Agents: 2
```
### Example 2: Sync After Adding New Agent
**Scenario**: A new agent has been registered and needs to appear in README
```bash
# Define new agent
/agent/define agents/workflow.optimizer/agent.yaml
# Sync README
/docs/sync/readme
```
The new agent will appear in the "### Agents Documentation" section.
### Example 3: Automated Sync in Workflow
**Scenario**: Include README sync as a workflow step after registering skills
```yaml
# workflows/skill_release.yaml
steps:
- skill: skill.define
args: ["skills/new.skill/skill.yaml"]
- skill: plugin.sync
args: []
- skill: docs.sync.readme
args: []
```
This ensures README, plugin.yaml, and registries stay in sync.
## Integration
### With skill.define
After defining skills, sync the README:
```bash
/skill/define skills/my.skill/skill.yaml
/docs/sync/readme
```
### With agent.define
After defining agents, sync the README:
```bash
/agent/define agents/my.agent/agent.yaml
/docs/sync/readme
```
### With Hooks
Auto-sync README when registries change:
```yaml
# .claude/hooks.yaml
- event: on_file_save
pattern: "registry/*.json"
command: python skills/docs.sync.readme/readme_sync.py
blocking: false
description: Auto-sync README when registries change
```
### With plugin.sync
Chain both sync operations:
```bash
/plugin/sync && /docs/sync/readme
```
## Categorization Rules
### Foundation Category
**Criteria**:
- Skill name starts with: `skill.`, `registry.`, `workflow.`
- Core Betty framework functionality
**Examples**:
- `skill.create`, `skill.define`
- `registry.update`, `registry.query`
- `workflow.compose`, `workflow.validate`
### API Category
**Criteria**:
- Skill name starts with: `api.`
- Tags include: `api`, `openapi`, `asyncapi`
**Examples**:
- `api.define`, `api.validate`
- `api.generate-models`, `api.compatibility`
### Infrastructure Category
**Criteria**:
- Tags include: `agents`, `command`, `hook`, `policy`, `plugin`, `registry`
- Infrastructure and orchestration skills
**Examples**:
- `agent.define`, `agent.run`
- `hook.define`, `hook.register`
- `plugin.sync`, `plugin.build`
### Governance Category
**Criteria**:
- Tags include: `governance`, `policy`, `audit`
- Policy enforcement and audit trails
**Examples**:
- `policy.enforce`
- `audit.log`
## Filtering Rules
### ✅ Included
- Skills with `status: active`
- Agents with `status: active` or `status: draft`
- Skills with meaningful descriptions
### ❌ Excluded
- Skills with `status: draft`
- Skills starting with `test.`
- Skills without names or descriptions
## Common Errors
| Error | Cause | Solution |
|-------|-------|----------|
| "README.md not found" | Missing README file | Ensure README.md exists in repo root |
| "Registry file not found" | Missing registry | Run skill.define to populate registry |
| "Failed to parse JSON" | Invalid JSON | Fix JSON syntax in registry files |
## Files Read
- `README.md` – Current README content
- `registry/skills.json` – Skills registry
- `registry/agents.json` – Agents registry
## Files Modified
- `README.md` – Updated with current skills and agents
- `skills/docs.sync.readme/sync_report.json` – Sync statistics
## Exit Codes
- **0**: Success (README updated successfully)
- **1**: Failure (error during sync)
## Logging
Logs sync progress:
```
INFO: Starting README.md sync from registries...
INFO: Loading registry files...
INFO: Generating updated README content...
INFO: ✅ Updated README.md
INFO: - Foundation skills: 5
INFO: - API skills: 4
INFO: - Infrastructure skills: 9
INFO: - Governance skills: 1
INFO: - Total active skills: 19
INFO: - Agents: 2
```
## Best Practices
1. **Run After Registry Changes**: Sync README whenever skills or agents are added/updated
2. **Include in CI/CD**: Add README sync to deployment pipelines
3. **Review Before Commit**: Check updated README before committing changes
4. **Use Hooks**: Set up auto-sync hooks for convenience
5. **Combine with plugin.sync**: Keep both plugin.yaml and README in sync
6. **Version Control**: Always commit README changes with skill/agent changes
## Troubleshooting
### README Not Updating
**Problem**: Changes to registry don't appear in README
**Solutions**:
- Ensure skills have `status: active`
- Check that skill names and descriptions are present
- Verify registry files are valid JSON
- Run `/skill/define` before syncing README
### Skills in Wrong Category
**Problem**: Skill appears in unexpected category
**Solutions**:
- Check skill tags in skill.yaml
- Verify tag categorization rules above
- Add appropriate tags to skill.yaml
- Re-run skill.define to update registry
### Section Markers Not Found
**Problem**: "Section marker not found" warnings
**Solutions**:
- Ensure README has expected section headers
- Check for typos in section headers
- Restore original README structure if modified
- Update section_marker strings in code if intentionally changed
## Architecture
### Skill Categories
**Documentation** – docs.sync.readme maintains the README documentation layer by syncing registry state to the top-level README.
### Design Principles
- **Single Source of Truth**: Registries are the source of truth
- **Preserve Structure**: Only update specific sections
- **Maintain Style**: Keep original tone and formatting
- **Clear Categorization**: Logical grouping of skills by function
- **Idempotent**: Can be run multiple times safely
## See Also
- **plugin.sync** – Sync plugin.yaml with registries ([SKILL.md](../plugin.sync/SKILL.md))
- **skill.define** – Validate and register skills ([SKILL.md](../skill.define/SKILL.md))
- **agent.define** – Validate and register agents ([SKILL.md](../agent.define/SKILL.md))
- **registry.update** – Update registries ([SKILL.md](../registry.update/SKILL.md))
- **Betty Architecture** – Framework overview ([betty-architecture.md](../../docs/betty-architecture.md))
## Dependencies
- **registry.update**: Registry management
- **betty.config**: Configuration constants and paths
- **betty.logging_utils**: Logging infrastructure
## Status
**Active** – Production-ready documentation skill
## Version History
- **0.1.0** (Oct 2025) – Initial implementation with skills categorization and agents documentation

1
skills/docs.sync.readme/__init__.py Normal file
View File

@@ -0,0 +1 @@
# Auto-generated package initializer for skills.

399
skills/docs.sync.readme/readme_sync.py Executable file
View File

@@ -0,0 +1,399 @@
#!/usr/bin/env python3
"""
readme_sync.py - Implementation of the docs.sync.readme Skill
Regenerates the top-level README.md to reflect all current registered skills and agents.
"""
import os
import sys
import json
import re
from typing import Dict, Any, List, Optional
from datetime import datetime, timezone
from pathlib import Path
from betty.config import BASE_DIR, REGISTRY_FILE, AGENTS_REGISTRY_FILE
from betty.logging_utils import setup_logger
logger = setup_logger(__name__)
def load_registry(registry_path: str) -> Dict[str, Any]:
"""
Load a JSON registry file.
Args:
registry_path: Path to registry JSON file
Returns:
Parsed registry data
"""
try:
with open(registry_path) as f:
return json.load(f)
except FileNotFoundError:
logger.warning(f"Registry file not found: {registry_path}")
return {"skills": []} if "skills" in registry_path else {"agents": []}
except json.JSONDecodeError as e:
logger.error(f"Failed to parse JSON from {registry_path}: {e}")
raise
def categorize_skills(skills: List[Dict[str, Any]]) -> Dict[str, List[Dict[str, Any]]]:
"""
Categorize skills by their tags into foundation, api, infrastructure, and governance groups.
Args:
skills: List of skill dictionaries from registry
Returns:
Dictionary mapping category names to lists of skills
"""
categories = {
"foundation": [],
"api": [],
"infrastructure": [],
"governance": []
}
for skill in skills:
# Only include active skills
if skill.get("status") != "active":
continue
# Skip test skills
if skill.get("name", "").startswith("test."):
continue
tags = skill.get("tags", [])
name = skill.get("name", "")
# Categorize based on tags or name patterns
if any(tag in ["api", "openapi", "asyncapi"] for tag in tags) or name.startswith("api."):
categories["api"].append(skill)
elif any(tag in ["agents", "command", "hook", "policy", "plugin", "registry"] for tag in tags):
categories["infrastructure"].append(skill)
elif any(tag in ["governance", "policy", "audit"] for tag in tags):
categories["governance"].append(skill)
elif name.startswith("skill.") or name.startswith("registry.") or name.startswith("workflow."):
categories["foundation"].append(skill)
else:
# Default to infrastructure if unclear
categories["infrastructure"].append(skill)
# Remove duplicates and sort by name
for category in categories:
seen = set()
unique_skills = []
for skill in categories[category]:
if skill["name"] not in seen:
seen.add(skill["name"])
unique_skills.append(skill)
categories[category] = sorted(unique_skills, key=lambda s: s["name"])
return categories
def format_skill_table(skills: List[Dict[str, Any]]) -> str:
"""
Format a list of skills as a markdown table.
Args:
skills: List of skill dictionaries
Returns:
Markdown table string
"""
if not skills:
return "| Skill | Purpose |\n|--------|----------|\n| _(No skills in this category)_ | |"
lines = ["| Skill | Purpose |", "|--------|----------|"]
for skill in skills:
name = skill.get("name", "")
# Get first line of description only
desc = skill.get("description", "").strip().split("\n")[0]
# Clean up description (remove extra whitespace)
desc = " ".join(desc.split())
lines.append(f"| **{name}** | {desc} |")
return "\n".join(lines)
def format_agents_docs(agents: List[Dict[str, Any]]) -> str:
"""
Format agent documentation links.
Args:
agents: List of agent dictionaries
Returns:
Markdown list of agent links
"""
if not agents:
return "_(No agents registered)_"
lines = []
for agent in agents:
name = agent.get("name", "")
# Get first line of description only
desc = agent.get("description", "").strip().split("\n")[0]
desc = " ".join(desc.split())
lines.append(f"* [{name}](agents/{name}/README.md) — {desc}")
return "\n".join(lines)
def update_readme_section(
content: str,
section_marker: str,
end_marker: str,
new_content: str
) -> str:
"""
Update a section of the README between two markers.
Args:
content: Full README content
section_marker: Start marker (e.g., "## 🧩 Current Core Skills")
end_marker: End marker (e.g., "---")
new_content: New content to insert between markers
Returns:
Updated README content
"""
# Find section start
section_start = content.find(section_marker)
if section_start == -1:
logger.warning(f"Section marker not found: {section_marker}")
return content
# Find section end after the start - look for the end marker on its own line
search_start = section_start + len(section_marker)
end_marker_pattern = f"\n{end_marker}\n"
section_end = content.find(end_marker_pattern, search_start)
if section_end == -1:
logger.warning(f"End marker not found after {section_marker}")
return content
# Replace the section (include the newline before end marker)
before = content[:section_start]
after = content[section_end + 1:] # +1 to skip the first newline
return before + section_marker + "\n\n" + new_content + "\n" + after
def generate_skills_section(categories: Dict[str, List[Dict[str, Any]]]) -> str:
"""
Generate the complete skills section content.
Args:
categories: Dictionary of categorized skills
Returns:
Markdown content for skills section
"""
lines = [
"Betty's self-referential \"kernel\" of skills bootstraps the rest of the system:",
""
]
# Foundation Skills
if categories["foundation"]:
lines.extend([
"### Foundation Skills",
"",
format_skill_table(categories["foundation"]),
""
])
# API Development Skills
if categories["api"]:
lines.extend([
"### API Development Skills",
"",
format_skill_table(categories["api"]),
""
])
# Infrastructure Skills
if categories["infrastructure"]:
lines.extend([
"### Infrastructure Skills",
"",
format_skill_table(categories["infrastructure"]),
""
])
# Governance Skills (if any)
if categories["governance"]:
lines.extend([
"### Governance Skills",
"",
format_skill_table(categories["governance"]),
""
])
lines.append("These skills form the baseline for an **AI-native SDLC** where creation, validation, registration, and orchestration are themselves skills.")
return "\n".join(lines)
def update_agents_section(content: str, agents: List[Dict[str, Any]]) -> str:
"""
Update the Agents Documentation section.
Args:
content: Full README content
agents: List of active agents
Returns:
Updated README content
"""
agents_docs = format_agents_docs(agents)
# Find the "### Agents Documentation" section
section_start = content.find("### Agents Documentation")
if section_start == -1:
logger.warning("Agents Documentation section not found")
return content
# Find the next ### or ## to determine section end
next_section = content.find("\n##", section_start + 25)
if next_section == -1:
next_section = len(content)
# Find "Each agent has a" line as the start of actual content
intro_start = content.find("Each agent has a `README.md` in its directory:", section_start)
if intro_start == -1:
intro_start = section_start + 25
else:
intro_start += len("Each agent has a `README.md` in its directory:")
before = content[:intro_start]
after = content[next_section:]
return before + "\n" + agents_docs + "\n\n" + after
def generate_readme(
skills_data: Dict[str, Any],
agents_data: Dict[str, Any]
) -> tuple[str, Dict[str, Any]]:
"""
Generate updated README.md content.
Args:
skills_data: Parsed skills.json
agents_data: Parsed agents.json
Returns:
Tuple of (updated_readme_content, report_dict)
"""
readme_path = os.path.join(BASE_DIR, "README.md")
# Read current README
try:
with open(readme_path) as f:
content = f.read()
except FileNotFoundError:
logger.error(f"README.md not found at {readme_path}")
raise
# Categorize skills
skills = skills_data.get("skills", [])
categories = categorize_skills(skills)
# Get active agents
agents = [a for a in agents_data.get("agents", []) if a.get("status") == "active" or a.get("status") == "draft"]
agents = sorted(agents, key=lambda a: a["name"])
# Generate new skills section
skills_section = generate_skills_section(categories)
# Update skills section
content = update_readme_section(
content,
"## 🧩 Current Core Skills",
"---",
skills_section
)
# Update agents section
content = update_agents_section(content, agents)
# Generate report
report = {
"skills_by_category": {
"foundation": len(categories["foundation"]),
"api": len(categories["api"]),
"infrastructure": len(categories["infrastructure"]),
"governance": len(categories["governance"])
},
"total_skills": sum(len(skills) for skills in categories.values()),
"agents_count": len(agents),
"timestamp": datetime.now(timezone.utc).isoformat()
}
return content, report
def main():
"""Main CLI entry point."""
logger.info("Starting README.md sync from registries...")
try:
# Load registries
logger.info("Loading registry files...")
skills_data = load_registry(REGISTRY_FILE)
agents_data = load_registry(AGENTS_REGISTRY_FILE)
# Generate updated README
logger.info("Generating updated README content...")
readme_content, report = generate_readme(skills_data, agents_data)
# Write README
readme_path = os.path.join(BASE_DIR, "README.md")
with open(readme_path, 'w') as f:
f.write(readme_content)
logger.info(f"✅ Updated README.md")
logger.info(f" - Foundation skills: {report['skills_by_category']['foundation']}")
logger.info(f" - API skills: {report['skills_by_category']['api']}")
logger.info(f" - Infrastructure skills: {report['skills_by_category']['infrastructure']}")
logger.info(f" - Governance skills: {report['skills_by_category']['governance']}")
logger.info(f" - Total active skills: {report['total_skills']}")
logger.info(f" - Agents: {report['agents_count']}")
# Write report
report_path = os.path.join(BASE_DIR, "skills", "docs.sync.readme", "sync_report.json")
with open(report_path, 'w') as f:
json.dump(report, f, indent=2)
result = {
"ok": True,
"status": "success",
"readme_path": readme_path,
"report": report
}
print(json.dumps(result, indent=2))
sys.exit(0)
except Exception as e:
logger.error(f"Failed to sync README: {e}")
result = {
"ok": False,
"status": "failed",
"error": str(e)
}
print(json.dumps(result, indent=2))
sys.exit(1)
if __name__ == "__main__":
main()

31
skills/docs.sync.readme/skill.yaml Normal file
View File

@@ -0,0 +1,31 @@
name: docs.sync.readme
version: 0.1.0
description: >
Regenerate the top-level README.md to reflect all current registered skills and agents.
Pulls from registry/skills.json and registry/agents.json, groups by category, and updates
documentation sections while maintaining repo style and tone.
inputs: []
outputs:
- README.md
- sync_report.json
dependencies:
- registry.update
status: active
entrypoints:
- command: /docs/sync/readme
handler: readme_sync.py
runtime: python
description: >
Sync README.md with current registry state. Updates skills tables, agents links,
and documentation map to reflect all registered skills and agents.
parameters: []
permissions:
- filesystem:read
- filesystem:write
tags:
- documentation
- registry
- automation
- maintenance

11
skills/docs.sync.readme/sync_report.json Normal file
View File

@@ -0,0 +1,11 @@
{
"skills_by_category": {
"foundation": 5,
"api": 4,
"infrastructure": 11,
"governance": 0
},
"total_skills": 20,
"agents_count": 2,
"timestamp": "2025-10-23T19:58:22.843605+00:00"
}