Initial commit

skills/api.generatemodels/SKILL.md

@@ -0,0 +1,299 @@
+# api.generate-models
+
+## Overview
+
+**api.generate-models** generates type-safe models from OpenAPI and AsyncAPI specifications, enabling shared models between frontend and backend using code generation.
+
+## Purpose
+
+Transform API specifications into type-safe code:
+- Generate TypeScript interfaces from OpenAPI schemas
+- Generate Python dataclasses/Pydantic models
+- Generate Java classes, Go structs, C# classes
+- Single source of truth: the API specification
+- Automatic synchronization when specs change
+
+## Usage
+
+### Basic Usage
+
+```bash
+python skills/api.generate-models/modelina_generate.py <spec_path> <language> [options]
+```
+
+### Parameters
+
+| Parameter | Required | Description | Default |
+|-----------|----------|-------------|---------|
+| `spec_path` | Yes | Path to API spec file | - |
+| `language` | Yes | Target language | - |
+| `--output-dir` | No | Output directory | `src/models` |
+| `--package-name` | No | Package/module name | - |
+
+### Supported Languages
+
+| Language | Extension | Status |
+|----------|-----------|--------|
+| `typescript` | `.ts` | ✅ Supported |
+| `python` | `.py` | ✅ Supported |
+| `java` | `.java` | 🚧 Planned |
+| `go` | `.go` | 🚧 Planned |
+| `csharp` | `.cs` | 🚧 Planned |
+| `rust` | `.rs` | 🚧 Planned |
+
+## Examples
+
+### Example 1: Generate TypeScript Models
+
+```bash
+python skills/api.generate-models/modelina_generate.py \
+  specs/user-service.openapi.yaml \
+  typescript \
+  --output-dir=src/models/user-service
+```
+
+**Generated files**:
+```
+src/models/user-service/
+├── User.ts
+├── UserCreate.ts
+├── UserUpdate.ts
+├── Pagination.ts
+└── Problem.ts
+```
+
+**Example TypeScript output**:
+```typescript
+// src/models/user-service/User.ts
+export interface User {
+  /** Unique identifier */
+  user_id: string;
+  /** Creation timestamp */
+  created_at: string;
+  /** Last update timestamp */
+  updated_at?: string;
+}
+
+// src/models/user-service/Pagination.ts
+export interface Pagination {
+  /** Number of items per page */
+  limit: number;
+  /** Number of items skipped */
+  offset: number;
+  /** Total number of items available */
+  total: number;
+}
+```
+
+### Example 2: Generate Python Models
+
+```bash
+python skills/api.generate-models/modelina_generate.py \
+  specs/user-service.openapi.yaml \
+  python \
+  --output-dir=src/models/user_service
+```
+
+**Generated files**:
+```
+src/models/user_service/
+└── models.py
+```
+
+**Example Python output**:
+```python
+# src/models/user_service/models.py
+from pydantic import BaseModel, Field
+from typing import Optional
+from datetime import datetime
+from uuid import UUID
+
+class User(BaseModel):
+    """User model"""
+    user_id: UUID = Field(..., description="Unique identifier")
+    created_at: datetime = Field(..., description="Creation timestamp")
+    updated_at: Optional[datetime] = Field(None, description="Last update timestamp")
+
+class Pagination(BaseModel):
+    """Pagination metadata"""
+    limit: int = Field(..., description="Number of items per page")
+    offset: int = Field(..., description="Number of items skipped")
+    total: int = Field(..., description="Total number of items available")
+```
+
+### Example 3: Generate for Multiple Languages
+
+```bash
+# TypeScript for frontend
+python skills/api.generate-models/modelina_generate.py \
+  specs/user-service.openapi.yaml \
+  typescript \
+  --output-dir=frontend/src/models
+
+# Python for backend
+python skills/api.generate-models/modelina_generate.py \
+  specs/user-service.openapi.yaml \
+  python \
+  --output-dir=backend/app/models
+```
+
+## Code Generators Used
+
+The skill uses multiple code generation approaches:
+
+### 1. datamodel-code-generator (Primary)
+
+**Best for**: OpenAPI specs → Python/TypeScript
+**Installation**: `pip install datamodel-code-generator`
+
+Generates:
+- Python: Pydantic v2 models with type hints
+- TypeScript: Type-safe interfaces
+- Validates schema during generation
+
+### 2. Simple Built-in Generator (Fallback)
+
+**Best for**: Basic models when external tools not available
+**Installation**: None required
+
+Generates:
+- Python: dataclasses
+- TypeScript: interfaces
+- Basic but reliable
+
+### 3. Modelina (Future)
+
+**Best for**: AsyncAPI specs, multiple languages
+**Installation**: `npm install -g @asyncapi/modelina`
+**Status**: Planned
+
+## Output
+
+### Success Response
+
+```json
+{
+  "status": "success",
+  "data": {
+    "models_path": "src/models/user-service",
+    "files_generated": [
+      "src/models/user-service/User.ts",
+      "src/models/user-service/UserCreate.ts",
+      "src/models/user-service/Pagination.ts",
+      "src/models/user-service/Problem.ts"
+    ],
+    "model_count": 4,
+    "generator_used": "datamodel-code-generator"
+  }
+}
+```
+
+## Integration with Workflows
+
+```yaml
+# workflows/api_first_development.yaml
+steps:
+  - skill: api.define
+    args:
+      - "user-service"
+      - "openapi"
+    output: spec_path
+
+  - skill: api.validate
+    args:
+      - "{spec_path}"
+      - "zalando"
+    required: true
+
+  - skill: api.generate-models
+    args:
+      - "{spec_path}"
+      - "typescript"
+      - "--output-dir=frontend/src/models"
+
+  - skill: api.generate-models
+    args:
+      - "{spec_path}"
+      - "python"
+      - "--output-dir=backend/app/models"
+```
+
+## Integration with Hooks
+
+Auto-regenerate models when specs change:
+
+```bash
+python skills/hook.define/hook_define.py \
+  on_file_save \
+  "python betty/skills/api.generate-models/modelina_generate.py {file_path} typescript --output-dir=src/models" \
+  --pattern="specs/*.openapi.yaml" \
+  --blocking=false \
+  --description="Auto-regenerate TypeScript models when OpenAPI specs change"
+```
+
+## Benefits
+
+### For Developers
+- ✅ **Type safety**: Catch errors at compile time, not runtime
+- ✅ **IDE autocomplete**: Full IntelliSense/autocomplete support
+- ✅ **No manual typing**: Models generated automatically
+- ✅ **Always in sync**: Regenerate when spec changes
+
+### For Teams
+- ✅ **Single source of truth**: API spec defines types
+- ✅ **Frontend/backend alignment**: Same types everywhere
+- ✅ **Reduced errors**: Type mismatches caught early
+- ✅ **Faster development**: No manual model creation
+
+### For Organizations
+- ✅ **Consistency**: All services use same model generation
+- ✅ **Maintainability**: Update spec → regenerate → done
+- ✅ **Documentation**: Types are self-documenting
+- ✅ **Quality**: Generated code is tested and reliable
+
+## Dependencies
+
+### Required
+- **PyYAML**: For YAML parsing (`pip install pyyaml`)
+
+### Optional (Better Output)
+- **datamodel-code-generator**: For high-quality Python/TypeScript (`pip install datamodel-code-generator`)
+- **Node.js + Modelina**: For AsyncAPI and more languages (`npm install -g @asyncapi/modelina`)
+
+## Examples with Real Specs
+
+Using the user-service spec from Phase 1:
+
+```bash
+# Generate TypeScript
+python skills/api.generate-models/modelina_generate.py \
+  specs/user-service.openapi.yaml \
+  typescript
+
+# Output:
+{
+  "status": "success",
+  "data": {
+    "models_path": "src/models",
+    "files_generated": [
+      "src/models/User.ts",
+      "src/models/UserCreate.ts",
+      "src/models/UserUpdate.ts",
+      "src/models/Pagination.ts",
+      "src/models/Problem.ts"
+    ],
+    "model_count": 5
+  }
+}
+```
+
+## See Also
+
+- [api.define](../api.define/SKILL.md) - Create OpenAPI specs
+- [api.validate](../api.validate/SKILL.md) - Validate specs
+- [Betty Architecture](../../docs/betty-architecture.md) - Five-layer model
+- [API-Driven Development](../../docs/api-driven-development.md) - Complete guide
+
+## Version
+
+**0.1.0** - Initial implementation with TypeScript and Python support

skills/api.generatemodels/__init__.py

@@ -0,0 +1 @@
+# Auto-generated package initializer for skills.

skills/api.generatemodels/modelina_generate.py

@@ -0,0 +1,549 @@
+#!/usr/bin/env python3
+"""
+Generate type-safe models from OpenAPI and AsyncAPI specifications using Modelina.
+
+This skill uses AsyncAPI Modelina to generate models in various languages
+from API specifications.
+"""
+
+import sys
+import json
+import argparse
+import subprocess
+import shutil
+from pathlib import Path
+from typing import Dict, Any, List
+
+# Add betty module to path
+
+from betty.logging_utils import setup_logger
+from betty.errors import format_error_response, BettyError
+from betty.validation import validate_path
+
+logger = setup_logger(__name__)
+
+# Supported languages
+SUPPORTED_LANGUAGES = [
+    "typescript",
+    "python",
+    "java",
+    "go",
+    "csharp",
+    "rust",
+    "kotlin",
+    "dart"
+]
+
+# Language-specific configurations
+LANGUAGE_CONFIG = {
+    "typescript": {
+        "extension": ".ts",
+        "package_json_required": False,
+        "modelina_generator": "typescript"
+    },
+    "python": {
+        "extension": ".py",
+        "package_json_required": False,
+        "modelina_generator": "python"
+    },
+    "java": {
+        "extension": ".java",
+        "package_json_required": False,
+        "modelina_generator": "java"
+    },
+    "go": {
+        "extension": ".go",
+        "package_json_required": False,
+        "modelina_generator": "go"
+    },
+    "csharp": {
+        "extension": ".cs",
+        "package_json_required": False,
+        "modelina_generator": "csharp"
+    }
+}
+
+
+def check_node_installed() -> bool:
+    """
+    Check if Node.js is installed.
+
+    Returns:
+        True if Node.js is available, False otherwise
+    """
+    try:
+        result = subprocess.run(
+            ["node", "--version"],
+            capture_output=True,
+            text=True,
+            timeout=5
+        )
+        if result.returncode == 0:
+            version = result.stdout.strip()
+            logger.info(f"Node.js found: {version}")
+            return True
+        return False
+    except (FileNotFoundError, subprocess.TimeoutExpired):
+        return False
+
+
+def check_npx_installed() -> bool:
+    """
+    Check if npx is installed.
+
+    Returns:
+        True if npx is available, False otherwise
+    """
+    try:
+        result = subprocess.run(
+            ["npx", "--version"],
+            capture_output=True,
+            text=True,
+            timeout=5
+        )
+        return result.returncode == 0
+    except (FileNotFoundError, subprocess.TimeoutExpired):
+        return False
+
+
+def generate_modelina_script(
+    spec_path: str,
+    language: str,
+    output_dir: str,
+    package_name: str = None
+) -> str:
+    """
+    Generate a Node.js script that uses Modelina to generate models.
+
+    Args:
+        spec_path: Path to spec file
+        language: Target language
+        output_dir: Output directory
+        package_name: Package name (optional)
+
+    Returns:
+        JavaScript code as string
+    """
+    # Modelina generator based on language
+    generator_map = {
+        "typescript": "TypeScriptGenerator",
+        "python": "PythonGenerator",
+        "java": "JavaGenerator",
+        "go": "GoGenerator",
+        "csharp": "CSharpGenerator"
+    }
+
+    generator_class = generator_map.get(language, "TypeScriptGenerator")
+
+    script = f"""
+const {{ {generator_class} }} = require('@asyncapi/modelina');
+const fs = require('fs');
+const path = require('path');
+
+async function generate() {{
+    try {{
+        // Read the spec file
+        const spec = fs.readFileSync('{spec_path}', 'utf8');
+        const specData = JSON.parse(spec);
+
+        // Create generator
+        const generator = new {generator_class}();
+
+        // Generate models
+        const models = await generator.generate(specData);
+
+        // Ensure output directory exists
+        const outputDir = '{output_dir}';
+        if (!fs.existsSync(outputDir)) {{
+            fs.mkdirSync(outputDir, {{ recursive: true }});
+        }}
+
+        // Write models to files
+        const filesGenerated = [];
+        for (const model of models) {{
+            const filePath = path.join(outputDir, model.name + model.extension);
+            fs.writeFileSync(filePath, model.result);
+            filesGenerated.push(filePath);
+        }}
+
+        // Output result
+        console.log(JSON.stringify({{
+            success: true,
+            files_generated: filesGenerated,
+            model_count: models.length
+        }}));
+
+    }} catch (error) {{
+        console.error(JSON.stringify({{
+            success: false,
+            error: error.message,
+            stack: error.stack
+        }}));
+        process.exit(1);
+    }}
+}}
+
+generate();
+"""
+    return script
+
+
+def generate_models_datamodel_code_generator(
+    spec_path: str,
+    language: str,
+    output_dir: str,
+    package_name: str = None
+) -> Dict[str, Any]:
+    """
+    Generate models using datamodel-code-generator (Python fallback).
+
+    This is used when Modelina/Node.js is not available.
+    Works for OpenAPI specs only, generating Python/TypeScript models.
+
+    Args:
+        spec_path: Path to specification file
+        language: Target language
+        output_dir: Output directory
+        package_name: Package name
+
+    Returns:
+        Result dictionary
+    """
+    try:
+        # Check if datamodel-code-generator is installed
+        result = subprocess.run(
+            ["datamodel-codegen", "--version"],
+            capture_output=True,
+            text=True,
+            timeout=5
+        )
+
+        if result.returncode != 0:
+            raise BettyError(
+                "datamodel-code-generator not installed. "
+                "Install with: pip install datamodel-code-generator"
+            )
+
+    except FileNotFoundError:
+        raise BettyError(
+            "datamodel-code-generator not found. "
+            "Install with: pip install datamodel-code-generator"
+        )
+
+    # Create output directory
+    output_path = Path(output_dir)
+    output_path.mkdir(parents=True, exist_ok=True)
+
+    # Determine output file based on language
+    if language == "python":
+        output_file = output_path / "models.py"
+        cmd = [
+            "datamodel-codegen",
+            "--input", spec_path,
+            "--output", str(output_file),
+            "--input-file-type", "openapi",
+            "--output-model-type", "pydantic_v2.BaseModel",
+            "--snake-case-field",
+            "--use-standard-collections"
+        ]
+    elif language == "typescript":
+        output_file = output_path / "models.ts"
+        cmd = [
+            "datamodel-codegen",
+            "--input", spec_path,
+            "--output", str(output_file),
+            "--input-file-type", "openapi",
+            "--output-model-type", "typescript"
+        ]
+    else:
+        raise BettyError(
+            f"datamodel-code-generator fallback only supports Python and TypeScript, not {language}"
+        )
+
+    # Run code generator
+    logger.info(f"Running datamodel-code-generator: {' '.join(cmd)}")
+    result = subprocess.run(
+        cmd,
+        capture_output=True,
+        text=True,
+        timeout=60
+    )
+
+    if result.returncode != 0:
+        raise BettyError(f"Code generation failed: {result.stderr}")
+
+    # Count generated files
+    files_generated = [str(output_file)]
+
+    return {
+        "models_path": str(output_path),
+        "files_generated": files_generated,
+        "model_count": 1,
+        "generator_used": "datamodel-code-generator"
+    }
+
+
+def generate_models_simple(
+    spec_path: str,
+    language: str,
+    output_dir: str,
+    package_name: str = None
+) -> Dict[str, Any]:
+    """
+    Simple model generation without external tools.
+
+    Generates basic model files from OpenAPI schemas as a last resort.
+
+    Args:
+        spec_path: Path to specification file
+        language: Target language
+        output_dir: Output directory
+        package_name: Package name
+
+    Returns:
+        Result dictionary
+    """
+    import yaml
+
+    # Load spec
+    with open(spec_path, 'r') as f:
+        spec = yaml.safe_load(f)
+
+    # Get schemas
+    schemas = spec.get("components", {}).get("schemas", {})
+
+    if not schemas:
+        raise BettyError("No schemas found in specification")
+
+    # Create output directory
+    output_path = Path(output_dir)
+    output_path.mkdir(parents=True, exist_ok=True)
+
+    files_generated = []
+
+    # Generate basic models for each schema
+    for schema_name, schema_def in schemas.items():
+        if language == "typescript":
+            content = generate_typescript_interface(schema_name, schema_def)
+            file_path = output_path / f"{schema_name}.ts"
+        elif language == "python":
+            content = generate_python_dataclass(schema_name, schema_def)
+            file_path = output_path / f"{schema_name.lower()}.py"
+        else:
+            raise BettyError(f"Simple generation only supports TypeScript and Python, not {language}")
+
+        with open(file_path, 'w') as f:
+            f.write(content)
+
+        files_generated.append(str(file_path))
+        logger.info(f"Generated {file_path}")
+
+    return {
+        "models_path": str(output_path),
+        "files_generated": files_generated,
+        "model_count": len(schemas),
+        "generator_used": "simple"
+    }
+
+
+def generate_typescript_interface(name: str, schema: Dict[str, Any]) -> str:
+    """Generate TypeScript interface from schema."""
+    properties = schema.get("properties") or {}
+    required = schema.get("required", [])
+
+    lines = [f"export interface {name} {{"]
+
+    if not properties:
+        lines.append("  // No properties defined")
+
+    for prop_name, prop_def in properties.items():
+        prop_type = map_openapi_type_to_typescript(prop_def.get("type", "any"))
+        optional = "" if prop_name in required else "?"
+        description = prop_def.get("description", "")
+
+        if description:
+            lines.append(f"  /** {description} */")
+        lines.append(f"  {prop_name}{optional}: {prop_type};")
+
+    lines.append("}")
+
+    return "\n".join(lines)
+
+
+def generate_python_dataclass(name: str, schema: Dict[str, Any]) -> str:
+    """Generate Python dataclass from schema."""
+    properties = schema.get("properties") or {}
+    required = schema.get("required", [])
+
+    lines = [
+        "from dataclasses import dataclass",
+        "from typing import Optional",
+        "from datetime import datetime",
+        "",
+        "@dataclass",
+        f"class {name}:"
+    ]
+
+    if not properties:
+        lines.append("    pass")
+    else:
+        for prop_name, prop_def in properties.items():
+            prop_type = map_openapi_type_to_python(prop_def)
+            description = prop_def.get("description", "")
+
+            if prop_name not in required:
+                prop_type = f"Optional[{prop_type}]"
+
+            if description:
+                lines.append(f"    # {description}")
+
+            default = " = None" if prop_name not in required else ""
+            lines.append(f"    {prop_name}: {prop_type}{default}")
+
+    return "\n".join(lines)
+
+
+def map_openapi_type_to_typescript(openapi_type: str) -> str:
+    """Map OpenAPI type to TypeScript type."""
+    type_map = {
+        "string": "string",
+        "number": "number",
+        "integer": "number",
+        "boolean": "boolean",
+        "array": "any[]",
+        "object": "object"
+    }
+    return type_map.get(openapi_type, "any")
+
+
+def map_openapi_type_to_python(prop_def: Dict[str, Any]) -> str:
+    """Map OpenAPI type to Python type."""
+    openapi_type = prop_def.get("type", "Any")
+    format_type = prop_def.get("format", "")
+
+    if openapi_type == "string":
+        if format_type == "date-time":
+            return "datetime"
+        elif format_type == "uuid":
+            return "str"  # or UUID from uuid module
+        return "str"
+    elif openapi_type == "number" or openapi_type == "integer":
+        return "int" if openapi_type == "integer" else "float"
+    elif openapi_type == "boolean":
+        return "bool"
+    elif openapi_type == "array":
+        return "list"
+    elif openapi_type == "object":
+        return "dict"
+    return "Any"
+
+
+def generate_models(
+    spec_path: str,
+    language: str,
+    output_dir: str = "src/models",
+    package_name: str = None
+) -> Dict[str, Any]:
+    """
+    Generate models from API specification.
+
+    Args:
+        spec_path: Path to specification file
+        language: Target language
+        output_dir: Output directory
+        package_name: Package name
+
+    Returns:
+        Result dictionary with generated files info
+
+    Raises:
+        BettyError: If generation fails
+    """
+    # Validate language
+    if language not in SUPPORTED_LANGUAGES:
+        raise BettyError(
+            f"Unsupported language '{language}'. "
+            f"Supported: {', '.join(SUPPORTED_LANGUAGES)}"
+        )
+
+    # Validate spec file exists
+    if not Path(spec_path).exists():
+        raise BettyError(f"Specification file not found: {spec_path}")
+
+    logger.info(f"Generating {language} models from {spec_path}")
+
+    # Try datamodel-code-generator first (most reliable for OpenAPI)
+    try:
+        logger.info("Attempting generation with datamodel-code-generator")
+        result = generate_models_datamodel_code_generator(
+            spec_path, language, output_dir, package_name
+        )
+        return result
+    except BettyError as e:
+        logger.warning(f"datamodel-code-generator not available: {e}")
+
+    # Fallback to simple generation
+    logger.info("Using simple built-in generator")
+    result = generate_models_simple(
+        spec_path, language, output_dir, package_name
+    )
+
+    return result
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Generate type-safe models from API specifications using Modelina"
+    )
+    parser.add_argument(
+        "spec_path",
+        type=str,
+        help="Path to API specification file (OpenAPI or AsyncAPI)"
+    )
+    parser.add_argument(
+        "language",
+        type=str,
+        choices=SUPPORTED_LANGUAGES,
+        help="Target language for generated models"
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default="src/models",
+        help="Output directory for generated models (default: src/models)"
+    )
+    parser.add_argument(
+        "--package-name",
+        type=str,
+        help="Package/module name for generated code"
+    )
+
+    args = parser.parse_args()
+
+    try:
+        # Validate inputs
+        validate_path(args.spec_path)
+
+        # Generate models
+        result = generate_models(
+            spec_path=args.spec_path,
+            language=args.language,
+            output_dir=args.output_dir,
+            package_name=args.package_name
+        )
+
+        # Return structured result
+        output = {
+            "status": "success",
+            "data": result
+        }
+        print(json.dumps(output, indent=2))
+
+    except Exception as e:
+        logger.error(f"Model generation failed: {e}")
+        print(json.dumps(format_error_response(e), indent=2))
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()

skills/api.generatemodels/skill.yaml

@@ -0,0 +1,53 @@
+name: api.generatemodels
+version: 0.1.0
+description: Generate type-safe models from OpenAPI and AsyncAPI specifications using Modelina
+
+inputs:
+  - name: spec_path
+    type: string
+    required: true
+    description: Path to API specification file (OpenAPI or AsyncAPI)
+
+  - name: language
+    type: string
+    required: true
+    description: Target language (typescript, python, java, go, csharp)
+
+  - name: output_dir
+    type: string
+    required: false
+    default: src/models
+    description: Output directory for generated models
+
+  - name: package_name
+    type: string
+    required: false
+    description: Package/module name for generated code
+
+outputs:
+  - name: models_path
+    type: string
+    description: Path to directory containing generated models
+
+  - name: files_generated
+    type: array
+    description: List of generated model files
+
+  - name: model_count
+    type: number
+    description: Number of models generated
+
+dependencies:
+  - context.schema
+
+entrypoints:
+  - command: /skill/api/generate-models
+    handler: modelina_generate.py
+    runtime: python
+    permissions:
+      - filesystem:read
+      - filesystem:write
+
+status: active
+
+tags: [api, codegen, modelina, openapi, asyncapi, typescript, python, java]