# Telemetry Integration Examples

This document provides practical examples of integrating telemetry tracking into Betty Framework CLI entrypoints, workflows, and agents.

## Quick Start

The Betty Framework provides two main approaches for telemetry integration:

1. **Decorator Pattern** - For CLI entrypoints with standard main() functions
2. **Manual Capture** - For workflows, agents, and custom integrations

## 1. Decorator Pattern (Recommended for CLI)

### Standard CLI with Return Code

For CLI entrypoints that return an exit code:

```python
#!/usr/bin/env python3
import sys
import os
from typing import Optional, List

# Ensure project root on path
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../..")))

from betty.logging_utils import setup_logger
from betty.telemetry_integration import telemetry_tracked

logger = setup_logger(__name__)

@telemetry_tracked(skill_name="my.skill", caller="cli")
def main(argv: Optional[List[str]] = None) -> int:
    """Entry point for CLI execution."""
    argv = argv or sys.argv[1:]

    # Your skill logic here
    if not argv:
        logger.error("Missing required arguments")
        return 1

    # Process and return success
    return 0

if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))
```

**What the decorator does:**
- Automatically measures execution time
- Captures success/failure based on return code (0 = success, non-zero = failed)
- Logs telemetry with sanitized inputs
- Non-blocking - telemetry failures don't affect the main function

### Example: workflow.validate Integration

The `workflow.validate` skill has been updated with telemetry tracking:

```python
# skills/workflow.validate/workflow_validate.py
from betty.telemetry_integration import telemetry_tracked

@telemetry_tracked(skill_name="workflow.validate", caller="cli")
def main(argv: Optional[List[str]] = None) -> int:
    """Entry point for CLI execution."""
    # ... existing validation logic ...
    return 0 if response["ok"] else 1
```

**Test it:**
```bash
python3 skills/workflow.validate/workflow_validate.py example.yaml
cat registry/telemetry.json
```

## 2. Manual Capture Pattern

### Direct Function Call

For programmatic telemetry capture:

```python
from skills.telemetry.capture.telemetry_capture import create_telemetry_entry, capture_telemetry
import time

# Track execution manually
start_time = time.time()
try:
    # Your logic here
    result = execute_my_skill(param1, param2)
    status = "success"
except Exception as e:
    status = "failed"
    raise
finally:
    duration_ms = int((time.time() - start_time) * 1000)

    entry = create_telemetry_entry(
        skill_name="my.skill",
        inputs={"param1": param1, "param2": param2},
        status=status,
        duration_ms=duration_ms,
        caller="api"
    )
    capture_telemetry(entry)
```

### Workflow Execution Tracking

For capturing telemetry within workflow execution:

```python
# In workflow.compose or similar workflow executor
from betty.telemetry_integration import track_skill_execution

def execute_workflow_step(step_config: dict, workflow_name: str):
    """Execute a workflow step with telemetry tracking."""

    skill_name = step_config["skill"]
    skill_args = step_config["args"]

    result = track_skill_execution(
        skill_name=skill_name,
        func=lambda: run_skill(skill_name, skill_args),
        inputs={"args": skill_args},
        workflow=workflow_name,
        caller="workflow"
    )

    return result
```

### Agent Skill Invocation

For tracking skills invoked by agents:

```python
# In agent skill runner
from betty.telemetry_integration import track_skill_execution

def run_agent_skill(agent_name: str, skill_name: str, **kwargs):
    """Run a skill on behalf of an agent with telemetry."""

    result = track_skill_execution(
        skill_name=skill_name,
        func=lambda: execute_skill(skill_name, **kwargs),
        inputs=kwargs,
        agent=agent_name,
        caller="agent"
    )

    return result
```

## 3. CLI Helper Function

For simple CLI telemetry without decorators:

```python
from betty.telemetry_integration import capture_cli_telemetry
import time

def main():
    start_time = time.time()
    status = "failed"

    try:
        # Your CLI logic
        process_command()
        status = "success"
        return 0
    except Exception as e:
        logger.error(f"Error: {e}")
        return 1
    finally:
        duration_ms = int((time.time() - start_time) * 1000)
        capture_cli_telemetry(
            skill_name="my.skill",
            inputs={"cli_args": sys.argv[1:]},
            status=status,
            duration_ms=duration_ms,
            caller="cli"
        )
```

## 4. Context-Rich Telemetry

### Full Context Example

Capture comprehensive context for deep analytics:

```python
entry = create_telemetry_entry(
    skill_name="agent.define",
    inputs={
        "name": "my-agent",
        "mode": "iterative",
        "capabilities": ["code-gen", "testing"]
    },
    status="success",
    duration_ms=2500,
    agent="meta-orchestrator",           # Which agent invoked this
    workflow="agent-creation-pipeline",  # Which workflow this is part of
    caller="api",                        # How it was invoked

    # Custom fields for advanced analytics
    user_id="dev-123",
    environment="staging",
    version="1.0.0"
)

capture_telemetry(entry)
```

## 5. Batch Operations

For tracking multiple skill executions in batch:

```python
from skills.telemetry.capture.telemetry_capture import create_telemetry_entry, capture_telemetry
import time

def process_batch(items: list):
    """Process items in batch with per-item telemetry."""

    for item in items:
        start_time = time.time()

        try:
            process_item(item)
            status = "success"
        except Exception as e:
            status = "failed"

        duration_ms = int((time.time() - start_time) * 1000)

        # Capture telemetry for each item
        entry = create_telemetry_entry(
            skill_name="batch.processor",
            inputs={"item_id": item.id},
            status=status,
            duration_ms=duration_ms,
            caller="batch"
        )
        capture_telemetry(entry)
```

## 6. Error Handling Best Practices

### Graceful Telemetry Failures

Always wrap telemetry in try-except to prevent failures from affecting main logic:

```python
def my_critical_function():
    """Critical function that must not fail due to telemetry issues."""

    start_time = time.time()
    result = None

    try:
        # Critical business logic
        result = perform_critical_operation()
        status = "success"
    except Exception as e:
        status = "failed"
        raise  # Re-raise the original error
    finally:
        # Capture telemetry (failures are logged but don't interrupt)
        try:
            duration_ms = int((time.time() - start_time) * 1000)
            capture_cli_telemetry(
                skill_name="critical.operation",
                inputs={},
                status=status,
                duration_ms=duration_ms
            )
        except Exception as telemetry_error:
            logger.warning(f"Telemetry capture failed: {telemetry_error}")
            # Don't raise - telemetry is not critical

    return result
```

## 7. Sensitive Data Protection

### Sanitize Inputs

Never capture sensitive data like passwords, tokens, or PII:

```python
def sanitize_inputs(inputs: dict) -> dict:
    """Remove sensitive fields from inputs before logging."""

    sensitive_keys = ["password", "token", "api_key", "secret", "ssn", "credit_card"]

    sanitized = {}
    for key, value in inputs.items():
        if any(sensitive in key.lower() for sensitive in sensitive_keys):
            sanitized[key] = "***REDACTED***"
        else:
            sanitized[key] = value

    return sanitized

# Use it
entry = create_telemetry_entry(
    skill_name="auth.login",
    inputs=sanitize_inputs({"username": "john", "password": "secret123"}),
    status="success",
    duration_ms=150
)
```

## 8. Testing with Telemetry

### Unit Test Example

```python
import unittest
from unittest.mock import patch, MagicMock

class TestMySkillWithTelemetry(unittest.TestCase):

    @patch('betty.telemetry_integration.capture_cli_telemetry')
    def test_skill_execution_captures_telemetry(self, mock_capture):
        """Test that telemetry is captured on skill execution."""

        # Execute the skill
        from skills.my.skill.my_skill import main
        result = main(["arg1", "arg2"])

        # Verify telemetry was captured
        self.assertEqual(result, 0)
        mock_capture.assert_called_once()

        # Verify telemetry parameters
        call_args = mock_capture.call_args
        self.assertEqual(call_args.kwargs['skill_name'], "my.skill")
        self.assertEqual(call_args.kwargs['status'], "success")
        self.assertGreater(call_args.kwargs['duration_ms'], 0)
```

## 9. Migration Checklist

To add telemetry to an existing skill:

1. **Import the telemetry integration:**
   ```python
   from betty.telemetry_integration import telemetry_tracked
   ```

2. **Apply the decorator to main():**
   ```python
   @telemetry_tracked(skill_name="your.skill", caller="cli")
   def main(argv: Optional[List[str]] = None) -> int:
   ```

3. **Ensure main() returns an int:**
   - Return 0 for success
   - Return non-zero for failure

4. **Test the integration:**
   ```bash
   python3 skills/your.skill/your_skill.py test-args
   cat registry/telemetry.json | jq '.[-1]'  # View latest entry
   ```

## 10. Querying Telemetry Data

### Basic Queries

```bash
# Count total telemetry entries
jq 'length' registry/telemetry.json

# Find all failed executions
jq '.[] | select(.status == "failed")' registry/telemetry.json

# Get average duration for a skill
jq '[.[] | select(.skill == "workflow.validate") | .duration_ms] | add / length' registry/telemetry.json

# Top 10 slowest executions
jq 'sort_by(.duration_ms) | reverse | .[0:10]' registry/telemetry.json

# Executions by caller
jq 'group_by(.caller) | map({caller: .[0].caller, count: length})' registry/telemetry.json
```

### Advanced Analytics

```bash
# Skills executed in last hour
jq --arg cutoff "$(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S)" \
   '.[] | select(.timestamp > $cutoff)' registry/telemetry.json

# Success rate by skill
jq 'group_by(.skill) | map({
    skill: .[0].skill,
    total: length,
    successful: ([.[] | select(.status == "success")] | length),
    success_rate: (([.[] | select(.status == "success")] | length) / length * 100)
})' registry/telemetry.json
```

## 11. Future: Prometheus Export

Example of how telemetry data could be exported to Prometheus (future implementation):

```python
# Future: skills/telemetry.capture/exporters/prometheus.py
from prometheus_client import Counter, Histogram, Gauge

skill_executions = Counter(
    'betty_skill_executions_total',
    'Total skill executions',
    ['skill', 'status', 'caller']
)

skill_duration = Histogram(
    'betty_skill_duration_seconds',
    'Skill execution duration',
    ['skill', 'caller']
)

def export_to_prometheus(telemetry_entry: dict):
    """Export telemetry entry to Prometheus metrics."""
    skill_executions.labels(
        skill=telemetry_entry['skill'],
        status=telemetry_entry['status'],
        caller=telemetry_entry.get('caller', 'unknown')
    ).inc()

    skill_duration.labels(
        skill=telemetry_entry['skill'],
        caller=telemetry_entry.get('caller', 'unknown')
    ).observe(telemetry_entry['duration_ms'] / 1000.0)
```

## Summary

- **CLI Skills**: Use `@telemetry_tracked` decorator
- **Workflows**: Use `track_skill_execution()` helper
- **Custom Code**: Use `create_telemetry_entry()` + `capture_telemetry()`
- **Always**: Handle telemetry failures gracefully
- **Never**: Capture sensitive data in inputs

For more details, see the [SKILL.md](SKILL.md) documentation.