gh-epieczko-betty/skills/hook.simulate/SKILL.md

hook.simulate

Version: 0.1.0 Status: Active Tags: hook, simulation, testing, validation, development

Overview

The hook.simulate skill allows developers to test a hook manifest before registering it in the Betty Framework. It validates the manifest structure, simulates hook triggers based on event types, and optionally executes the hook commands in a controlled environment.

This skill is essential for:

• Testing hooks before deployment - Catch errors early in development
• Understanding hook behavior - See which files match patterns and how commands execute
• Debugging hook issues - Validate patterns, commands, and execution flow
• Safe experimentation - Test hooks without affecting the live system

Features

• ✅ Manifest Validation - Validates all required and optional fields
• ✅ Pattern Matching - Shows which files match the hook's pattern
• ✅ Event Simulation - Simulates different hook events (on_file_edit, on_commit, etc.)
• ✅ Command Execution - Runs hook commands with dry-run or actual execution
• ✅ Timeout Testing - Validates timeout behavior
• ✅ Blocking Simulation - Shows how blocking hooks would behave
• ✅ JSON Output - Structured results for automation and analysis

Usage

Command Line

# Basic validation and simulation
python skills/hook.simulate/hook_simulate.py examples/test-hook.yaml

# Simulate with dry-run command execution
python skills/hook.simulate/hook_simulate.py examples/test-hook.yaml --execute --dry-run

# Actually execute the command
python skills/hook.simulate/hook_simulate.py examples/test-hook.yaml --execute --no-dry-run

# Get JSON output for scripting
python skills/hook.simulate/hook_simulate.py examples/test-hook.yaml --output json

As a Skill

from skills.hook.simulate.hook_simulate import simulate_hook

# Simulate a hook
results = simulate_hook(
    manifest_path="examples/test-hook.yaml",
    dry_run=True,
    execute=True
)

# Check validation
if results["valid"]:
    print("✅ Hook is valid")
else:
    print("❌ Validation errors:", results["validation_errors"])

# Check if hook would trigger
if results["trigger_simulation"]["would_trigger"]:
    print("Hook would trigger!")
    print("Matching files:", results["trigger_simulation"]["matching_files"])

Input Parameters

Parameter	Type	Required	Default	Description
manifest_path	string	Yes	-	Path to the hook.yaml manifest file
execute	boolean	No	false	Whether to execute the hook command
dry_run	boolean	No	true	If true, simulate without actually running commands

Output Schema

{
  "manifest_path": "path/to/hook.yaml",
  "timestamp": "2025-10-23 12:34:56",
  "valid": true,
  "validation_errors": [],
  "manifest": {
    "name": "hook-name",
    "version": "0.1.0",
    "description": "Hook description",
    "event": "on_file_edit",
    "command": "python script.py {file_path}",
    "when": {
      "pattern": "*.yaml"
    },
    "blocking": true,
    "timeout": 30000
  },
  "trigger_simulation": {
    "would_trigger": true,
    "reason": "Found 5 file(s) matching pattern: *.yaml",
    "matching_files": ["file1.yaml", "file2.yaml"],
    "pattern": "*.yaml"
  },
  "command_executions": [
    {
      "command": "python script.py file1.yaml",
      "executed": true,
      "dry_run": false,
      "stdout": "Validation passed",
      "stderr": "",
      "return_code": 0,
      "execution_time_ms": 123.45,
      "success": true,
      "file": "file1.yaml"
    }
  ],
  "blocking": true,
  "timeout_ms": 30000
}

Examples

Example 1: Validate OpenAPI Hook

Create a hook manifest openapi-validator.yaml:

name: validate-openapi
version: 0.1.0
description: "Validate OpenAPI specifications on edit"

event: on_file_edit

command: "python betty/skills/api.validate/api_validate.py {file_path} zalando"

when:
  pattern: "**/*.openapi.yaml"

blocking: true
timeout: 30000
on_failure: show_errors

status: draft
tags: [api, validation, openapi]

Simulate it:

python skills/hook.simulate/hook_simulate.py openapi-validator.yaml

Output:

=== Hook Simulation Results ===
Manifest: openapi-validator.yaml
Timestamp: 2025-10-23 12:34:56

✅ VALIDATION PASSED

Hook: validate-openapi v0.1.0
Event: on_file_edit
Command: python betty/skills/api.validate/api_validate.py {file_path} zalando
Blocking: True
Timeout: 30000ms

✅ WOULD TRIGGER
Reason: Found 3 file(s) matching pattern: **/*.openapi.yaml

Matching files (3):
  - specs/petstore.openapi.yaml
  - specs/users.openapi.yaml
  - api/products.openapi.yaml

Example 2: Test Commit Hook

Create pre-commit.yaml:

name: pre-commit-linter
version: 1.0.0
description: "Run linter before commits"

event: on_commit

command: "pylint src/"

blocking: true
timeout: 60000

status: active
tags: [lint, quality]

Simulate with execution:

python skills/hook.simulate/hook_simulate.py pre-commit.yaml --execute --dry-run

Output:

=== Hook Simulation Results ===
Manifest: pre-commit.yaml
Timestamp: 2025-10-23 12:35:00

✅ VALIDATION PASSED

Hook: pre-commit-linter v1.0.0
Event: on_commit
Command: pylint src/
Blocking: True
Timeout: 60000ms

✅ WOULD TRIGGER
Reason: on_commit hook would trigger with 5 changed file(s)

Changed files (5):
  - src/main.py
  - src/utils.py
  - tests/test_main.py
  - README.md
  - setup.py

=== Command Execution Results (1) ===

[1] N/A
Command: pylint src/
Mode: DRY RUN (not executed)

Example 3: Test with Actual Execution

python skills/hook.simulate/hook_simulate.py openapi-validator.yaml --execute --no-dry-run

Output:

=== Hook Simulation Results ===
...

=== Command Execution Results (3) ===

[1] specs/petstore.openapi.yaml
Command: python betty/skills/api.validate/api_validate.py specs/petstore.openapi.yaml zalando
Executed: Yes
Return code: 0
Execution time: 234.56ms
Status: ✅ SUCCESS

Stdout:
✅ OpenAPI spec is valid

[2] specs/users.openapi.yaml
Command: python betty/skills/api.validate/api_validate.py specs/users.openapi.yaml zalando
Executed: Yes
Return code: 1
Execution time: 189.23ms
Status: ❌ FAILED

Stderr:
Error: Missing required field 'info.contact'

[3] api/products.openapi.yaml
Command: python betty/skills/api.validate/api_validate.py api/products.openapi.yaml zalando
Executed: Yes
Return code: 0
Execution time: 201.45ms
Status: ✅ SUCCESS

Example 4: JSON Output for Automation

python skills/hook.simulate/hook_simulate.py my-hook.yaml --output json > results.json

Then process with scripts:

import json

with open("results.json") as f:
    results = json.load(f)

if not results["valid"]:
    print("Validation failed:")
    for error in results["validation_errors"]:
        print(f"  - {error}")
    exit(1)

if results["trigger_simulation"]["would_trigger"]:
    files = results["trigger_simulation"]["matching_files"]
    print(f"Hook would run on {len(files)} files")

for execution in results["command_executions"]:
    if not execution["success"]:
        print(f"Failed on {execution['file']}")
        print(f"Error: {execution['stderr']}")

Event Type Support

Event	Simulation Support	Description
on_file_edit	✅ Full	Matches files by pattern, simulates editing
on_file_save	✅ Full	Similar to on_file_edit
on_commit	✅ Full	Checks git status, shows changed files
on_push	⚠️ Partial	Notes hook would trigger, no git simulation
on_tool_use	⚠️ Partial	Notes hook would trigger, no tool simulation
on_agent_start	⚠️ Partial	Notes hook would trigger, no agent simulation
on_workflow_end	⚠️ Partial	Notes hook would trigger, no workflow simulation

Validation Rules

The skill validates all Betty Framework hook requirements:

1. Required Fields: name, version, description, event, command
2. Name Format: Lowercase, starts with letter, allows hyphens/underscores
3. Version Format: Semantic versioning (e.g., 1.0.0, 0.1.0-alpha)
4. Event: Must be a valid hook event
5. Command: Non-empty string
6. Blocking: Must be boolean if specified
7. Timeout: Must be positive number in milliseconds if specified
8. Status: Must be draft, active, disabled, or archived if specified
9. Pattern: Must be non-empty string if specified in when.pattern

Pattern Matching

The skill supports glob patterns for file matching:

Pattern	Description	Example Matches
*.yaml	Files in current directory	config.yaml, spec.yaml
**/*.yaml	YAML files anywhere	src/config.yaml, specs/api/v1.yaml
src/**/*.py	Python files in src/	src/main.py, src/utils/helper.py
*.openapi.yaml	OpenAPI files only	petstore.openapi.yaml
tests/**/*.test.js	Test files	tests/unit/api.test.js

Dry-Run vs Execution Modes

Dry-Run Mode (Default)

python skills/hook.simulate/hook_simulate.py hook.yaml --execute --dry-run

• ✅ Validates manifest
• ✅ Shows matching files
• ✅ Shows command that would run
• ❌ Does NOT execute command
• ⚡ Safe for testing

Execution Mode

python skills/hook.simulate/hook_simulate.py hook.yaml --execute --no-dry-run

• ✅ Validates manifest
• ✅ Shows matching files
• ✅ Actually executes command
• ✅ Captures stdout/stderr
• ✅ Reports execution time
• ⚠️ Use with caution

Validation Only (No Execution)

python skills/hook.simulate/hook_simulate.py hook.yaml

• ✅ Validates manifest
• ✅ Shows if hook would trigger
• ✅ Shows matching files
• ❌ Does NOT execute command
• ⚡ Fastest mode

Integration with Hook Workflow

Development Workflow

# 1. Create hook manifest
cat > my-hook.yaml <<EOF
name: my-custom-hook
version: 0.1.0
description: "My hook description"
event: on_file_edit
command: "echo 'Hook triggered on {file_path}'"
when:
  pattern: "*.md"
EOF

# 2. Simulate and validate
python skills/hook.simulate/hook_simulate.py my-hook.yaml

# 3. Test with execution
python skills/hook.simulate/hook_simulate.py my-hook.yaml --execute --dry-run

# 4. Test actual execution
python skills/hook.simulate/hook_simulate.py my-hook.yaml --execute --no-dry-run

# 5. Register if tests pass
python skills/hook.register/hook_register.py my-hook.yaml

CI/CD Integration

# .github/workflows/test-hooks.yml
name: Test Hooks
on: [push, pull_request]

jobs:
  test-hooks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Validate all hook manifests
        run: |
          for hook in hooks/*.yaml; do
            echo "Testing $hook"
            python skills/hook.simulate/hook_simulate.py "$hook" --output json || exit 1
          done

      - name: Test hook execution
        run: |
          python skills/hook.simulate/hook_simulate.py hooks/validator.yaml --execute --no-dry-run

Error Handling

Validation Errors

$ python skills/hook.simulate/hook_simulate.py invalid-hook.yaml

=== Hook Simulation Results ===
❌ VALIDATION FAILED

Errors:
  - Missing required field: 'version'
  - Invalid event: 'on_invalid_event'. Must be one of: on_file_edit, on_file_save, on_commit, on_push, on_tool_use, on_agent_start, on_workflow_end
  - timeout must be a positive number (in milliseconds)

Execution Failures

$ python skills/hook.simulate/hook_simulate.py hook.yaml --execute --no-dry-run

...

=== Command Execution Results (1) ===

[1] test.yaml
Command: python nonexistent.py test.yaml
Executed: Yes
Return code: -1
Execution time: 45.23ms
Status: ❌ FAILED

Stderr:
Error executing command: [Errno 2] No such file or directory: 'python nonexistent.py test.yaml'

Timeout Handling

$ python skills/hook.simulate/hook_simulate.py slow-hook.yaml --execute --no-dry-run

...

[1] test.yaml
Command: sleep 100
Executed: Yes
Return code: -1
Execution time: 5000.12ms
Status: ❌ FAILED

Stderr:
Command timed out after 5000ms

Troubleshooting

Hook Doesn't Trigger

Problem: would_trigger: false

Solutions:

1. Check pattern matches files: ls -la **/*.yaml
2. Verify event type is appropriate
3. Ensure files exist in the manifest's directory
4. Check git status for on_commit events

Command Fails

Problem: Command returns non-zero exit code

Solutions:

1. Test command manually: python script.py file.yaml
2. Check file paths and placeholders
3. Verify dependencies are installed
4. Check timeout is sufficient

Pattern Not Matching

Problem: Expected files not showing in matching_files

Solutions:

1. Use **/*.ext for recursive matching
2. Check you're in the right directory
3. Test pattern with: ls **/*.yaml
4. Ensure pattern is in when.pattern field

Best Practices

1. Always simulate before registering - Catch issues early
2. Test with dry-run first - Understand what will execute
3. Verify pattern matching - Ensure correct files are targeted
4. Test timeout values - Make sure commands complete in time
5. Use JSON output for automation - Integrate with CI/CD
6. Test blocking behavior - Understand impact on workflow
7. Validate on representative data - Test with real files

Related Skills

• hook.define - Create hooks dynamically and add to .claude/hooks.yaml
• hook.register - Register validated hooks in the registry
• api.validate - Example validation skill often used in hooks

Version History

• 0.1.0 (2025-10-23) - Initial release
  • Manifest validation
  • Event simulation (on_file_edit, on_commit)
  • Pattern matching
  • Command execution (dry-run and actual)
  • JSON and summary output formats

License

MIT License - Part of the Betty Framework

Contributing

To improve this skill:

1. Add support for more event types
2. Enhance simulation accuracy
3. Add more validation rules
4. Improve error messages
5. Add performance benchmarks