Initial commit

skills/test-orchestrator/SKILL.md

@@ -0,0 +1,52 @@
+---
+name: orchestrating-test-workflows
+description: |
+  This skill enables Claude to orchestrate complex test workflows using the test-orchestrator plugin. It allows Claude to define test execution graphs with dependencies, execute tests in parallel, and intelligently select tests to run based on code changes. Use this skill when a user requests test orchestration, dependency management for tests, parallel test execution, smart test selection, or CI/CD integration for testing. Trigger terms include "orchestrate tests", "test workflow", "parallel testing", "smart test selection", "test dependencies", and "/orchestrate".
+allowed-tools: Read, Bash, Grep, Glob
+version: 1.0.0
+---
+
+## Overview
+
+This skill empowers Claude to manage and execute complex test suites efficiently. It leverages the test-orchestrator plugin to handle test dependencies, parallel execution, and intelligent test selection, resulting in faster and more reliable testing processes.
+
+## How It Works
+
+1. **Workflow Definition**: Claude uses the plugin to define the test workflow, specifying dependencies between tests.
+2. **Parallelization**: The plugin identifies independent tests and executes them in parallel to reduce overall execution time.
+3. **Smart Selection**: Based on code changes, the plugin selects only the affected tests to run, minimizing unnecessary execution.
+
+## When to Use This Skill
+
+This skill activates when you need to:
+- Optimize test execution time.
+- Manage dependencies between tests.
+- Run only relevant tests after code changes.
+
+## Examples
+
+### Example 1: Optimizing Regression Testing
+
+User request: "Orchestrate the regression tests for the user authentication module after the recent code changes."
+
+The skill will:
+1. Use the test-orchestrator plugin to identify the tests affected by the changes in the user authentication module.
+2. Execute those tests in parallel, respecting any dependencies.
+
+### Example 2: Setting up a CI/CD Pipeline
+
+User request: "Set up a test workflow for the CI/CD pipeline that runs unit tests, integration tests, and end-to-end tests with appropriate dependencies."
+
+The skill will:
+1. Define a test workflow using the test-orchestrator plugin, specifying the order and dependencies between the different test suites (unit, integration, end-to-end).
+2. Configure the pipeline to trigger the orchestrated test execution upon code commits.
+
+## Best Practices
+
+- **Dependency Mapping**: Clearly define dependencies between tests to ensure correct execution order.
+- **Granularity**: Break down large test suites into smaller, more manageable units for better parallelization.
+- **Change Tracking**: Integrate the test-orchestrator with version control to accurately identify affected tests.
+
+## Integration
+
+This skill integrates with the test-orchestrator plugin and can be incorporated into CI/CD pipelines. It can also be used in conjunction with other code analysis tools to further refine test selection.

skills/test-orchestrator/assets/README.md

@@ -0,0 +1,7 @@
+# Assets
+
+Bundled resources for test-orchestrator skill
+
+- [ ] example_test_graph.json: Example test graph file demonstrating dependencies and test execution order.
+- [ ] test_report_template.html: HTML template for generating test reports.
+- [ ] configuration_template.yaml: Template for configuring the test orchestrator.

skills/test-orchestrator/assets/configuration_template.yaml

@@ -0,0 +1,116 @@
+# Configuration template for the Test Orchestrator plugin
+
+# Global settings for the test orchestration process
+global:
+  # Maximum number of parallel test executions
+  max_parallel_tests: 4  # Default: 4. Adjust based on available resources.
+
+  # Default timeout for individual test execution (in seconds)
+  default_test_timeout: 300 # Default: 300 seconds (5 minutes).  Use 0 for no timeout.
+
+  # Number of retry attempts for failed tests
+  retry_attempts: 2  # Default: 2. Set to 0 to disable retries.
+
+  # Base directory for test execution (relative to the repository root)
+  test_base_dir: "test" # Default: "test".  Change if your tests are in a different directory.
+
+  # Environment variables to apply to all test executions
+  environment:
+    # Example: API_KEY: "YOUR_API_KEY"
+    # REPLACE_ME : YOUR_VALUE_HERE
+
+# Definition of test suites and their dependencies
+test_suites:
+  # Example test suite definition
+  integration_tests:
+    # Description of the test suite
+    description: "Integration tests for the core application functionality"
+
+    # Test files to include in this suite (glob patterns)
+    include:
+      - "test/integration/**/*.test.js"
+      - "test/integration/**/*_spec.js"
+
+    # Test files to exclude from this suite (glob patterns)
+    exclude:
+      - "test/integration/deprecated/*.test.js" # Example: Exclude deprecated tests
+
+    # Dependencies: List of other test suites that must complete successfully before this one runs.
+    dependencies:
+      - unit_tests
+      # Example: - security_tests
+
+    # Environment variables specific to this test suite (overrides global environment)
+    environment:
+      # Example: DATABASE_URL: "YOUR_INTEGRATION_DATABASE_URL"
+      # REPLACE_ME : YOUR_VALUE_HERE
+
+  unit_tests:
+    description: "Unit tests for individual modules"
+    include:
+      - "test/unit/**/*.test.js"
+      - "test/unit/**/*_spec.js"
+    exclude: []
+    dependencies: []
+    environment: {}
+
+  # Add more test suites as needed
+  # Example:
+  # security_tests:
+  #   description: "Security vulnerability tests"
+  #   include:
+  #     - "test/security/**/*.test.js"
+  #   exclude: []
+  #   dependencies:
+  #     - integration_tests
+  #   environment:
+  #     SECURE_API_KEY: "YOUR_SECURE_API_KEY"
+
+# Smart test selection configuration (optional)
+smart_selection:
+  # Enable or disable smart test selection
+  enabled: false # Default: false. Set to true to enable.
+
+  # Git branch to compare against for changed files (e.g., "main", "develop")
+  base_branch: "main" # Default: "main".  The branch to compare against for changes.
+
+  # Regular expressions to identify source code files.
+  source_code_patterns:
+    - "src/**/*.js"
+    - "lib/**/*.py"
+    # Add more file patterns as needed
+
+  # Mapping of source code files to associated test files.
+  # Each key is a glob pattern matching source code files.
+  # Each value is a list of glob patterns matching associated test files.
+  source_to_test_mapping:
+    "src/auth/*.js":
+      - "test/unit/auth/*.test.js"
+      - "test/integration/auth/*.test.js"
+    # Add more mappings as needed
+
+# Reporting configuration
+reporting:
+  # Type of report to generate (e.g., "junit", "html")
+  report_type: "junit" # Default: "junit".  Available options depend on the reporting plugins.
+
+  # Output file for the test report
+  report_output_file: "test-results.xml" # Default: "test-results.xml".
+
+  # Additional reporting options (plugin-specific)
+  options:
+    # Example: include_console_output: true
+    # REPLACE_ME : YOUR_VALUE_HERE
+
+# CI/CD integration configuration (optional)
+ci_cd:
+  # Enable or disable CI/CD integration
+  enabled: false # Default: false. Set to true to enable.
+
+  # Type of CI/CD system (e.g., "github_actions", "gitlab_ci")
+  system_type: "github_actions" # Default: "github_actions".
+
+  # Additional CI/CD options (system-specific)
+  options:
+    # Example: upload_results_to_s3: true
+    # REPLACE_ME : YOUR_VALUE_HERE

skills/test-orchestrator/assets/example_test_graph.json

@@ -0,0 +1,86 @@
+{
+  "_comment": "Example test graph file for the Test Orchestrator plugin.",
+  "name": "Integration Test Suite",
+  "description": "Comprehensive integration tests covering various system components.",
+  "tests": [
+    {
+      "id": "test_db_connection",
+      "name": "Database Connection Test",
+      "description": "Verifies connectivity to the database.",
+      "command": "python tests/integration/test_db.py",
+      "dependencies": [],
+      "priority": "high",
+      "retry_attempts": 2,
+      "timeout": 60,
+       "tags": ["database", "connectivity"]
+    },
+    {
+      "id": "test_api_auth",
+      "name": "API Authentication Test",
+      "description": "Tests API authentication and authorization.",
+      "command": "python tests/integration/test_api_auth.py",
+      "dependencies": ["test_db_connection"],
+      "priority": "high",
+      "retry_attempts": 1,
+      "timeout": 45,
+      "tags": ["api", "authentication", "authorization"]
+    },
+    {
+      "id": "test_user_registration",
+      "name": "User Registration Test",
+      "description": "Tests user registration functionality.",
+      "command": "python tests/integration/test_user_registration.py",
+      "dependencies": ["test_db_connection"],
+      "priority": "medium",
+      "retry_attempts": 0,
+      "timeout": 90,
+      "tags": ["user", "registration"]
+    },
+    {
+      "id": "test_payment_processing",
+      "name": "Payment Processing Test",
+      "description": "Tests payment processing workflow.",
+      "command": "python tests/integration/test_payment.py",
+      "dependencies": ["test_api_auth", "test_user_registration"],
+      "priority": "high",
+      "retry_attempts": 3,
+      "timeout": 120,
+      "tags": ["payment", "api"]
+    },
+    {
+      "id": "test_report_generation",
+      "name": "Report Generation Test",
+      "description": "Tests report generation functionality.",
+      "command": "python tests/integration/test_report.py",
+      "dependencies": ["test_db_connection"],
+      "priority": "low",
+      "retry_attempts": 1,
+      "timeout": 180,
+      "tags": ["report", "database"]
+    },
+    {
+      "id": "test_data_migration",
+      "name": "Data Migration Test",
+      "description": "Tests the data migration process.",
+      "command": "python tests/integration/test_migration.py",
+      "dependencies": [],
+      "priority": "medium",
+      "retry_attempts": 0,
+      "timeout": 300,
+      "tags": ["migration", "database"]
+    }
+  ],
+  "configuration": {
+    "_comment": "Global configuration options for the test suite.",
+    "max_parallel_tests": 4,
+    "default_retry_attempts": 1,
+    "default_timeout": 60,
+    "test_directory": "tests/integration",
+    "report_format": "json"
+  },
+  "smart_selection": {
+    "_comment": "Configuration for smart test selection based on code changes.",
+    "enabled": true,
+    "mapping_file": "test_mapping.json"
+  }
+}

skills/test-orchestrator/assets/test_report_template.html

@@ -0,0 +1,138 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Test Orchestrator Report</title>
+    <style>
+        /* Inline CSS for styling */
+        body {
+            font-family: sans-serif;
+            margin: 0;
+            padding: 0;
+            background-color: #f4f4f4;
+            color: #333;
+        }
+
+        .container {
+            width: 90%;
+            max-width: 1200px;
+            margin: 20px auto;
+            background-color: #fff;
+            padding: 20px;
+            border-radius: 8px;
+            box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
+        }
+
+        h1 {
+            text-align: center;
+            color: #3498db;
+        }
+
+        h2 {
+            margin-top: 20px;
+            color: #2c3e50;
+        }
+
+        table {
+            width: 100%;
+            border-collapse: collapse;
+            margin-top: 10px;
+        }
+
+        th, td {
+            padding: 12px 15px;
+            text-align: left;
+            border-bottom: 1px solid #ddd;
+        }
+
+        th {
+            background-color: #3498db;
+            color: white;
+        }
+
+        tr:nth-child(even) {
+            background-color: #f2f2f2;
+        }
+
+        .status-passed {
+            color: green;
+        }
+
+        .status-failed {
+            color: red;
+        }
+
+        .status-skipped {
+            color: orange;
+        }
+
+        .summary {
+            margin-top: 30px;
+            padding: 15px;
+            background-color: #eaf2ff;
+            border-left: 5px solid #3498db;
+            border-radius: 4px;
+        }
+
+        .summary p {
+            margin: 5px 0;
+        }
+
+        /* Responsive Design */
+        @media (max-width: 768px) {
+            .container {
+                width: 95%;
+                padding: 10px;
+            }
+
+            th, td {
+                padding: 8px 10px;
+                font-size: 0.9em;
+            }
+        }
+    </style>
+</head>
+<body>
+    <div class="container">
+        <!-- Main Heading -->
+        <h1>Test Orchestrator Report</h1>
+
+        <!-- Report Summary -->
+        <div class="summary">
+            <h2>Summary</h2>
+            <p><strong>Total Tests:</strong> {{total_tests}}</p>
+            <p><strong>Passed:</strong> <span class="status-passed">{{passed_tests}}</span></p>
+            <p><strong>Failed:</strong> <span class="status-failed">{{failed_tests}}</span></p>
+            <p><strong>Skipped:</strong> <span class="status-skipped">{{skipped_tests}}</span></p>
+            <p><strong>Pass Rate:</strong> {{pass_rate}}%</p>
+            <p><strong>Start Time:</strong> {{start_time}}</p>
+            <p><strong>End Time:</strong> {{end_time}}</p>
+            <p><strong>Duration:</strong> {{duration}}</p>
+        </div>
+
+        <!-- Test Results Table -->
+        <h2>Test Results</h2>
+        <table>
+            <thead>
+                <tr>
+                    <th>Test Name</th>
+                    <th>Status</th>
+                    <th>Duration</th>
+                    <th>Error Message</th>
+                </tr>
+            </thead>
+            <tbody>
+                {{test_results}}
+            </tbody>
+        </table>
+
+        <!-- Additional Information -->
+        <h2>Additional Information</h2>
+        <p>{{additional_info}}</p>
+
+        <!-- Footer -->
+        <p style="text-align: center; margin-top: 20px;">Generated by Test Orchestrator</p>
+    </div>
+</body>
+</html>

skills/test-orchestrator/references/README.md

@@ -0,0 +1,7 @@
+# References
+
+Bundled resources for test-orchestrator skill
+
+- [ ] test_graph_schema.md: Documentation of the test graph schema used to define test dependencies.
+- [ ] test_mapping.md: Documentation on how tests are mapped to code changes for smart test selection.
+- [ ] ci_cd_integration.md: Best practices for integrating the test orchestrator into CI/CD pipelines.

skills/test-orchestrator/scripts/README.md

@@ -0,0 +1,7 @@
+# Scripts
+
+Bundled resources for test-orchestrator skill
+
+- [ ] test_selector.py: Script to intelligently select tests to run based on code changes, using git diff and test mapping.
+- [ ] test_executor.py: Script to execute tests in parallel, managing dependencies and retrying flaky tests.
+- [ ] report_generator.py: Script to generate test reports in various formats (JUnit, HTML).