zhongwei/gh-jeremylongshore-claude-code-plugins-plus-plugins-testing-regression-test-tracker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-30 08:23:23 +08:00
commit c959b5bb31
11 changed files with 594 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
.claude-plugin/plugin.json Normal file
View File

@@ -0,0 +1,15 @@
{
"name": "regression-test-tracker",
"description": "Track and run regression tests to ensure new changes don't break existing functionality",
"version": "1.0.0",
"author": {
"name": "Claude Code Plugins",
"email": "[email protected]"
},
"skills": [
"./skills"
],
"commands": [
"./commands"
]
}

3
README.md Normal file
View File

@@ -0,0 +1,3 @@
# regression-test-tracker
Track and run regression tests to ensure new changes don't break existing functionality

82
commands/track-regression.md Normal file
View File

@@ -0,0 +1,82 @@
---
description: Track and run regression tests for existing functionality
shortcut: reg
---
# Regression Test Tracker
Track critical tests and ensure new changes don't break existing functionality.
## Purpose
Maintain stability by:
- **Tracking critical tests** - Mark tests for regression suite
- **Automated execution** - Run before deployments
- **Change impact analysis** - Which tests affected by changes
- **Test history** - Track pass/fail over time
- **Flaky test detection** - Identify unreliable tests
## Usage
```bash
/track-regression # Run regression suite
/track-regression --mark # Mark current test as regression
/track-regression --history # Show test history
/reg # Shortcut
```
## Regression Suite Management
Tag tests as regression tests:
```javascript
// Jest
describe('User Login', () => {
it('[REGRESSION] should login with valid credentials', async () => {
// Critical login test
});
});
// pytest
@pytest.mark.regression
def test_payment_processing():
# Critical payment test
pass
```
## Report Format
```
Regression Test Report
======================
Date: 2025-10-11
Suite: Full Regression
Tests: 45 critical tests
Duration: 3m 42s
Results: 44/45 passed (97.8%)
FAILED:
test_checkout_with_coupon
Last passed: 2025-10-09
Failures: 2 consecutive
Introduced by: commit abc123f
Flaky Tests Detected:
️ test_email_delivery
Pass rate: 85% (17/20 recent runs)
Recommendation: Investigate timing issues
Impact Analysis:
Changed files: src/api/orders.js
Potentially affected: 8 tests
Recommended: Run full order test suite
```
## Best Practices
- Tag critical business flows
- Run before every deployment
- Keep regression suite fast (<5 min)
- Monitor for flaky tests
- Update when requirements change

73
plugin.lock.json Normal file
View File

@@ -0,0 +1,73 @@
{
"$schema": "internal://schemas/plugin.lock.v1.json",
"pluginId": "gh:jeremylongshore/claude-code-plugins-plus:plugins/testing/regression-test-tracker",
"normalized": {
"repo": null,
"ref": "refs/tags/v20251128.0",
"commit": "9a611b866881ca1e3ad690fc867e75503314aec5",
"treeHash": "7cbd3a9da4188caa35c6d663d957a62401a783a30feb71079fb455cd72da7d84",
"generatedAt": "2025-11-28T10:18:41.642718Z",
"toolVersion": "publish_plugins.py@0.2.0"
},
"origin": {
"remote": "git@github.com:zhongweili/42plugin-data.git",
"branch": "master",
"commit": "aa1497ed0949fd50e99e70d6324a29c5b34f9390",
"repoRoot": "/Users/zhongweili/projects/openmind/42plugin-data"
},
"manifest": {
"name": "regression-test-tracker",
"description": "Track and run regression tests to ensure new changes don't break existing functionality",
"version": "1.0.0"
},
"content": {
"files": [
{
"path": "README.md",
"sha256": "7d8e05c32ceccba8f587db15e4bda42cee278df5415a55857d16d458aeb474cc"
},
{
"path": ".claude-plugin/plugin.json",
"sha256": "5634e509a241e0e4d080f403022868c56b6e58d52bdc639cadf44c8f5d8f41be"
},
{
"path": "commands/track-regression.md",
"sha256": "ae9b07a0b2860d3c8c9055b928ec2b752384a53ec83719576a44f2f934297446"
},
{
"path": "skills/regression-test-tracker/SKILL.md",
"sha256": "d9594ee33836f275720178b5d138ba56b37993fe93bc204bac86380902a9a81f"
},
{
"path": "skills/regression-test-tracker/references/README.md",
"sha256": "d7193e08b0fcb8cff7187f04459b11456c3f2f1920f383d97809feb89e329934"
},
{
"path": "skills/regression-test-tracker/scripts/README.md",
"sha256": "afe44eadab363535ce604e34be06bb7508c8b9325efc4eaff8b664810de0443f"
},
{
"path": "skills/regression-test-tracker/assets/test_report_template.md",
"sha256": "33eb56a87e987ceab163acb44231a484c26010259bcf2dcfc37ff3c4f4978241"
},
{
"path": "skills/regression-test-tracker/assets/README.md",
"sha256": "1cdb75912b3f7a008904d6fe439cd62d58d8ed5ac10e9dc1e50e815e7e64cea1"
},
{
"path": "skills/regression-test-tracker/assets/example_test_history.json",
"sha256": "1420fd6d1c2662675383a36ee3fdfb2cf9d0b86cdc9e798df923e24ec39384cd"
},
{
"path": "skills/regression-test-tracker/assets/config_template.yaml",
"sha256": "a9161ab63653954db65d15336c5ed60e52ca5de01c988725068e1db49de570d6"
}
],
"dirSha256": "7cbd3a9da4188caa35c6d663d957a62401a783a30feb71079fb455cd72da7d84"
},
"security": {
"scannedAt": null,
"scannerVersion": null,
"flags": []
}
}

52
skills/regression-test-tracker/SKILL.md Normal file
View File

@@ -0,0 +1,52 @@
---
name: tracking-regression-tests
description: |
This skill enables Claude to track and run regression tests, ensuring new changes don't break existing functionality. It is triggered when the user asks to "track regression", "run regression tests", or uses the shortcut "reg". The skill helps in maintaining code stability by identifying critical tests, automating their execution, and analyzing the impact of changes. It also provides insights into test history and identifies flaky tests. The skill uses the `regression-test-tracker` plugin.
allowed-tools: Read, Bash, Grep, Glob
version: 1.0.0
---
## Overview
This skill allows Claude to track and execute regression tests, which are crucial for maintaining software quality and preventing unintended consequences from new code changes. By automating the regression testing process, Claude can quickly identify and address potential issues before they impact users.
## How It Works
1. **Identify Regression Tests**: The user marks specific tests as part of the regression suite using the `--mark` flag.
2. **Execute Regression Suite**: The skill runs the designated regression tests.
3. **Analyze Results**: The skill analyzes the test results, highlighting failures and potential flaky tests.
## When to Use This Skill
This skill activates when you need to:
- Run the regression test suite before deploying new code.
- Mark a specific test as part of the regression suite.
- Investigate potential regressions after making code changes.
## Examples
### Example 1: Running the Regression Suite
User request: "run regression tests"
The skill will:
1. Execute all tests marked as part of the regression suite.
2. Report the results, including any failures or flaky tests.
### Example 2: Marking a Test for Regression
User request: "track regression --mark test_example"
The skill will:
1. Mark the test `test_example` as part of the regression suite.
2. Confirm that the test has been added to the suite.
## Best Practices
- **Test Selection**: Choose tests that cover critical functionality and are likely to be affected by changes.
- **Frequency**: Run the regression suite frequently, especially before deployments.
- **Analysis**: Carefully analyze test failures to identify the root cause of regressions.
## Integration
This skill can be integrated with other testing and CI/CD tools to automate the regression testing process as part of a larger development workflow.

7
skills/regression-test-tracker/assets/README.md Normal file
View File

@@ -0,0 +1,7 @@
# Assets
Bundled resources for regression-test-tracker skill
- [ ] test_report_template.md: Markdown template for generating regression test reports.
- [ ] example_test_history.json: Example JSON file showing the format of test history data.
- [ ] config_template.yaml: Template for configuring the regression test tracker.

67
skills/regression-test-tracker/assets/config_template.yaml Normal file
View File

@@ -0,0 +1,67 @@
# Regression Test Tracker Plugin Configuration
# Database settings (for storing test results and history)
database:
type: "sqlite" # Options: sqlite, postgresql, mysql, etc.
connection_string: "regression_tests.db" # REPLACE_ME: Database connection string
# Example PostgreSQL connection string:
# connection_string: "postgresql://user:password@host:port/database"
# Test execution settings
test_execution:
runner: "pytest" # REPLACE_ME: Test runner command (e.g., pytest, unittest)
test_directory: "tests" # Directory containing test files
pattern: "test_*.py" # Pattern to identify test files
concurrency: 4 # Number of parallel test processes
timeout: 60 # Timeout in seconds for each test
retry_attempts: 2 # Number of retry attempts for failed tests
report_format: "xml" # Test report format (e.g., xml, json, html)
report_file: "test_report.xml" # Path to the generated test report
# Regression test suite definition
regression_suite:
tags: # List of tags to identify regression tests
- "regression"
- "critical"
# Optionally, specify specific test files or directories to include/exclude
include_paths: # Paths to include in the regression test suite. Leave empty to include all tests tagged above.
- "tests/important_feature_tests"
exclude_paths: # Paths to exclude from the regression test suite.
- "tests/experimental_tests"
# Change impact analysis settings
change_impact:
vcs: "git" # Version control system (e.g., git, mercurial)
# Command to get list of changed files since last regression test run.
# REPLACE_ME: You might need to customize this based on your workflow.
changed_files_command: "git diff --name-only HEAD^ HEAD"
mapping: # Mapping between files and affected tests
"src/feature_a.py": # Example: if feature_a.py is changed, run these tests
- "tests/test_feature_a.py::test_function_1"
- "tests/test_integration.py::test_feature_a_integration"
"src/common_utils.py": # Example: If a common utility file changes, run a broad suite of tests
- "tests/test_all.py"
# Flaky test detection settings
flaky_test_detection:
enabled: true # Enable flaky test detection
failure_threshold: 3 # Number of failures within a window to consider a test flaky
window_size: 10 # Number of test runs to consider for flaky test detection
# Notification settings (e.g., Slack, email)
notifications:
enabled: false # Enable notifications
type: "slack" # Notification type (e.g., slack, email)
webhook_url: "YOUR_VALUE_HERE" # REPLACE_ME: Slack webhook URL
# OR email settings
# email:
# sender: "test_tracker@example.com"
# recipient: "developers@example.com"
# smtp_server: "smtp.example.com"
# smtp_port: 587
# smtp_username: "YOUR_VALUE_HERE"
# smtp_password: "YOUR_VALUE_HERE"
# Advanced settings
advanced:
debug_mode: false # Enable debug mode for verbose logging

158
skills/regression-test-tracker/assets/example_test_history.json Normal file
View File

@@ -0,0 +1,158 @@
{
"_comment": "Example test history data for the Regression Test Tracker plugin",
"test_suite_name": "Production Deployment Tests",
"test_history": [
{
"run_id": "run-2024-10-26-001",
"_comment": "Unique identifier for this test run",
"timestamp": "2024-10-26T10:00:00Z",
"_comment": "ISO 8601 timestamp of the test run",
"environment": "Production",
"_comment": "Environment where the tests were executed",
"commit_hash": "a1b2c3d4e5f678901234567890abcdef01234567",
"_comment": "Git commit hash of the code being tested",
"tests": [
{
"test_name": "Verify user login",
"result": "passed",
"duration_ms": 1200,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Check product listing page",
"result": "passed",
"duration_ms": 850,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Add item to cart",
"result": "failed",
"duration_ms": 2100,
"error_message": "Cart service unavailable",
"is_flaky": false
},
{
"test_name": "Process checkout",
"result": "passed",
"duration_ms": 5500,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Verify email sending",
"result": "failed",
"duration_ms": 300,
"error_message": "Timeout waiting for SMTP response",
"is_flaky": true,
"_comment": "Example of a flaky test"
}
],
"summary": {
"total_tests": 5,
"passed": 3,
"failed": 2,
"flaky": 1
}
},
{
"run_id": "run-2024-10-25-002",
"timestamp": "2024-10-25T18:30:00Z",
"environment": "Staging",
"commit_hash": "f0e1d2c3b4a59876543210fedcba09876543210f",
"tests": [
{
"test_name": "Verify user login",
"result": "passed",
"duration_ms": 1100,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Check product listing page",
"result": "passed",
"duration_ms": 900,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Add item to cart",
"result": "passed",
"duration_ms": 1800,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Process checkout",
"result": "passed",
"duration_ms": 5200,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Verify email sending",
"result": "passed",
"duration_ms": 250,
"error_message": null,
"is_flaky": true
}
],
"summary": {
"total_tests": 5,
"passed": 5,
"failed": 0,
"flaky": 1
}
},
{
"run_id": "run-2024-10-25-001",
"timestamp": "2024-10-25T10:00:00Z",
"environment": "Production",
"commit_hash": "f0e1d2c3b4a59876543210fedcba09876543210f",
"tests": [
{
"test_name": "Verify user login",
"result": "passed",
"duration_ms": 1150,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Check product listing page",
"result": "passed",
"duration_ms": 875,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Add item to cart",
"result": "passed",
"duration_ms": 1950,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Process checkout",
"result": "passed",
"duration_ms": 5350,
"error_message": null,
"is_flaky": false
},
{
"test_name": "Verify email sending",
"result": "failed",
"duration_ms": 310,
"error_message": "Timeout waiting for SMTP response",
"is_flaky": true
}
],
"summary": {
"total_tests": 5,
"passed": 4,
"failed": 1,
"flaky": 1
}
}
]
}

123
skills/regression-test-tracker/assets/test_report_template.md Normal file
View File

@@ -0,0 +1,123 @@
# Regression Test Report
**Plugin: Regression Test Tracker**
**Date:** `[Insert Date of Report Generation]`
**Report Version:** `[Insert Report Version Number, e.g., 1.0]`
## 1. Executive Summary
`[Provide a brief, high-level overview of the regression test results. Include the overall pass/fail rate and any critical issues identified. Example: "The regression test suite achieved an overall pass rate of 95%. However, critical tests related to user authentication failed and require immediate investigation."]`
## 2. Test Suite Details
* **Test Suite Name:** `[Insert Name of Test Suite, e.g., "Full Regression Suite", "Critical Path Tests"]`
* **Description:** `[Describe the purpose and scope of the test suite. Example: "This suite covers all core functionalities of the application, ensuring no regressions are introduced by recent code changes."]`
* **Number of Tests:** `[Insert Total Number of Tests in the Suite]`
* **Environment:** `[Specify the environment the tests were run in, e.g., "Staging", "Production", "Development"]`
* **Build/Version:** `[Specify the build number or version of the software being tested. Example: "Build 1234", "Version 2.5.1"]`
* **Commit Hash:** `[Insert the specific commit hash tested]`
## 3. Overall Results
* **Tests Passed:** `[Insert Number of Tests Passed]`
* **Tests Failed:** `[Insert Number of Tests Failed]`
* **Tests Skipped:** `[Insert Number of Tests Skipped]`
* **Pass Rate:** `[Calculate and Insert Pass Rate (Tests Passed / Total Tests * 100)%]`
**Visual Representation (Optional):**
`[Include a chart or graph visualizing the test results (e.g., a pie chart showing the proportion of passed, failed, and skipped tests). This can be easily created using Markdown tables or by linking to an external image.]`
Example Markdown Table:
| Status | Count | Percentage |
| -------- | ----- | ---------- |
| Passed | `[Insert]` | `[Insert]` |
| Failed | `[Insert]` | `[Insert]` |
| Skipped | `[Insert]` | `[Insert]` |
## 4. Detailed Test Results
This section provides a detailed breakdown of the results for each test within the suite.
### 4.1 Failed Tests
`[List each failed test with the following information. Include as much detail as possible to aid in debugging.]`
* **Test Case ID:** `[Insert Test Case ID, e.g., TC_001]`
* **Test Name:** `[Insert Descriptive Test Name, e.g., "Verify User Login with Valid Credentials"]`
* **Description:** `[Briefly describe what the test is supposed to do.]`
* **Error Message:** `[Insert the exact error message returned during the test.]`
* **Stack Trace (if applicable):** `[Insert the stack trace to help pinpoint the source of the error.]`
* **Expected Result:** `[Describe the expected outcome of the test.]`
* **Actual Result:** `[Describe the actual outcome of the test.]`
* **Screenshots/Logs:** `[Link to any relevant screenshots or log files that provide more context.]`
* **Priority:** `[Assign a priority level to the failure (e.g., Critical, High, Medium, Low).]`
* **Assigned To:** `[Indicate who is responsible for investigating and fixing the failure.]`
* **Status:** `[Indicate the current status of the failure (e.g., Open, Investigating, Fixed, Verified).]`
**Example Failed Test Entry:**
* **Test Case ID:** TC_005
* **Test Name:** Verify Password Reset Functionality
* **Description:** Checks if a user can successfully reset their password after requesting a password reset link.
* **Error Message:** `AssertionError: Password reset failed. Password was not updated in the database.`
* **Stack Trace:** `[Insert Stack Trace Here]`
* **Expected Result:** User should be able to reset their password and log in with the new password.
* **Actual Result:** Password was not updated, and user was unable to log in with the new password.
* **Screenshots/Logs:** [Link to Screenshot](path/to/screenshot.png), [Link to Log File](path/to/log.txt)
* **Priority:** Critical
* **Assigned To:** John Doe
* **Status:** Investigating
`[Repeat the above format for each failed test.]`
### 4.2 Skipped Tests
`[List each skipped test with the following information.]`
* **Test Case ID:** `[Insert Test Case ID]`
* **Test Name:** `[Insert Test Name]`
* **Reason for Skipping:** `[Explain why the test was skipped (e.g., "Feature not yet implemented", "Dependency unavailable", "Known issue").]`
**Example Skipped Test Entry:**
* **Test Case ID:** TC_012
* **Test Name:** Verify Integration with Payment Gateway X
* **Reason for Skipping:** Payment Gateway X is currently unavailable in the test environment.
`[Repeat the above format for each skipped test.]`
## 5. Change Impact Analysis
`[Analyze the impact of the code changes on the test results. Identify which changes are likely responsible for the failures and skipped tests. Relate the failed tests back to the specific changes made in the commit.]`
Example:
"The failures in the user authentication tests (TC_001 and TC_002) are likely related to the changes made to the user authentication module in commit `[Insert Commit Hash]`. Specifically, the changes to the password hashing algorithm may have introduced a bug."
## 6. Recommendations
`[Provide recommendations based on the test results. This might include suggestions for fixing bugs, improving test coverage, or addressing performance issues.]`
Example:
* "Investigate and fix the failures in the user authentication tests immediately."
* "Add more test cases to cover edge cases in the user authentication module."
* "Consider implementing a more robust logging system to aid in debugging."
## 7. Conclusion
`[Summarize the overall findings of the regression test and reiterate any critical issues that need to be addressed. State whether the build is ready for deployment or not.]`
Example:
"The regression test suite identified critical failures in the user authentication module. These failures must be addressed before deploying the build to production. Further testing is recommended after the fixes are implemented."
## 8. Appendix (Optional)
`[Include any additional information that may be relevant, such as detailed log files, performance metrics, or configuration settings.]`
`[Link to full test run results if available.]`

7
skills/regression-test-tracker/references/README.md Normal file
View File

@@ -0,0 +1,7 @@
# References
Bundled resources for regression-test-tracker skill
- [ ] regression_test_best_practices.md: Document outlining best practices for writing and maintaining regression tests.
- [ ] test_selection_strategies.md: Document detailing different strategies for selecting tests for the regression suite (e.g., based on code changes).
- [ ] flaky_test_detection_methods.md: Document describing methods for identifying and handling flaky tests.

7
skills/regression-test-tracker/scripts/README.md Normal file
View File

@@ -0,0 +1,7 @@
# Scripts
Bundled resources for regression-test-tracker skill
- [ ] run_regression_tests.sh: Script to execute the regression test suite and report results.
- [ ] mark_test_as_regression.sh: Script to mark a specific test as part of the regression suite.
- [ ] analyze_test_history.py: Python script to analyze test history and identify flaky tests.