Initial commit
This commit is contained in:
Zhongwei Li
128
skills/analyze-regressions/README.md
Normal file
128
skills/analyze-regressions/README.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# HTML Report Generation for Component Health Analysis
|
||||
|
||||
This directory contains resources for generating interactive HTML reports from component health regression data.
|
||||
|
||||
## Files
|
||||
|
||||
- `report_template.html` - HTML template with placeholders for data
|
||||
- `generate_html_report.py` - Python script to generate reports from JSON data
|
||||
- `README.md` - This file
|
||||
|
||||
## Template Variables
|
||||
|
||||
The HTML template uses the following placeholders (enclosed in `{{}}` double curly braces):
|
||||
|
||||
### Overall Metrics
|
||||
- `{{RELEASE}}` - Release version (e.g., "4.20")
|
||||
- `{{RELEASE_PERIOD}}` - Development period description
|
||||
- `{{DATE_RANGE}}` - Date range for the analysis
|
||||
- `{{GENERATED_DATE}}` - Report generation date
|
||||
|
||||
### Triage Coverage Metrics
|
||||
- `{{TRIAGE_COVERAGE}}` - Percentage (e.g., "25.7")
|
||||
- `{{TRIAGE_COVERAGE_CLASS}}` - CSS class (good/warning/poor)
|
||||
- `{{TRIAGE_COVERAGE_GRADE}}` - Grade text with emoji
|
||||
- `{{TRIAGE_COVERAGE_GRADE_CLASS}}` - Grade CSS class
|
||||
- `{{TOTAL_REGRESSIONS}}` - Total regression count
|
||||
- `{{TRIAGED_REGRESSIONS}}` - Triaged count
|
||||
- `{{UNTRIAGED_REGRESSIONS}}` - Untriaged count
|
||||
|
||||
### Triage Timeliness Metrics
|
||||
- `{{TRIAGE_TIME_AVG}}` - Average hours to triage
|
||||
- `{{TRIAGE_TIME_AVG_DAYS}}` - Average days to triage
|
||||
- `{{TRIAGE_TIME_MAX}}` - Maximum hours to triage
|
||||
- `{{TRIAGE_TIME_MAX_DAYS}}` - Maximum days to triage
|
||||
- `{{TRIAGE_TIME_CLASS}}` - CSS class
|
||||
- `{{TRIAGE_TIME_GRADE}}` - Grade text
|
||||
- `{{TRIAGE_TIME_GRADE_CLASS}}` - Grade CSS class
|
||||
|
||||
### Resolution Speed Metrics
|
||||
- `{{RESOLUTION_TIME_AVG}}` - Average hours to close
|
||||
- `{{RESOLUTION_TIME_AVG_DAYS}}` - Average days to close
|
||||
- `{{RESOLUTION_TIME_MAX}}` - Maximum hours to close
|
||||
- `{{RESOLUTION_TIME_MAX_DAYS}}` - Maximum days to close
|
||||
- `{{RESOLUTION_TIME_CLASS}}` - CSS class
|
||||
- `{{RESOLUTION_TIME_GRADE}}` - Grade text
|
||||
- `{{RESOLUTION_TIME_GRADE_CLASS}}` - Grade CSS class
|
||||
|
||||
### Open/Closed Breakdown
|
||||
- `{{OPEN_REGRESSIONS}}` - Open regression count
|
||||
- `{{OPEN_TRIAGE_PERCENTAGE}}` - Open triage percentage
|
||||
- `{{CLOSED_REGRESSIONS}}` - Closed regression count
|
||||
- `{{CLOSED_TRIAGE_PERCENTAGE}}` - Closed triage percentage
|
||||
- `{{OPEN_AGE_AVG}}` - Average age of open regressions (hours)
|
||||
- `{{OPEN_AGE_AVG_DAYS}}` - Average age of open regressions (days)
|
||||
|
||||
### Dynamic Content
|
||||
- `{{COMPONENT_ROWS}}` - HTML table rows for all components
|
||||
- `{{ATTENTION_SECTIONS}}` - Alert boxes for critical issues
|
||||
- `{{INSIGHTS}}` - List items for key insights
|
||||
- `{{RECOMMENDATIONS}}` - List items for recommendations
|
||||
|
||||
## Usage with Python Script
|
||||
|
||||
### Using data files:
|
||||
```bash
|
||||
python3 generate_html_report.py \
|
||||
--release 4.20 \
|
||||
--data regression_data.json \
|
||||
--dates release_dates.json \
|
||||
--output report.html
|
||||
```
|
||||
|
||||
### Using stdin:
|
||||
```bash
|
||||
cat regression_data.json | python3 generate_html_report.py \
|
||||
--release 4.20 \
|
||||
--dates release_dates.json \
|
||||
--output report.html
|
||||
```
|
||||
|
||||
## Manual Template Usage (for Claude Code)
|
||||
|
||||
When generating reports directly in Claude Code without the Python script:
|
||||
|
||||
1. Read the template file
|
||||
2. Replace all `{{VARIABLE}}` placeholders with actual values
|
||||
3. Generate component rows dynamically
|
||||
4. Build attention sections based on the data
|
||||
5. Write the final HTML to `.work/component-health-{release}/report.html`
|
||||
6. Open with `open` command (macOS) or equivalent
|
||||
|
||||
## Grading Criteria
|
||||
|
||||
### Triage Coverage
|
||||
- **Excellent (✅)**: 90-100%
|
||||
- **Good (✅)**: 70-89%
|
||||
- **Needs Improvement (⚠️)**: 50-69%
|
||||
- **Poor (❌)**: <50%
|
||||
|
||||
### Triage Timeliness
|
||||
- **Excellent (✅)**: <24 hours
|
||||
- **Good (⚠️)**: 24-72 hours
|
||||
- **Needs Improvement (⚠️)**: 72-168 hours (1 week)
|
||||
- **Poor (❌)**: >168 hours
|
||||
|
||||
### Resolution Speed
|
||||
- **Excellent (✅)**: <168 hours (1 week)
|
||||
- **Good (⚠️)**: 168-336 hours (1-2 weeks)
|
||||
- **Needs Improvement (⚠️)**: 336-720 hours (2-4 weeks)
|
||||
- **Poor (❌)**: >720 hours (4+ weeks)
|
||||
|
||||
## Features
|
||||
|
||||
- **Interactive Filtering**: Search components by name and filter by health grade
|
||||
- **Responsive Design**: Works on desktop and mobile devices
|
||||
- **Visual Indicators**: Color-coded metrics (red/yellow/green)
|
||||
- **Hover Effects**: Enhanced UX with hover states
|
||||
- **Alert Sections**: Automatically highlights critical issues
|
||||
- **Auto-generated Content**: Component rows and alerts generated from data
|
||||
|
||||
## Customization
|
||||
|
||||
To customize the report appearance:
|
||||
|
||||
1. Edit `report_template.html` - Modify CSS in the `<style>` section
|
||||
2. Update color schemes by changing gradient values
|
||||
3. Adjust thresholds in the grading logic
|
||||
4. Add new sections by modifying the template structure
|
||||
796
skills/analyze-regressions/SKILL.md
Normal file
796
skills/analyze-regressions/SKILL.md
Normal file
@@ -0,0 +1,796 @@
|
||||
---
|
||||
name: Analyze Regressions
|
||||
description: Grade component health based on regression triage metrics for OpenShift releases
|
||||
---
|
||||
|
||||
# Analyze Regressions
|
||||
|
||||
This skill provides functionality to analyze and grade component health for OpenShift releases based on regression management metrics. It evaluates how well components are managing their test regressions by analyzing triage coverage, triage timeliness, and resolution speed.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
Use this skill when you need to:
|
||||
|
||||
- Grade component health for a specific OpenShift release
|
||||
- Identify components that need help with regression handling
|
||||
- Track triage and resolution efficiency across releases
|
||||
- Generate component quality scorecards
|
||||
- Produce health reports (text or HTML) for stakeholders
|
||||
|
||||
**Important Note**: Grading is subjective and not meant to be a critique of team performance. This is intended to help identify where help is needed and track progress as we try to improve our regression response rates.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Python 3 Installation**
|
||||
|
||||
- Check if installed: `which python3`
|
||||
- Python 3.6 or later is required
|
||||
- Comes pre-installed on most systems
|
||||
|
||||
2. **Network Access**
|
||||
|
||||
- The scripts require network access to reach the component health API and release dates API
|
||||
- Ensure you can make HTTPS requests
|
||||
|
||||
3. **Required Scripts**
|
||||
|
||||
- `plugins/component-health/skills/get-release-dates/get_release_dates.py`
|
||||
- `plugins/component-health/skills/list-regressions/list_regressions.py`
|
||||
- `plugins/component-health/skills/analyze-regressions/generate_html_report.py` (for HTML reports)
|
||||
- `plugins/component-health/skills/analyze-regressions/report_template.html` (for HTML reports)
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Parse Arguments
|
||||
|
||||
Extract the release version and optional component filter from the command arguments:
|
||||
|
||||
- **Release format**: "X.Y" (e.g., "4.17", "4.21")
|
||||
- **Components** (optional): List of component names to filter by
|
||||
|
||||
**Example argument parsing**:
|
||||
|
||||
```
|
||||
/component-health:analyze-regressions 4.17
|
||||
/component-health:analyze-regressions 4.21 --components Monitoring etcd
|
||||
```
|
||||
|
||||
### Step 2: Fetch Release Dates
|
||||
|
||||
Run the `get_release_dates.py` script to determine the development window for the release:
|
||||
|
||||
```bash
|
||||
python3 plugins/component-health/skills/get-release-dates/get_release_dates.py \
|
||||
--release 4.17
|
||||
```
|
||||
|
||||
**Expected output** (JSON on stdout):
|
||||
|
||||
```json
|
||||
{
|
||||
"release": "4.17",
|
||||
"development_start": "2024-05-17T00:00:00Z",
|
||||
"feature_freeze": "2024-08-26T00:00:00Z",
|
||||
"code_freeze": "2024-09-30T00:00:00Z",
|
||||
"ga": "2024-10-29T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
**Processing steps**:
|
||||
|
||||
1. Parse the JSON output
|
||||
2. Extract `development_start` date - convert to YYYY-MM-DD format
|
||||
3. Extract `ga` date - convert to YYYY-MM-DD format (may be null for in-development releases)
|
||||
4. Handle null dates appropriately:
|
||||
- `development_start`: Usually always present; if null, omit `--start` parameter
|
||||
- `ga`: Will be null for in-development releases; if null, omit `--end` parameter
|
||||
|
||||
**Date conversion example**:
|
||||
|
||||
```
|
||||
"2024-05-17T00:00:00Z" → "2024-05-17"
|
||||
null → do not use this parameter
|
||||
```
|
||||
|
||||
### Step 3: Execute List Regressions Script
|
||||
|
||||
Run the `list_regressions.py` script with the appropriate arguments:
|
||||
|
||||
```bash
|
||||
python3 plugins/component-health/skills/list-regressions/list_regressions.py \
|
||||
--release 4.17 \
|
||||
--start 2024-05-17 \
|
||||
--end 2024-10-29 \
|
||||
--short
|
||||
```
|
||||
|
||||
**Parameter rules**:
|
||||
|
||||
- `--release`: Always required (from Step 1)
|
||||
- `--components`: Optional, only if specified by user (from Step 1)
|
||||
- `--start`: Use `development_start` date from Step 2 (if not null)
|
||||
- **Always applied** for both GA'd and in-development releases
|
||||
- Excludes regressions closed before development started (not relevant to this release)
|
||||
- `--end`: Use `ga` date from Step 2 (only if not null)
|
||||
- **Only applied for GA'd releases** (when GA date is not null)
|
||||
- Excludes regressions opened after GA (post-release regressions, often not monitored/triaged)
|
||||
- **Not applied for in-development releases** (when GA date is null)
|
||||
- `--short`: **Always include** this flag
|
||||
- Excludes regression data arrays from response
|
||||
- Only includes summary statistics
|
||||
- Prevents truncation problems with large datasets
|
||||
|
||||
**Example for GA'd release** (4.17):
|
||||
|
||||
```bash
|
||||
python3 plugins/component-health/skills/list-regressions/list_regressions.py \
|
||||
--release 4.17 \
|
||||
--start 2024-05-17 \
|
||||
--end 2024-10-29 \
|
||||
--short
|
||||
```
|
||||
|
||||
**Example for in-development release** (4.21 with null GA):
|
||||
|
||||
```bash
|
||||
python3 plugins/component-health/skills/list-regressions/list_regressions.py \
|
||||
--release 4.21 \
|
||||
--start 2025-09-02 \
|
||||
--short
|
||||
```
|
||||
|
||||
**Example with component filter**:
|
||||
|
||||
```bash
|
||||
python3 plugins/component-health/skills/list-regressions/list_regressions.py \
|
||||
--release 4.21 \
|
||||
--components Monitoring etcd \
|
||||
--start 2025-09-02 \
|
||||
--short
|
||||
```
|
||||
|
||||
### Step 4: Parse Output Structure
|
||||
|
||||
The script outputs JSON to stdout with the following structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"summary": {
|
||||
"total": 62,
|
||||
"triaged": 59,
|
||||
"triage_percentage": 95.2,
|
||||
"filtered_suspected_infra_regressions": 8,
|
||||
"time_to_triage_hrs_avg": 68,
|
||||
"time_to_triage_hrs_max": 240,
|
||||
"time_to_close_hrs_avg": 168,
|
||||
"time_to_close_hrs_max": 480,
|
||||
"open": {
|
||||
"total": 2,
|
||||
"triaged": 1,
|
||||
"triage_percentage": 50.0,
|
||||
"time_to_triage_hrs_avg": 48,
|
||||
"time_to_triage_hrs_max": 48,
|
||||
"open_hrs_avg": 120,
|
||||
"open_hrs_max": 200
|
||||
},
|
||||
"closed": {
|
||||
"total": 60,
|
||||
"triaged": 58,
|
||||
"triage_percentage": 96.7,
|
||||
"time_to_triage_hrs_avg": 72,
|
||||
"time_to_triage_hrs_max": 240,
|
||||
"time_to_close_hrs_avg": 168,
|
||||
"time_to_close_hrs_max": 480,
|
||||
"time_triaged_closed_hrs_avg": 96,
|
||||
"time_triaged_closed_hrs_max": 240
|
||||
}
|
||||
},
|
||||
"components": {
|
||||
"ComponentName": {
|
||||
"summary": {
|
||||
"total": 15,
|
||||
"triaged": 13,
|
||||
"triage_percentage": 86.7,
|
||||
"filtered_suspected_infra_regressions": 0,
|
||||
"time_to_triage_hrs_avg": 68,
|
||||
"time_to_triage_hrs_max": 180,
|
||||
"time_to_close_hrs_avg": 156,
|
||||
"time_to_close_hrs_max": 360,
|
||||
"open": {
|
||||
"total": 1,
|
||||
"triaged": 0,
|
||||
"triage_percentage": 0.0,
|
||||
"time_to_triage_hrs_avg": null,
|
||||
"time_to_triage_hrs_max": null,
|
||||
"open_hrs_avg": 72,
|
||||
"open_hrs_max": 72
|
||||
},
|
||||
"closed": {
|
||||
"total": 14,
|
||||
"triaged": 13,
|
||||
"triage_percentage": 92.9,
|
||||
"time_to_triage_hrs_avg": 68,
|
||||
"time_to_triage_hrs_max": 180,
|
||||
"time_to_close_hrs_avg": 156,
|
||||
"time_to_close_hrs_max": 360,
|
||||
"time_triaged_closed_hrs_avg": 88,
|
||||
"time_triaged_closed_hrs_max": 180
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**CRITICAL - Use Summary Counts**:
|
||||
|
||||
- **ALWAYS use `summary.total`, `summary.open.total`, `summary.closed.total`** for counts
|
||||
- **ALWAYS use `components.*.summary.*`** for per-component counts
|
||||
- Do NOT attempt to count regression arrays (they are excluded with `--short` flag)
|
||||
- This ensures accuracy even with large datasets
|
||||
|
||||
**Key Metrics to Extract**:
|
||||
|
||||
From `summary` object:
|
||||
|
||||
- `summary.total` - Total regressions
|
||||
- `summary.triaged` - Total triaged regressions
|
||||
- `summary.triage_percentage` - **KEY HEALTH METRIC**: Percentage triaged
|
||||
- `summary.filtered_suspected_infra_regressions` - Count of filtered infrastructure regressions
|
||||
- `summary.time_to_triage_hrs_avg` - **KEY HEALTH METRIC**: Average hours to triage
|
||||
- `summary.time_to_triage_hrs_max` - Maximum hours to triage
|
||||
- `summary.time_to_close_hrs_avg` - **KEY HEALTH METRIC**: Average hours to close
|
||||
- `summary.time_to_close_hrs_max` - Maximum hours to close
|
||||
- `summary.open.total` - Open regressions count
|
||||
- `summary.open.triaged` - Open triaged count
|
||||
- `summary.open.triage_percentage` - Open triage percentage
|
||||
- `summary.closed.total` - Closed regressions count
|
||||
- `summary.closed.triaged` - Closed triaged count
|
||||
- `summary.closed.triage_percentage` - Closed triage percentage
|
||||
|
||||
From `components` object:
|
||||
|
||||
- Same fields as summary, but per-component
|
||||
- Use `components.*.summary.*` for all per-component statistics
|
||||
|
||||
### Step 5: Calculate Health Grades
|
||||
|
||||
**IMPORTANT - Closed Regression Triage**:
|
||||
|
||||
- **DO NOT recommend retroactively triaging closed regressions** - the tooling does not support this
|
||||
- When identifying untriaged regressions that need attention, **only consider open regressions**: `summary.open.total - summary.open.triaged`
|
||||
- Closed regression triage percentages are provided for historical analysis only, not as actionable items
|
||||
|
||||
#### Overall Health Grade
|
||||
|
||||
Calculate grades based on three key metrics:
|
||||
|
||||
**1. Triage Coverage** (`summary.triage_percentage`):
|
||||
|
||||
- 90-100%: Excellent ✅
|
||||
- 70-89%: Good ⚠️
|
||||
- 50-69%: Needs Improvement ⚠️
|
||||
- <50%: Poor ❌
|
||||
|
||||
**2. Triage Timeliness** (`summary.time_to_triage_hrs_avg`):
|
||||
|
||||
- <24 hours: Excellent ✅
|
||||
- 24-72 hours: Good ⚠️
|
||||
- 72-168 hours (1 week): Needs Improvement ⚠️
|
||||
- > 168 hours: Poor ❌
|
||||
|
||||
**3. Resolution Speed** (`summary.time_to_close_hrs_avg`):
|
||||
|
||||
- <168 hours (1 week): Excellent ✅
|
||||
- 168-336 hours (1-2 weeks): Good ⚠️
|
||||
- 336-720 hours (2-4 weeks): Needs Improvement ⚠️
|
||||
- > 720 hours (4+ weeks): Poor ❌
|
||||
|
||||
#### Per-Component Health Grades
|
||||
|
||||
For each component in `components`:
|
||||
|
||||
1. Calculate the same three grades using `components.*.summary.*` fields
|
||||
2. Rank components from best to worst health
|
||||
3. Highlight components needing attention:
|
||||
- Low triage coverage (<50%)
|
||||
- Slow triage response (>72 hours average)
|
||||
- Slow resolution time (>336 hours / 2 weeks average)
|
||||
- High open regression counts
|
||||
- High overall regression counts
|
||||
|
||||
### Step 6: Display Text Report
|
||||
|
||||
Present a well-formatted text report with:
|
||||
|
||||
#### Overall Health Grade Section
|
||||
|
||||
Display overall statistics from `summary`:
|
||||
|
||||
```
|
||||
=== Overall Health Grade for Release 4.17 ===
|
||||
Development Window: 2024-05-17 to 2024-10-29 (GA'd release)
|
||||
|
||||
Total Regressions: 62
|
||||
Filtered Infrastructure Regressions: 8
|
||||
Triaged: 59 (95.2%)
|
||||
Open: 2 (50.0% triaged)
|
||||
Closed: 60 (96.7% triaged)
|
||||
|
||||
Triage Coverage: ✅ Excellent (95.2%)
|
||||
Triage Timeliness: ⚠️ Good (68 hours average, 240 hours max)
|
||||
Resolution Speed: ✅ Excellent (168 hours average, 480 hours max)
|
||||
```
|
||||
|
||||
**Important**: If the GA date is null (in-development release), note:
|
||||
|
||||
```
|
||||
Development Window: 2025-09-02 onwards (In Development)
|
||||
```
|
||||
|
||||
#### Per-Component Health Scorecard
|
||||
|
||||
Display ranked table from `components.*.summary`:
|
||||
|
||||
```
|
||||
=== Component Health Scorecard ===
|
||||
|
||||
| Component | Triage Coverage | Triage Time | Resolution Time | Open | Grade |
|
||||
|-----------------|-----------------|-------------|-----------------|------|-------|
|
||||
| kube-apiserver | 100.0% | 58 hrs | 144 hrs | 1 | ✅ |
|
||||
| etcd | 95.0% | 84 hrs | 192 hrs | 0 | ✅ |
|
||||
| Monitoring | 86.7% | 68 hrs | 156 hrs | 1 | ⚠️ |
|
||||
```
|
||||
|
||||
#### Components Needing Attention
|
||||
|
||||
Highlight specific components with issues:
|
||||
|
||||
```
|
||||
=== Components Needing Attention ===
|
||||
|
||||
Monitoring:
|
||||
- 1 open untriaged regression (needs triage)
|
||||
- Triage coverage: 86.7% (below 90%)
|
||||
|
||||
Example-Component:
|
||||
- 5 open untriaged regressions (needs triage)
|
||||
- Slow triage response: 120 hours average
|
||||
- High open count: 5 open regressions
|
||||
```
|
||||
|
||||
**CRITICAL**: When listing untriaged regressions that need action:
|
||||
|
||||
- **Only list OPEN untriaged regressions** - these are actionable
|
||||
- **Do NOT recommend triaging closed regressions** - the tooling does not support retroactive triage
|
||||
- Calculate actionable untriaged count as: `components.*.summary.open.total - components.*.summary.open.triaged`
|
||||
|
||||
### Step 7: Offer HTML Report Generation
|
||||
|
||||
After displaying the text report, ask the user if they want an interactive HTML report:
|
||||
|
||||
```
|
||||
Would you like me to generate an interactive HTML report? (yes/no)
|
||||
```
|
||||
|
||||
If the user responds affirmatively:
|
||||
|
||||
#### Step 7a: Prepare Data for HTML Report
|
||||
|
||||
The HTML report requires data in a specific structure. Transform the JSON data:
|
||||
|
||||
```python
|
||||
# Prepare component data for HTML template
|
||||
component_data = []
|
||||
for component_name, component_obj in components.items():
|
||||
summary = component_obj['summary']
|
||||
component_data.append({
|
||||
'name': component_name,
|
||||
'total': summary['total'],
|
||||
'open': summary['open']['total'],
|
||||
'closed': summary['closed']['total'],
|
||||
'triaged': summary['triaged'],
|
||||
'triage_percentage': summary['triage_percentage'],
|
||||
'time_to_triage_hrs_avg': summary.get('time_to_triage_hrs_avg'),
|
||||
'time_to_close_hrs_avg': summary.get('time_to_close_hrs_avg'),
|
||||
'health_grade': calculate_health_grade(summary) # Calculate combined grade
|
||||
})
|
||||
```
|
||||
|
||||
#### Step 7b: Generate HTML Report
|
||||
|
||||
Use the `generate_html_report.py` script (or inline Python code):
|
||||
|
||||
```bash
|
||||
python3 plugins/component-health/skills/analyze-regressions/generate_html_report.py \
|
||||
--release 4.17 \
|
||||
--data regression_data.json \
|
||||
--output .work/component-health-4.17/report.html
|
||||
```
|
||||
|
||||
Or use inline Python with the template:
|
||||
|
||||
```python
|
||||
import json
|
||||
from datetime import datetime
|
||||
|
||||
# Load template
|
||||
with open('plugins/component-health/skills/analyze-regressions/report_template.html', 'r') as f:
|
||||
template = f.read()
|
||||
|
||||
# Replace placeholders
|
||||
template = template.replace('{{RELEASE}}', '4.17')
|
||||
template = template.replace('{{GENERATED_DATE}}', datetime.now().isoformat())
|
||||
template = template.replace('{{SUMMARY_DATA}}', json.dumps(summary))
|
||||
template = template.replace('{{COMPONENT_DATA}}', json.dumps(component_data))
|
||||
|
||||
# Write output
|
||||
output_path = '.work/component-health-4.17/report.html'
|
||||
os.makedirs(os.path.dirname(output_path), exist_ok=True)
|
||||
with open(output_path, 'w') as f:
|
||||
f.write(template)
|
||||
```
|
||||
|
||||
#### Step 7c: Open the Report
|
||||
|
||||
Open the HTML report in the user's default browser:
|
||||
|
||||
**macOS**:
|
||||
|
||||
```bash
|
||||
open .work/component-health-4.17/report.html
|
||||
```
|
||||
|
||||
**Linux**:
|
||||
|
||||
```bash
|
||||
xdg-open .work/component-health-4.17/report.html
|
||||
```
|
||||
|
||||
**Windows**:
|
||||
|
||||
```bash
|
||||
start .work/component-health-4.17/report.html
|
||||
```
|
||||
|
||||
Display the file path to the user:
|
||||
|
||||
```
|
||||
HTML report generated: .work/component-health-4.17/report.html
|
||||
Opening in your default browser...
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Common Errors
|
||||
|
||||
1. **Network Errors**
|
||||
|
||||
- **Symptom**: `URLError` or connection timeout
|
||||
- **Solution**: Check network connectivity and firewall rules
|
||||
- **Retry**: Both scripts have 30-second timeouts
|
||||
|
||||
2. **Invalid Release Format**
|
||||
|
||||
- **Symptom**: Empty results or error response
|
||||
- **Solution**: Verify the release format (e.g., "4.17", not "v4.17" or "4.17.0")
|
||||
|
||||
3. **Release Dates Not Found**
|
||||
|
||||
- **Symptom**: `get_release_dates.py` returns error
|
||||
- **Solution**: Verify the release exists in the system; may be too old or not yet created
|
||||
- **Fallback**: Proceed without date filtering (omit `--start` and `--end` parameters)
|
||||
|
||||
4. **No Regressions Found**
|
||||
|
||||
- **Symptom**: Empty components object
|
||||
- **Solution**: Verify the release has regression data; may be too early in development
|
||||
- **Action**: Inform user that no regressions exist yet for this release
|
||||
|
||||
5. **Component Filter No Matches**
|
||||
|
||||
- **Symptom**: Empty components object after filtering
|
||||
- **Solution**: Check component name spelling; component names are case-insensitive
|
||||
- **Action**: List available components from unfiltered query
|
||||
|
||||
6. **HTML Template Not Found**
|
||||
- **Symptom**: FileNotFoundError when generating HTML report
|
||||
- **Solution**: Verify template exists at `plugins/component-health/skills/analyze-regressions/report_template.html`
|
||||
- **Fallback**: Offer text report only
|
||||
|
||||
### Debugging
|
||||
|
||||
Enable verbose output by examining stderr:
|
||||
|
||||
```bash
|
||||
python3 plugins/component-health/skills/list-regressions/list_regressions.py \
|
||||
--release 4.17 \
|
||||
--short 2>&1 | tee debug.log
|
||||
```
|
||||
|
||||
Diagnostic messages include:
|
||||
|
||||
- URL being queried
|
||||
- Number of regressions fetched
|
||||
- Number after filtering
|
||||
- Number of suspected infrastructure regressions filtered
|
||||
|
||||
## Output Format
|
||||
|
||||
### Text Report Structure
|
||||
|
||||
The text report should include:
|
||||
|
||||
1. **Header**
|
||||
|
||||
- Release version
|
||||
- Development window dates (start and end/GA)
|
||||
- Release status (GA'd or In Development)
|
||||
|
||||
2. **Overall Health Grade**
|
||||
|
||||
- Total regressions
|
||||
- Filtered infrastructure regressions count
|
||||
- Open/closed breakdown
|
||||
- Triage coverage score with grade
|
||||
- Triage timeliness score with grade
|
||||
- Resolution speed score with grade
|
||||
|
||||
3. **Component Health Scorecard**
|
||||
|
||||
- Ranked table of all components
|
||||
- Key metrics per component
|
||||
- Health grade per component
|
||||
|
||||
4. **Components Needing Attention**
|
||||
|
||||
- List of components with specific issues
|
||||
- Actionable recommendations (only for open untriaged regressions)
|
||||
- Context for each issue
|
||||
|
||||
5. **Footer**
|
||||
- Link to Sippy dashboard (if applicable)
|
||||
- Timestamp of report generation
|
||||
|
||||
### HTML Report Features
|
||||
|
||||
The HTML report should include:
|
||||
|
||||
- **Interactive table** with sorting and filtering
|
||||
- **Visual indicators** for health grades (colors, icons)
|
||||
- **Charts/graphs** showing:
|
||||
- Triage coverage by component
|
||||
- Time to triage distribution
|
||||
- Open vs closed breakdown
|
||||
- **Detailed metrics** on hover or click
|
||||
- **Export functionality** (CSV, PDF)
|
||||
- **Responsive design** for mobile viewing
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Grade Overall Release Health
|
||||
|
||||
```
|
||||
/component-health:analyze-regressions 4.17
|
||||
```
|
||||
|
||||
**Execution flow**:
|
||||
|
||||
1. Fetch release dates for 4.17
|
||||
2. Run list_regressions.py with --start and --end (GA'd release)
|
||||
3. Display overall health grade
|
||||
4. Display per-component scorecard
|
||||
5. Highlight components needing attention
|
||||
6. Offer HTML report generation
|
||||
|
||||
### Example 2: Grade Specific Components
|
||||
|
||||
```
|
||||
/component-health:analyze-regressions 4.21 --components Monitoring etcd
|
||||
```
|
||||
|
||||
**Execution flow**:
|
||||
|
||||
1. Fetch release dates for 4.21 (may have null GA)
|
||||
2. Run list_regressions.py with --components and --start only (in-development)
|
||||
3. Display health grades for Monitoring and etcd only
|
||||
4. Compare the two components
|
||||
5. Identify which needs more attention
|
||||
|
||||
### Example 3: Grade Single Component
|
||||
|
||||
```
|
||||
/component-health:analyze-regressions 4.21 --components "kube-apiserver"
|
||||
```
|
||||
|
||||
**Execution flow**:
|
||||
|
||||
1. Fetch release dates for 4.21
|
||||
2. Run list_regressions.py with single component filter
|
||||
3. Display detailed health metrics for kube-apiserver
|
||||
4. Show open vs closed breakdown
|
||||
5. List count of open untriaged regressions (if any)
|
||||
|
||||
## Health Grade Calculation Details
|
||||
|
||||
### Combined Health Grade
|
||||
|
||||
To calculate an overall health grade for a component, consider all three metrics:
|
||||
|
||||
```python
|
||||
def calculate_health_grade(summary):
|
||||
"""Calculate combined health grade based on three key metrics."""
|
||||
triage_coverage = summary['triage_percentage']
|
||||
triage_time = summary.get('time_to_triage_hrs_avg')
|
||||
resolution_time = summary.get('time_to_close_hrs_avg')
|
||||
|
||||
# Score each metric (0-3)
|
||||
coverage_score = (
|
||||
3 if triage_coverage >= 90 else
|
||||
2 if triage_coverage >= 70 else
|
||||
1 if triage_coverage >= 50 else
|
||||
0
|
||||
)
|
||||
|
||||
time_score = 3 # Default to excellent if no data
|
||||
if triage_time is not None:
|
||||
time_score = (
|
||||
3 if triage_time < 24 else
|
||||
2 if triage_time < 72 else
|
||||
1 if triage_time < 168 else
|
||||
0
|
||||
)
|
||||
|
||||
resolution_score = 3 # Default to excellent if no data
|
||||
if resolution_time is not None:
|
||||
resolution_score = (
|
||||
3 if resolution_time < 168 else
|
||||
2 if resolution_time < 336 else
|
||||
1 if resolution_time < 720 else
|
||||
0
|
||||
)
|
||||
|
||||
# Average the scores
|
||||
avg_score = (coverage_score + time_score + resolution_score) / 3
|
||||
|
||||
# Return grade
|
||||
if avg_score >= 2.5:
|
||||
return "Excellent ✅"
|
||||
elif avg_score >= 1.5:
|
||||
return "Good ⚠️"
|
||||
elif avg_score >= 0.5:
|
||||
return "Needs Improvement ⚠️"
|
||||
else:
|
||||
return "Poor ❌"
|
||||
```
|
||||
|
||||
### Prioritizing Components Needing Attention
|
||||
|
||||
Rank components by priority based on:
|
||||
|
||||
1. **High open untriaged count** (most urgent)
|
||||
|
||||
- Calculate: `summary.open.total - summary.open.triaged`
|
||||
- Threshold: >3 open untriaged regressions
|
||||
|
||||
2. **Low triage coverage** (second priority)
|
||||
|
||||
- Use: `summary.triage_percentage`
|
||||
- Threshold: <50%
|
||||
|
||||
3. **Slow triage response** (third priority)
|
||||
|
||||
- Use: `summary.time_to_triage_hrs_avg`
|
||||
- Threshold: >72 hours
|
||||
|
||||
4. **High total regression count** (fourth priority)
|
||||
- Use: `summary.total`
|
||||
- Threshold: Component-relative (top quartile)
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### Trend Analysis (Future Enhancement)
|
||||
|
||||
Compare metrics across releases:
|
||||
|
||||
```
|
||||
/component-health:analyze-regressions 4.17 --compare 4.16
|
||||
```
|
||||
|
||||
### Export to CSV
|
||||
|
||||
Generate CSV report for spreadsheet analysis:
|
||||
|
||||
```
|
||||
/component-health:analyze-regressions 4.17 --export-csv
|
||||
```
|
||||
|
||||
### Custom Thresholds
|
||||
|
||||
Allow users to customize health grade thresholds:
|
||||
|
||||
```
|
||||
/component-health:analyze-regressions 4.17 --triage-threshold 80
|
||||
```
|
||||
|
||||
## Integration with Other Commands
|
||||
|
||||
This skill can be used by:
|
||||
|
||||
- `/component-health:analyze-regressions` command (primary)
|
||||
- Quality metrics dashboards
|
||||
- Release readiness reports
|
||||
- Team performance tracking tools
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `get-release-dates` - Fetches release development window dates
|
||||
- `list-regressions` - Fetches raw regression data
|
||||
- `prow-job:analyze-test-failure` - Analyzes individual test failures
|
||||
|
||||
## Notes
|
||||
|
||||
- All scripts use Python's standard library only (no external dependencies)
|
||||
- Output is cached in `.work/` directory for performance
|
||||
- Regression data is fetched in real-time from the API
|
||||
- HTML reports are standalone (no external dependencies, embedded CSS/JS)
|
||||
- The `--short` flag is critical to prevent output truncation with large datasets
|
||||
- Health grades are subjective and intended as guidance, not criticism
|
||||
- Infrastructure regressions (closed within 96 hours on high-volume days) are automatically filtered
|
||||
- Retroactive triage of closed regressions is not supported by the tooling
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: Report Shows 0 Regressions
|
||||
|
||||
**Possible causes**:
|
||||
|
||||
1. Release is too early in development
|
||||
2. Date filtering excluded all regressions
|
||||
3. Component filter didn't match any components
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Check release dates with `get_release_dates.py`
|
||||
2. Try without date filtering
|
||||
3. List available components without filter first
|
||||
|
||||
### Issue: Triage Percentages Seem Low
|
||||
|
||||
**Context**:
|
||||
|
||||
- Many teams are still ramping up regression triage practices
|
||||
- Low percentages indicate opportunity for improvement, not failure
|
||||
- Focus on the trend over time rather than absolute numbers
|
||||
|
||||
**Actions**:
|
||||
|
||||
- Identify specific untriaged open regressions that need attention
|
||||
- Prioritize by regression severity and frequency
|
||||
- Track improvement over subsequent releases
|
||||
|
||||
### Issue: HTML Report Not Opening
|
||||
|
||||
**Possible causes**:
|
||||
|
||||
1. Browser security restrictions on local files
|
||||
2. Incorrect file path
|
||||
3. Missing file permissions
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Manually open the file from file explorer
|
||||
2. Verify the file was created at the expected path
|
||||
3. Check file permissions: `ls -la .work/component-health-*/report.html`
|
||||
|
||||
## Summary
|
||||
|
||||
This skill provides comprehensive component health analysis by:
|
||||
|
||||
1. Fetching release development window dates
|
||||
2. Retrieving regression data filtered to the development window
|
||||
3. Calculating health grades based on triage metrics
|
||||
4. Generating actionable reports (text and HTML)
|
||||
5. Identifying components that need help
|
||||
|
||||
The key focus is on **actionable insights** - particularly identifying open untriaged regressions that need immediate attention, while avoiding recommendations for closed regressions which cannot be retroactively triaged.
|
||||
315
skills/analyze-regressions/generate_html_report.py
Normal file
315
skills/analyze-regressions/generate_html_report.py
Normal file
@@ -0,0 +1,315 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generate HTML component health report from JSON data.
|
||||
|
||||
This script reads regression data and generates an interactive HTML report
|
||||
using the template. It handles all the data processing and HTML generation.
|
||||
|
||||
Usage:
|
||||
python3 generate_html_report.py --release 4.20 --data data.json --output report.html
|
||||
|
||||
Or pipe JSON data directly:
|
||||
cat data.json | python3 generate_html_report.py --release 4.20 --output report.html
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
def format_hours_to_days(hours):
|
||||
"""Convert hours to days with one decimal place."""
|
||||
if hours is None:
|
||||
return "N/A"
|
||||
return f"{hours / 24:.1f}"
|
||||
|
||||
|
||||
def get_grade_class(value, thresholds, reverse=False):
|
||||
"""
|
||||
Get CSS class based on value and thresholds.
|
||||
|
||||
Args:
|
||||
value: The value to grade
|
||||
thresholds: Dict with 'excellent', 'good', 'warning' keys
|
||||
reverse: If True, lower is better (for time metrics)
|
||||
"""
|
||||
if value is None:
|
||||
return "poor"
|
||||
|
||||
if reverse:
|
||||
if value < thresholds['excellent']:
|
||||
return "good"
|
||||
elif value < thresholds['good']:
|
||||
return "good"
|
||||
elif value < thresholds['warning']:
|
||||
return "warning"
|
||||
else:
|
||||
return "poor"
|
||||
else:
|
||||
if value >= thresholds['excellent']:
|
||||
return "good"
|
||||
elif value >= thresholds['good']:
|
||||
return "good"
|
||||
elif value >= thresholds['warning']:
|
||||
return "warning"
|
||||
else:
|
||||
return "poor"
|
||||
|
||||
|
||||
def get_grade_text(value, thresholds, reverse=False):
|
||||
"""Get grade text (emoji + text) based on value."""
|
||||
grade_class = get_grade_class(value, thresholds, reverse)
|
||||
|
||||
grade_map = {
|
||||
'good': '✅ EXCELLENT',
|
||||
'warning': '⚠️ NEEDS IMPROVEMENT',
|
||||
'poor': '❌ POOR'
|
||||
}
|
||||
|
||||
# Special handling for coverage and timeliness
|
||||
if not reverse: # Coverage
|
||||
if value is None:
|
||||
return '❌ POOR'
|
||||
elif value >= 90:
|
||||
return '✅ EXCELLENT'
|
||||
elif value >= 70:
|
||||
return '✅ GOOD'
|
||||
elif value >= 50:
|
||||
return '⚠️ NEEDS IMPROVEMENT'
|
||||
else:
|
||||
return '❌ POOR'
|
||||
else: # Timeliness
|
||||
if value is None:
|
||||
return 'N/A'
|
||||
elif value < 24:
|
||||
return '✅ EXCELLENT'
|
||||
elif value < 72:
|
||||
return '⚠️ GOOD'
|
||||
elif value < 168:
|
||||
return '⚠️ NEEDS IMPROVEMENT'
|
||||
else:
|
||||
return '❌ POOR'
|
||||
|
||||
|
||||
def get_component_grade(component_data):
|
||||
"""Calculate overall grade for a component."""
|
||||
triage_pct = component_data['summary']['triage_percentage']
|
||||
|
||||
if triage_pct >= 70:
|
||||
return 'good', '✅ GOOD'
|
||||
elif triage_pct >= 50:
|
||||
return 'good', '⚠️ GOOD'
|
||||
elif triage_pct >= 25:
|
||||
return 'warning', '⚠️ NEEDS IMPROVEMENT'
|
||||
else:
|
||||
return 'poor', '❌ POOR'
|
||||
|
||||
|
||||
def format_time_value(value):
|
||||
"""Format time value, handling None."""
|
||||
if value is None:
|
||||
return '-'
|
||||
return f"{int(value)} hrs"
|
||||
|
||||
|
||||
def format_percentage_value(value):
|
||||
"""Format percentage value with grade class."""
|
||||
if value is None or value == 0:
|
||||
return '<span class="grade-poor">0.0%</span>'
|
||||
elif value >= 90:
|
||||
return f'<span class="grade-excellent">{value:.1f}%</span>'
|
||||
elif value >= 70:
|
||||
return f'<span class="grade-good">{value:.1f}%</span>'
|
||||
elif value >= 50:
|
||||
return f'<span class="grade-warning">{value:.1f}%</span>'
|
||||
else:
|
||||
return f'<span class="grade-poor">{value:.1f}%</span>'
|
||||
|
||||
|
||||
def generate_component_row(name, data):
|
||||
"""Generate a table row for a component."""
|
||||
summary = data['summary']
|
||||
grade_class, grade_text = get_component_grade(data)
|
||||
|
||||
return f'''<tr data-grade="{grade_class}">
|
||||
<td class="component-name">{name}</td>
|
||||
<td>{summary['total']}</td>
|
||||
<td>{format_percentage_value(summary['triage_percentage'])}</td>
|
||||
<td>{format_time_value(summary['time_to_triage_hrs_avg'])}</td>
|
||||
<td>{format_time_value(summary['time_to_close_hrs_avg'])}</td>
|
||||
<td>{summary['open']['total']}</td>
|
||||
<td class="grade-{grade_class}">{grade_text}</td>
|
||||
</tr>'''
|
||||
|
||||
|
||||
def generate_html_report(release, data, release_dates, output_path):
|
||||
"""Generate HTML report from data."""
|
||||
template_path = Path(__file__).parent / "report_template.html"
|
||||
|
||||
with open(template_path, 'r') as f:
|
||||
template = f.read()
|
||||
|
||||
# Extract summary data
|
||||
summary = data['summary']
|
||||
|
||||
# Calculate derived values
|
||||
triage_coverage = summary['triage_percentage']
|
||||
triage_time_avg = summary['time_to_triage_hrs_avg'] or 0
|
||||
resolution_time_avg = summary['time_to_close_hrs_avg'] or 0
|
||||
|
||||
# Determine release period
|
||||
dev_start = release_dates.get('development_start', 'Unknown')
|
||||
ga_date = release_dates.get('ga')
|
||||
|
||||
if dev_start != 'Unknown':
|
||||
dev_start = dev_start.split('T')[0]
|
||||
|
||||
if ga_date:
|
||||
ga_date = ga_date.split('T')[0]
|
||||
release_period = f"Development Period: {dev_start} - {ga_date} (GA)"
|
||||
date_range = f"{dev_start} (Development Start) to {ga_date} (GA)"
|
||||
else:
|
||||
release_period = f"Development Period: {dev_start} - Present (In Development)"
|
||||
date_range = f"{dev_start} (Development Start) - Present"
|
||||
|
||||
# Calculate grades
|
||||
triage_coverage_class = get_grade_class(triage_coverage,
|
||||
{'excellent': 90, 'good': 70, 'warning': 50})
|
||||
triage_time_class = get_grade_class(triage_time_avg,
|
||||
{'excellent': 24, 'good': 72, 'warning': 168},
|
||||
reverse=True)
|
||||
resolution_time_class = get_grade_class(resolution_time_avg,
|
||||
{'excellent': 168, 'good': 336, 'warning': 720},
|
||||
reverse=True)
|
||||
|
||||
# Build component rows (sorted by health)
|
||||
components = data.get('components', {})
|
||||
component_rows = []
|
||||
|
||||
for name, comp_data in sorted(components.items(),
|
||||
key=lambda x: x[1]['summary']['triage_percentage'],
|
||||
reverse=True):
|
||||
component_rows.append(generate_component_row(name, comp_data))
|
||||
|
||||
# Generate attention sections
|
||||
attention_sections = []
|
||||
|
||||
# Zero triage components
|
||||
zero_triage = [name for name, comp in components.items()
|
||||
if comp['summary']['triage_percentage'] == 0]
|
||||
if zero_triage:
|
||||
items = '\n'.join([f'<li><strong>{name}</strong> - {components[name]["summary"]["total"]} regressions</li>'
|
||||
for name in zero_triage])
|
||||
attention_sections.append(f'''<div class="alert-box">
|
||||
<h3>🚨 Zero Triage Coverage (0% triaged)</h3>
|
||||
<ul>
|
||||
{items}
|
||||
</ul>
|
||||
</div>''')
|
||||
|
||||
# Low triage components
|
||||
low_triage = [(name, comp) for name, comp in components.items()
|
||||
if 0 < comp['summary']['triage_percentage'] < 25]
|
||||
if low_triage:
|
||||
items = '\n'.join([f'<li><strong>{name}</strong> - {comp["summary"]["total"]} regressions, only {comp["summary"]["triage_percentage"]:.1f}% triaged</li>'
|
||||
for name, comp in low_triage])
|
||||
attention_sections.append(f'''<div class="alert-box">
|
||||
<h3>⚠️ Low Triage Coverage (<25%)</h3>
|
||||
<ul>
|
||||
{items}
|
||||
</ul>
|
||||
</div>''')
|
||||
|
||||
# High volume components
|
||||
high_volume = [(name, comp) for name, comp in components.items()
|
||||
if comp['summary']['total'] >= 20]
|
||||
if high_volume:
|
||||
high_volume.sort(key=lambda x: x[1]['summary']['total'], reverse=True)
|
||||
items = '\n'.join([f'<li><strong>{name}</strong> - {comp["summary"]["total"]} regressions ({comp["summary"]["triage_percentage"]:.1f}% triaged)</li>'
|
||||
for name, comp in high_volume[:5]])
|
||||
attention_sections.append(f'''<div class="alert-box">
|
||||
<h3>📊 High Regression Volume (Top 5)</h3>
|
||||
<ul>
|
||||
{items}
|
||||
</ul>
|
||||
</div>''')
|
||||
|
||||
# Substitutions
|
||||
substitutions = {
|
||||
'RELEASE': release,
|
||||
'RELEASE_PERIOD': release_period,
|
||||
'DATE_RANGE': date_range,
|
||||
'TRIAGE_COVERAGE': f"{triage_coverage:.1f}",
|
||||
'TRIAGE_COVERAGE_CLASS': triage_coverage_class,
|
||||
'TRIAGE_COVERAGE_GRADE': get_grade_text(triage_coverage, {}, False),
|
||||
'TRIAGE_COVERAGE_GRADE_CLASS': f"grade-{triage_coverage_class}",
|
||||
'TOTAL_REGRESSIONS': str(summary['total']),
|
||||
'TRIAGED_REGRESSIONS': str(summary['triaged']),
|
||||
'UNTRIAGED_REGRESSIONS': str(summary['total'] - summary['triaged']),
|
||||
'TRIAGE_TIME_AVG': str(int(triage_time_avg)) if triage_time_avg else 'N/A',
|
||||
'TRIAGE_TIME_AVG_DAYS': format_hours_to_days(triage_time_avg),
|
||||
'TRIAGE_TIME_MAX': str(int(summary['time_to_triage_hrs_max'])) if summary['time_to_triage_hrs_max'] else 'N/A',
|
||||
'TRIAGE_TIME_MAX_DAYS': format_hours_to_days(summary['time_to_triage_hrs_max']),
|
||||
'TRIAGE_TIME_CLASS': triage_time_class,
|
||||
'TRIAGE_TIME_GRADE': get_grade_text(triage_time_avg, {}, True),
|
||||
'TRIAGE_TIME_GRADE_CLASS': f"grade-{triage_time_class}",
|
||||
'RESOLUTION_TIME_AVG': str(int(resolution_time_avg)) if resolution_time_avg else 'N/A',
|
||||
'RESOLUTION_TIME_AVG_DAYS': format_hours_to_days(resolution_time_avg),
|
||||
'RESOLUTION_TIME_MAX': str(int(summary['time_to_close_hrs_max'])) if summary['time_to_close_hrs_max'] else 'N/A',
|
||||
'RESOLUTION_TIME_MAX_DAYS': format_hours_to_days(summary['time_to_close_hrs_max']),
|
||||
'RESOLUTION_TIME_CLASS': resolution_time_class,
|
||||
'RESOLUTION_TIME_GRADE': get_grade_text(resolution_time_avg, {}, True),
|
||||
'RESOLUTION_TIME_GRADE_CLASS': f"grade-{resolution_time_class}",
|
||||
'OPEN_REGRESSIONS': str(summary['open']['total']),
|
||||
'OPEN_TRIAGE_PERCENTAGE': f"{summary['open']['triage_percentage']:.1f}",
|
||||
'CLOSED_REGRESSIONS': str(summary['closed']['total']),
|
||||
'CLOSED_TRIAGE_PERCENTAGE': f"{summary['closed']['triage_percentage']:.1f}",
|
||||
'OPEN_AGE_AVG': str(int(summary['open']['open_hrs_avg'])) if summary['open']['open_hrs_avg'] else 'N/A',
|
||||
'OPEN_AGE_AVG_DAYS': format_hours_to_days(summary['open']['open_hrs_avg']),
|
||||
'COMPONENT_ROWS': '\n'.join(component_rows),
|
||||
'ATTENTION_SECTIONS': '\n'.join(attention_sections),
|
||||
'INSIGHTS': '<li>Report generated automatically from regression data</li>',
|
||||
'RECOMMENDATIONS': '<li>Review components with zero triage coverage</li><li>Address high-volume components</li>',
|
||||
'GENERATED_DATE': datetime.now().strftime("%B %d, %Y"),
|
||||
}
|
||||
|
||||
# Apply substitutions
|
||||
for key, value in substitutions.items():
|
||||
template = template.replace(f'{{{{{key}}}}}', value)
|
||||
|
||||
# Write output
|
||||
with open(output_path, 'w') as f:
|
||||
f.write(template)
|
||||
|
||||
print(f"HTML report generated: {output_path}", file=sys.stderr)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description='Generate HTML component health report')
|
||||
parser.add_argument('--release', required=True, help='Release version (e.g., 4.20)')
|
||||
parser.add_argument('--data', help='Path to JSON data file (or read from stdin)')
|
||||
parser.add_argument('--dates', help='Path to release dates JSON file')
|
||||
parser.add_argument('--output', required=True, help='Output HTML file path')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Read regression data
|
||||
if args.data:
|
||||
with open(args.data, 'r') as f:
|
||||
data = json.load(f)
|
||||
else:
|
||||
data = json.load(sys.stdin)
|
||||
|
||||
# Read release dates
|
||||
release_dates = {}
|
||||
if args.dates:
|
||||
with open(args.dates, 'r') as f:
|
||||
release_dates = json.load(f)
|
||||
|
||||
generate_html_report(args.release, data, release_dates, args.output)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
486
skills/analyze-regressions/report_template.html
Normal file
486
skills/analyze-regressions/report_template.html
Normal file
@@ -0,0 +1,486 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Component Health Report - OpenShift {{RELEASE}}</title>
|
||||
<style>
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif;
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
padding: 20px;
|
||||
line-height: 1.6;
|
||||
}
|
||||
|
||||
.container {
|
||||
max-width: 1400px;
|
||||
margin: 0 auto;
|
||||
background: white;
|
||||
border-radius: 12px;
|
||||
box-shadow: 0 20px 60px rgba(0,0,0,0.3);
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.header {
|
||||
background: linear-gradient(135deg, #2c3e50 0%, #34495e 100%);
|
||||
color: white;
|
||||
padding: 40px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.header h1 {
|
||||
font-size: 2.5em;
|
||||
margin-bottom: 10px;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.header .subtitle {
|
||||
font-size: 1.1em;
|
||||
opacity: 0.9;
|
||||
margin-top: 10px;
|
||||
}
|
||||
|
||||
.content {
|
||||
padding: 40px;
|
||||
}
|
||||
|
||||
.section {
|
||||
margin-bottom: 50px;
|
||||
}
|
||||
|
||||
.section h2 {
|
||||
color: #2c3e50;
|
||||
font-size: 1.8em;
|
||||
margin-bottom: 20px;
|
||||
padding-bottom: 10px;
|
||||
border-bottom: 3px solid #667eea;
|
||||
}
|
||||
|
||||
.metrics-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(300px, 1fr));
|
||||
gap: 20px;
|
||||
margin-bottom: 30px;
|
||||
}
|
||||
|
||||
.metric-card {
|
||||
background: #f8f9fa;
|
||||
border-radius: 8px;
|
||||
padding: 25px;
|
||||
border-left: 5px solid #667eea;
|
||||
transition: transform 0.2s, box-shadow 0.2s;
|
||||
}
|
||||
|
||||
.metric-card:hover {
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 4px 12px rgba(0,0,0,0.1);
|
||||
}
|
||||
|
||||
.metric-card.poor {
|
||||
border-left-color: #e74c3c;
|
||||
background: #fee;
|
||||
}
|
||||
|
||||
.metric-card.warning {
|
||||
border-left-color: #f39c12;
|
||||
background: #fff8e1;
|
||||
}
|
||||
|
||||
.metric-card.good {
|
||||
border-left-color: #27ae60;
|
||||
background: #e8f5e9;
|
||||
}
|
||||
|
||||
.metric-label {
|
||||
font-size: 0.9em;
|
||||
color: #666;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
|
||||
.metric-value {
|
||||
font-size: 2.5em;
|
||||
font-weight: 700;
|
||||
color: #2c3e50;
|
||||
margin-bottom: 5px;
|
||||
}
|
||||
|
||||
.metric-assessment {
|
||||
font-size: 1.1em;
|
||||
font-weight: 600;
|
||||
margin-top: 10px;
|
||||
}
|
||||
|
||||
.metric-details {
|
||||
font-size: 0.9em;
|
||||
color: #555;
|
||||
margin-top: 10px;
|
||||
}
|
||||
|
||||
table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
margin-top: 20px;
|
||||
background: white;
|
||||
border-radius: 8px;
|
||||
overflow: hidden;
|
||||
box-shadow: 0 2px 8px rgba(0,0,0,0.1);
|
||||
}
|
||||
|
||||
thead {
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
color: white;
|
||||
}
|
||||
|
||||
th {
|
||||
padding: 15px;
|
||||
text-align: left;
|
||||
font-weight: 600;
|
||||
font-size: 0.9em;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
}
|
||||
|
||||
td {
|
||||
padding: 12px 15px;
|
||||
border-bottom: 1px solid #ecf0f1;
|
||||
}
|
||||
|
||||
tbody tr:hover {
|
||||
background: #f8f9fa;
|
||||
}
|
||||
|
||||
tbody tr:last-child td {
|
||||
border-bottom: none;
|
||||
}
|
||||
|
||||
.grade-excellent {
|
||||
color: #27ae60;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.grade-good {
|
||||
color: #3498db;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.grade-warning {
|
||||
color: #f39c12;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.grade-poor {
|
||||
color: #e74c3c;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.alert-box {
|
||||
background: #fee;
|
||||
border-left: 5px solid #e74c3c;
|
||||
padding: 20px;
|
||||
border-radius: 8px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.alert-box h3 {
|
||||
color: #c0392b;
|
||||
margin-bottom: 15px;
|
||||
font-size: 1.3em;
|
||||
}
|
||||
|
||||
.alert-box ul {
|
||||
list-style-position: inside;
|
||||
color: #555;
|
||||
}
|
||||
|
||||
.alert-box li {
|
||||
margin-bottom: 8px;
|
||||
padding-left: 10px;
|
||||
}
|
||||
|
||||
.insight-box {
|
||||
background: #e8f5e9;
|
||||
border-left: 5px solid #27ae60;
|
||||
padding: 20px;
|
||||
border-radius: 8px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.insight-box h3 {
|
||||
color: #27ae60;
|
||||
margin-bottom: 15px;
|
||||
font-size: 1.3em;
|
||||
}
|
||||
|
||||
.insight-box ul {
|
||||
list-style-position: inside;
|
||||
color: #555;
|
||||
}
|
||||
|
||||
.insight-box li {
|
||||
margin-bottom: 8px;
|
||||
padding-left: 10px;
|
||||
}
|
||||
|
||||
.recommendation-box {
|
||||
background: #fff8e1;
|
||||
border-left: 5px solid #f39c12;
|
||||
padding: 20px;
|
||||
border-radius: 8px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.recommendation-box h3 {
|
||||
color: #e67e22;
|
||||
margin-bottom: 15px;
|
||||
font-size: 1.3em;
|
||||
}
|
||||
|
||||
.recommendation-box ol {
|
||||
list-style-position: inside;
|
||||
color: #555;
|
||||
}
|
||||
|
||||
.recommendation-box li {
|
||||
margin-bottom: 10px;
|
||||
padding-left: 10px;
|
||||
}
|
||||
|
||||
.stats-row {
|
||||
display: flex;
|
||||
gap: 15px;
|
||||
flex-wrap: wrap;
|
||||
margin-top: 15px;
|
||||
}
|
||||
|
||||
.stat-pill {
|
||||
background: white;
|
||||
padding: 8px 15px;
|
||||
border-radius: 20px;
|
||||
font-size: 0.9em;
|
||||
border: 2px solid #ddd;
|
||||
}
|
||||
|
||||
.stat-pill strong {
|
||||
color: #2c3e50;
|
||||
}
|
||||
|
||||
.footer {
|
||||
background: #f8f9fa;
|
||||
padding: 20px 40px;
|
||||
text-align: center;
|
||||
color: #666;
|
||||
font-size: 0.9em;
|
||||
border-top: 1px solid #ddd;
|
||||
}
|
||||
|
||||
.component-name {
|
||||
font-weight: 600;
|
||||
color: #2c3e50;
|
||||
}
|
||||
|
||||
.filter-controls {
|
||||
margin: 20px 0;
|
||||
padding: 20px;
|
||||
background: #f8f9fa;
|
||||
border-radius: 8px;
|
||||
}
|
||||
|
||||
.filter-controls label {
|
||||
margin-right: 15px;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.filter-controls input[type="text"] {
|
||||
padding: 8px 12px;
|
||||
border: 2px solid #ddd;
|
||||
border-radius: 4px;
|
||||
font-size: 1em;
|
||||
width: 300px;
|
||||
}
|
||||
|
||||
.filter-controls select {
|
||||
padding: 8px 12px;
|
||||
border: 2px solid #ddd;
|
||||
border-radius: 4px;
|
||||
font-size: 1em;
|
||||
margin-left: 10px;
|
||||
}
|
||||
|
||||
.hidden {
|
||||
display: none;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<div class="header">
|
||||
<h1>Component Health Report</h1>
|
||||
<div class="subtitle">OpenShift {{RELEASE}} Release</div>
|
||||
<div class="subtitle">{{RELEASE_PERIOD}}</div>
|
||||
</div>
|
||||
|
||||
<div class="content">
|
||||
<!-- Overall Health Section -->
|
||||
<div class="section">
|
||||
<h2>Overall Health Grade</h2>
|
||||
|
||||
<div class="metrics-grid">
|
||||
<div class="metric-card {{TRIAGE_COVERAGE_CLASS}}">
|
||||
<div class="metric-label">Triage Coverage</div>
|
||||
<div class="metric-value">{{TRIAGE_COVERAGE}}%</div>
|
||||
<div class="metric-assessment {{TRIAGE_COVERAGE_GRADE_CLASS}}">{{TRIAGE_COVERAGE_GRADE}}</div>
|
||||
<div class="metric-details">
|
||||
<div>Total: {{TOTAL_REGRESSIONS}} regressions</div>
|
||||
<div>Triaged: {{TRIAGED_REGRESSIONS}}</div>
|
||||
<div>Untriaged: {{UNTRIAGED_REGRESSIONS}}</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="metric-card {{TRIAGE_TIME_CLASS}}">
|
||||
<div class="metric-label">Triage Timeliness</div>
|
||||
<div class="metric-value">{{TRIAGE_TIME_AVG}} hrs</div>
|
||||
<div class="metric-assessment {{TRIAGE_TIME_GRADE_CLASS}}">{{TRIAGE_TIME_GRADE}}</div>
|
||||
<div class="metric-details">
|
||||
<div>Average: {{TRIAGE_TIME_AVG_DAYS}} days</div>
|
||||
<div>Maximum: {{TRIAGE_TIME_MAX}} hrs ({{TRIAGE_TIME_MAX_DAYS}} days)</div>
|
||||
<div>Target: <72 hours</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="metric-card {{RESOLUTION_TIME_CLASS}}">
|
||||
<div class="metric-label">Resolution Speed</div>
|
||||
<div class="metric-value">{{RESOLUTION_TIME_AVG}} hrs</div>
|
||||
<div class="metric-assessment {{RESOLUTION_TIME_GRADE_CLASS}}">{{RESOLUTION_TIME_GRADE}}</div>
|
||||
<div class="metric-details">
|
||||
<div>Average: {{RESOLUTION_TIME_AVG_DAYS}} days</div>
|
||||
<div>Maximum: {{RESOLUTION_TIME_MAX}} hrs ({{RESOLUTION_TIME_MAX_DAYS}} days)</div>
|
||||
<div>Target: 1-2 weeks</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="stats-row">
|
||||
<div class="stat-pill">
|
||||
<strong>Open:</strong> {{OPEN_REGRESSIONS}} regressions ({{OPEN_TRIAGE_PERCENTAGE}}% triaged)
|
||||
</div>
|
||||
<div class="stat-pill">
|
||||
<strong>Closed:</strong> {{CLOSED_REGRESSIONS}} regressions ({{CLOSED_TRIAGE_PERCENTAGE}}% triaged)
|
||||
</div>
|
||||
<div class="stat-pill">
|
||||
<strong>Avg Age (Open):</strong> {{OPEN_AGE_AVG}} hours ({{OPEN_AGE_AVG_DAYS}} days)
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Component Scorecard -->
|
||||
<div class="section">
|
||||
<h2>Per-Component Health Scorecard</h2>
|
||||
|
||||
<div class="filter-controls">
|
||||
<label for="searchInput">Search Component:</label>
|
||||
<input type="text" id="searchInput" placeholder="Type component name...">
|
||||
|
||||
<label for="gradeFilter">Filter by Grade:</label>
|
||||
<select id="gradeFilter">
|
||||
<option value="all">All Grades</option>
|
||||
<option value="excellent">Excellent</option>
|
||||
<option value="good">Good</option>
|
||||
<option value="warning">Needs Improvement</option>
|
||||
<option value="poor">Poor</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<table id="componentTable">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Component</th>
|
||||
<th>Total Regressions</th>
|
||||
<th>Triage Coverage</th>
|
||||
<th>Avg Triage Time</th>
|
||||
<th>Avg Resolution Time</th>
|
||||
<th>Open</th>
|
||||
<th>Health Grade</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="componentTableBody">
|
||||
{{COMPONENT_ROWS}}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- Critical Attention Section -->
|
||||
<div class="section">
|
||||
<h2>Components Needing Critical Attention</h2>
|
||||
{{ATTENTION_SECTIONS}}
|
||||
</div>
|
||||
|
||||
<!-- Key Insights -->
|
||||
<div class="section">
|
||||
<h2>Key Insights</h2>
|
||||
<div class="insight-box">
|
||||
<h3>📈 Analysis Summary</h3>
|
||||
<ul>
|
||||
{{INSIGHTS}}
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Recommendations -->
|
||||
<div class="section">
|
||||
<h2>Recommendations</h2>
|
||||
<div class="recommendation-box">
|
||||
<h3>🎯 Action Items</h3>
|
||||
<ol>
|
||||
{{RECOMMENDATIONS}}
|
||||
</ol>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="footer">
|
||||
<strong>Data Source:</strong> Sippy Component Readiness API<br>
|
||||
<strong>Date Range:</strong> {{DATE_RANGE}}<br>
|
||||
<strong>Total Regressions Analyzed:</strong> {{TOTAL_REGRESSIONS}}<br>
|
||||
<strong>Generated:</strong> {{GENERATED_DATE}}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Search functionality
|
||||
const searchInput = document.getElementById('searchInput');
|
||||
const gradeFilter = document.getElementById('gradeFilter');
|
||||
const tableBody = document.getElementById('componentTableBody');
|
||||
const rows = tableBody.getElementsByTagName('tr');
|
||||
|
||||
function filterTable() {
|
||||
const searchTerm = searchInput.value.toLowerCase();
|
||||
const gradeValue = gradeFilter.value;
|
||||
|
||||
for (let row of rows) {
|
||||
const componentName = row.querySelector('.component-name').textContent.toLowerCase();
|
||||
const grade = row.getAttribute('data-grade');
|
||||
|
||||
const matchesSearch = componentName.includes(searchTerm);
|
||||
const matchesGrade = gradeValue === 'all' || grade === gradeValue;
|
||||
|
||||
if (matchesSearch && matchesGrade) {
|
||||
row.style.display = '';
|
||||
} else {
|
||||
row.style.display = 'none';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
searchInput.addEventListener('input', filterTable);
|
||||
gradeFilter.addEventListener('change', filterTable);
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
Reference in New Issue
Block a user