gh-openshift-eng-ai-helpers-plugins-sosreport/commands/analyze.md at master

zhongwei/gh-openshift-eng-ai-helpers-plugins-sosreport

Files

Zhongwei Li 7560761450 Initial commit

2025-11-30 08:46:21 +08:00

13 KiB

Raw Permalink Blame History

description, argument-hint

description	argument-hint
Analyze sosreport archive for system diagnostics and issues	<path-to-sosreport> [--only <areas>] [--skip <areas>]

Name

sosreport:analyze

Synopsis

/sosreport:analyze <path-to-sosreport> [--only <areas>] [--skip <areas>]

Analysis Areas:

logs: Analyze system and application logs (journald, syslog, dmesg, application logs)
- Identifies errors, warnings, critical messages
- Detects OOM killer events, kernel panics, segfaults
- Counts and categorizes errors by severity
- Provides timeline of critical events
resources: Analyze system resource usage (memory, CPU, disk, processes)
- Memory usage, swap, and pressure indicators
- CPU information and load averages
- Disk usage and filesystem capacity
- Top resource consumers and zombie processes
network: Analyze network configuration and connectivity
- Network interface status and IP addresses
- Routing table and default gateway
- Active connections and listening services
- Firewall rules (firewalld/iptables)
- DNS configuration and hostname resolution
system-config: Analyze system configuration (packages, services, security)
- OS version and kernel information
- Installed package versions
- Systemd service status and failures
- SELinux/AppArmor configuration and denials
- Kernel parameters and resource limits

Description

The sosreport:analyze command performs comprehensive analysis of a sosreport archive (from https://github.com/sosreport/sos) to identify system issues, configuration problems, and potential causes of failures. It examines system logs, resource usage, network configuration, installed packages, and other diagnostic data collected by sosreport.

By default, all analysis areas are executed. Use --only to run specific areas or --skip to exclude areas from analysis.

Arguments

$1 (required): Path to the sosreport archive file (.tar.gz or .tar.xz) or extracted directory
--only <areas> (optional): Comma-separated list of analysis areas to run. Valid areas: logs, resources, network, system-config. If not specified, all areas are analyzed.
--skip <areas> (optional): Comma-separated list of analysis areas to skip. Valid areas: logs, resources, network, system-config. Cannot be used with --only.

Implementation

The sosreport analysis is organized into several specialized phases, each with detailed implementation guidance in separate skill documents. The command supports selective analysis through optional arguments.

1. Parse Arguments and Determine Analysis Scope

Parse command-line arguments
- Extract the sosreport path (required first argument)
- Check for --only flag and parse comma-separated areas
- Check for --skip flag and parse comma-separated areas
- Validate that --only and --skip are not used together
Validate analysis areas
- Valid areas: logs, resources, network, system-config
- If invalid area specified, return error with list of valid areas
- Normalize area names (case-insensitive, accept variations like system for system-config)
Determine which skills to run
- If no flags specified: Run all skills (default comprehensive analysis)
- If --only specified: Run only the specified skills
- If --skip specified: Run all skills except the specified ones
- Store the list of skills to execute for later phases

Example argument parsing:

# Parse: /sosreport:analyze /path/sos.tar.gz --only logs,network
# Result: Run only logs-analysis and network-analysis skills

# Parse: /sosreport:analyze /path/sos.tar.gz --skip resources
# Result: Run logs, network, and system-config (skip resources)

# Parse: /sosreport:analyze /path/sos.tar.gz
# Result: Run all skills (comprehensive analysis)

2. Extract and Validate Sosreport

Check if path exists
- Verify the provided path points to a valid file or directory
- If file doesn't exist, return error with helpful message
Extract archive if needed
- If path is a .tar.gz or .tar.xz file:
  - Create extraction directory: .work/sosreport-analyze/{timestamp}/
  - Extract archive: tar -xf <path> -C .work/sosreport-analyze/{timestamp}/
  - Store extracted directory path for analysis
- If path is already a directory:
  - Verify it's a valid sosreport directory (check for sos_commands/, sos_logs/, etc.)
  - Use the directory directly
Identify sosreport structure
- Locate the root directory (usually has format sosreport-{hostname}-{date}/)
- Verify expected directories exist: sos_commands/, sos_logs/, sos_reports/

3. Analyze System Logs

Run condition: Only if logs area is selected (or no filters specified) Detailed implementation: See plugins/sosreport/skills/logs-analysis/SKILL.md

Perform comprehensive log analysis including:

Journald logs (journalctl output)
System logs (messages, dmesg, secure)
Application-specific logs
Error counting and categorization
Timeline of critical events
OOM killer events, kernel panics, segfaults

Key outputs:

Error statistics by severity
Top error messages by frequency
Critical findings with timestamps
Log file locations for investigation

4. Analyze Resource Usage

Run condition: Only if resources area is selected (or no filters specified) Detailed implementation: See plugins/sosreport/skills/resource-analysis/SKILL.md

Perform resource analysis including:

Memory usage and pressure indicators
CPU information and load averages
Disk usage and I/O errors
Process analysis (top consumers, zombies)
Resource exhaustion patterns

Key outputs:

Memory usage metrics and swap status
CPU count and load per CPU
Filesystems near capacity
Top CPU and memory-consuming processes
Resource-related issues and recommendations

5. Analyze Network Configuration

Run condition: Only if network area is selected (or no filters specified) Detailed implementation: See plugins/sosreport/skills/network-analysis/SKILL.md

Perform network analysis including:

Network interface configuration and status
Routing table and default gateway
Active connections and listening services
Firewall rules (firewalld/iptables)
DNS configuration and hostname resolution
Network errors from logs

Key outputs:

Interface status with IP addresses
Routing configuration
Connection statistics by state
Firewall configuration summary
DNS and hostname settings
Network-related errors and issues

6. Analyze Installed Packages and System Configuration

Run condition: Only if system-config area is selected (or no filters specified) Detailed implementation: See plugins/sosreport/skills/system-config-analysis/SKILL.md

Perform system configuration analysis including:

OS version and kernel information
Installed package versions
Systemd service status
Failed services with reasons
SELinux/AppArmor configuration and denials
Kernel parameters and resource limits

Key outputs:

System information summary
Key package versions
Failed services with failure reasons
SELinux status and denial count
Configuration issues and recommendations

7. Generate Interactive Summary

Create findings structure
- Organize findings by category (Critical, High, Medium, Low, Info)
- Include only findings from the selected analysis areas
- For each finding, include:
  - Severity level
  - Category (logs, resources, network, packages, config)
  - Description of the issue
  - Evidence (file paths, log snippets, metrics)
  - Recommended actions
Display summary in terminal
- Show executive summary with key statistics
- List critical and high-severity findings
- Provide file paths for detailed investigation
- Include timeline of significant events
- Suggest next steps for troubleshooting

Format output

SOSREPORT ANALYSIS SUMMARY
==========================

System: {hostname}
Report Date: {date}
OS: {os_version}
Kernel: {kernel_version}

CRITICAL ISSUES (count)
-----------------------
- [Issue description with file reference]

HIGH PRIORITY (count)
---------------------
- [Issue description with file reference]

MEDIUM PRIORITY (count)
-----------------------
- [Issue description with file reference]

RESOURCE SUMMARY
----------------
- Memory: X GB used / Y GB total (Z% used)
- Disk: Most full filesystem at X%
- Load Average: X.XX, X.XX, X.XX

TOP ERRORS IN LOGS
------------------
1. [Error message] (count occurrences)
2. [Error message] (count occurrences)

FAILED SERVICES
---------------
- [service name]: [reason]

RECOMMENDATIONS
---------------
1. [Actionable recommendation]
2. [Actionable recommendation]

ANALYSIS LOCATION
-----------------
Extracted to: {extraction_path}

Interactive drill-down
- Offer to explore specific areas in more detail
- Allow user to ask follow-up questions about findings
- Provide file paths for manual investigation

Return Value

Format: Interactive summary displayed in terminal with categorized findings
Exit code:
- 0 if analysis completes successfully
- 1 if sosreport path is invalid
- 2 if sosreport structure is malformed

Examples

Comprehensive analysis (default):
```
/sosreport:analyze /tmp/sosreport-server01-2024-01-15.tar.xz
```
Extracts archive to .work/sosreport-analyze/{timestamp}/ and performs comprehensive analysis using all skills (logs, resources, network, system-config).
Analyze only logs and network:
```
/sosreport:analyze /tmp/sosreport-server01-2024-01-15.tar.xz --only logs,network
```
Performs only log analysis and network analysis. Useful when investigating connectivity or service issues without needing full resource analysis.
Skip resource analysis:
```
/sosreport:analyze /tmp/sosreport.tar.gz --skip resources
```
Performs all analysis except resource analysis. Useful when you already know resource metrics and want to focus on configuration and logs.
Quick log-only analysis:
```
/sosreport:analyze /tmp/sosreport.tar.xz --only logs
```
Performs only log analysis. Fastest option for quickly identifying errors and critical events without analyzing configuration or resources.
Analyze extracted sosreport directory:
```
/sosreport:analyze /tmp/sosreport-server01-2024-01-15/
```
Analyzes an already extracted sosreport directory with comprehensive analysis.
Selective analysis on extracted directory:
```
/sosreport:analyze /tmp/sosreport-server01-2024-01-15/ --only system-config,network
```
Analyzes only system configuration and network from an already extracted directory.

Follow-up investigation:

User: /sosreport:analyze /tmp/sosreport.tar.gz --only logs
Agent: [Shows log analysis summary]
User: Can you now analyze the resources as well?
Agent: /sosreport:analyze /tmp/sosreport.tar.gz --only resources
Agent: [Shows resource analysis]

Notes

Sosreport structure varies by OS version and sosreport version
Command handles both compressed archives and extracted directories
Analysis focuses on common issues but can be extended for specific use cases
For OpenShift/Kubernetes sosreports, additional pod/container analysis may be relevant
Large sosreports (>1GB) may take several minutes to analyze
Selective analysis: Use --only or --skip to run specific analysis areas for faster results
Performance: Running only needed analysis areas reduces analysis time significantly
Valid areas: logs, resources, network, system-config

Prerequisites

tar utility: Required for extracting compressed sosreports
- Check: which tar
- Usually pre-installed on Linux/macOS
Sufficient disk space: Extracted sosreports can be large
- Check available space: df -h .work/
- Recommend at least 2x the compressed archive size

13 KiB

Raw Permalink Blame History

Name

Synopsis

Description

Arguments

Implementation

1. Parse Arguments and Determine Analysis Scope

2. Extract and Validate Sosreport

3. Analyze System Logs

4. Analyze Resource Usage

5. Analyze Network Configuration

6. Analyze Installed Packages and System Configuration

7. Generate Interactive Summary

Return Value

Examples

Notes

Prerequisites

See Also

Analysis Skills

External Resources

13 KiB Raw Permalink Blame History

Name

Synopsis

Description

Arguments

Implementation

1. Parse Arguments and Determine Analysis Scope

2. Extract and Validate Sosreport

3. Analyze System Logs

4. Analyze Resource Usage

5. Analyze Network Configuration

6. Analyze Installed Packages and System Configuration

7. Generate Interactive Summary

Return Value

Examples

Notes

Prerequisites

See Also

Analysis Skills

External Resources

13 KiB

Raw Permalink Blame History