gh-konflux-ci-skills-skills-navigating-github-to-konflux-pipelines/baseline-analysis.md

Baseline Test Analysis (RED Phase)

Identified Failure Patterns

Without the skill, agents exhibit these consistent problems:

1. Doesn't Know GitHub CLI Commands for Checks

• Symptom: Asks user for more context instead of using available commands
• Example: "Could you share the specific error messages?" instead of gh pr checks
• Root Cause: Doesn't know gh pr checks or gh api check-runs commands exist

2. Incorrect Konflux Check Pattern Matching

• Symptom: Suggests wrong prefixes like konflux/, build/, ci/
• Correct Pattern: Check NAME contains "konflux" (case-insensitive) or URL contains "konflux"
• Examples of Correct Names: "Red Hat Konflux / ...", "Konflux Fedora / ...", "Integration Service / ..."

3. Suggests Manual Browser-Based Workflows

• Symptom: Recommends gh pr view --web to open browser
• Better Approach: Use gh pr view --json or gh api for programmatic access
• Impact: Can't automate or script the workflow

4. Confuses GitHub Actions with Konflux Checks

• Symptom: Suggests looking at "Actions" tab or gh run list
• Correct: Konflux uses GitHub Checks API (check-runs), not Actions workflows
• Impact: Sends user to wrong place entirely

5. Doesn't Know How to Extract URLs from Check Output

• Symptom: Suggests checking cluster with kubectl instead of getting URL from GitHub first
• Missing Knowledge: Integration test checks have PipelineRun URLs in output.text field
• Command Needed: gh api repos/.../commits/.../check-runs --jq '.check_runs[].output.text'

6. Good at Manual URL Parsing (Positive Finding!)

• Works Well: When given a URL directly, can extract cluster/namespace/pipelinerun
• Keep: Agents understand URL structure, just need to know how to GET the URLs

7. Doesn't Distinguish Build vs Integration Test Checks

• Missing: Knowledge that -on-pull-request and -on-push are build checks with direct URLs
• Missing: Integration test checks need output parsing

Key Gaps to Address in Skill

1. Teach gh pr checks and gh api commands
2. Correct Konflux check identification pattern (case-insensitive "konflux" in name/URL)
3. Distinguish build checks (direct URL) from integration tests (parse output)
4. Show -on-pull-request vs -on-push naming
5. Emphasize programmatic API access over browser workflows
6. Clarify Checks API vs Actions workflows

Rationalizations Observed

• "I don't have direct access" → Actually does via gh CLI
• "I cannot use WebSearch" → Doesn't need to, can use gh API
• "I recommend you check manually" → Should provide commands
• "Would you like me to help you with..." → Should just DO it

Success Criteria for GREEN Phase

Agent should:

• ✅ Use gh pr checks or gh api immediately (no asking for context)
• ✅ Filter for "konflux" (case-insensitive) in check names
• ✅ Extract PipelineRun URLs programmatically
• ✅ Distinguish -on-pull-request vs -on-push
• ✅ Know build checks have direct URLs, integration tests need parsing
• ✅ Provide complete kubectl-ready information (cluster, namespace, pipelinerun)