gh-tachyon-beep-skillpacks-plugins-axiom-system-archaeologist/skills/using-system-archaeologist/validating-architecture-analysis.md

Validating Architecture Analysis

Purpose

Validate architecture analysis artifacts (subsystem catalogs, diagrams, reports) against contract requirements and cross-document consistency standards, producing actionable validation reports with clear approval/revision status.

When to Use

• Coordinator delegates validation after document production
• Task specifies validating 02-subsystem-catalog.md, 03-diagrams.md, or 04-final-report.md
• Validation gate required before proceeding to next phase
• Need independent quality check with fresh eyes
• Output determines whether work progresses or requires revision

Core Principle: Systematic Verification

Good validation finds all issues systematically. Poor validation misses violations or invents false positives.

Your goal: Thorough, objective, evidence-based validation with specific, actionable feedback.

Validation Types

Type 1: Contract Compliance

Validate single document against its contract:

Example contracts:

• Subsystem Catalog (02-subsystem-catalog.md): 8 required sections per entry (Location, Responsibility, Key Components, Dependencies [Inbound/Outbound format], Patterns Observed, Concerns, Confidence, separator)
• Architecture Diagrams (03-diagrams.md): Context + Container + 2-3 Component diagrams, titles/descriptions/legends, assumptions section
• Final Report (04-final-report.md): Executive summary, TOC, diagrams integrated, key findings, appendices

Validation approach:

1. Read contract specification from task or skill documentation
2. Check document systematically against each requirement
3. Flag missing sections, extra sections, wrong formats
4. Distinguish CRITICAL (contract violations) vs WARNING (quality issues)

Type 2: Cross-Document Consistency

Validate that multiple documents align:

Common checks:

• Catalog dependencies match diagram arrows
• Diagram subsystems listed in catalog
• Final report references match source documents
• Confidence levels consistent across documents

Validation approach:

1. Extract key elements from each document
2. Cross-reference systematically
3. Flag inconsistencies with specific citations
4. Provide fixes that maintain consistency

Output: Validation Report

File Path (CRITICAL)

Write to workspace temp/ directory:

<workspace>/temp/validation-<document-name>.md

Examples:

• Workspace: docs/arch-analysis-2025-11-12-1234/
• Catalog validation: docs/arch-analysis-2025-11-12-1234/temp/validation-catalog.md
• Diagram validation: docs/arch-analysis-2025-11-12-1234/temp/validation-diagrams.md
• Consistency validation: docs/arch-analysis-2025-11-12-1234/temp/validation-consistency.md

DO NOT use absolute paths like /home/user/skillpacks/temp/ - write to workspace temp/.

Report Structure (Template)

# Validation Report: [Document Name]

**Document:** `<path to validated document>`
**Validation Date:** YYYY-MM-DD
**Overall Status:** APPROVED | NEEDS_REVISION (CRITICAL) | NEEDS_REVISION (WARNING)


## Contract Requirements

[List the contract requirements being validated against]


## Validation Results

### [Entry/Section 1]

**CRITICAL VIOLATIONS:**
1. [Specific issue with line numbers]
2. [Specific issue with line numbers]

**WARNINGS:**
1. [Quality issue, not blocking]

**Passes:**
- ✓ [What's correct]
- ✓ [What's correct]

**Summary:** X CRITICAL, Y WARNING


### [Entry/Section 2]

...


## Overall Assessment

**Total [Entries/Sections] Analyzed:** N
**[Entries/Sections] with CRITICAL:** X
**Total CRITICAL Violations:** Y
**Total WARNINGS:** Z

### Violations by Type:
1. **[Type]:** Count
2. **[Type]:** Count


## Recommended Actions

### For [Entry/Section]:
[Specific fix with code block]


## Validation Approach

**Methodology:**
[How you validated]

**Checklist:**
[Systematic verification steps]


## Self-Assessment

**Did I find all violations?**
[YES/NO with reasoning]

**Coverage:**
[What was checked]

**Confidence:** [High/Medium/Low]


## Summary

**Status:** [APPROVED or NEEDS_REVISION]
**Critical Issues:** [Count]
**Warnings:** [Count]

[Final disposition]

Validation Status Levels

APPROVED

When to use:

• All contract requirements met
• No CRITICAL violations
• Minor quality issues acceptable (or none)

Report should:

• Confirm compliance
• List what was checked
• Note any minor observations

NEEDS_REVISION (WARNING)

When to use:

• Contract compliant
• Quality issues present (vague descriptions, weak reasoning)
• NOT blocking progression

Report should:

• Confirm contract compliance
• List quality improvements suggested
• Note: "Not blocking, but recommended"
• Distinguish from CRITICAL

NEEDS_REVISION (CRITICAL)

When to use:

• Contract violations (missing/extra sections, wrong format)
• Cross-document inconsistencies
• BLOCKS progression to next phase

Report should:

• List all CRITICAL violations
• Provide specific fixes for each
• Be clear this blocks progression

Systematic Validation Checklist

For Subsystem Catalog

Per entry:

[ ] Section 1: Location with absolute path in backticks?
[ ] Section 2: Responsibility as single sentence?
[ ] Section 3: Key Components as bulleted list?
[ ] Section 4: Dependencies in "Inbound: X / Outbound: Y" format?
[ ] Section 5: Patterns Observed as bulleted list?
[ ] Section 6: Concerns present (or "None observed")?
[ ] Section 7: Confidence (High/Medium/Low) with reasoning?
[ ] Section 8: Separator "---" after entry?
[ ] No extra sections beyond these 8?
[ ] Sections in correct order?

Whole document:

[ ] All subsystems have entries?
[ ] No placeholder text ("[TODO]", "[Fill in]")?
[ ] File named "02-subsystem-catalog.md"?

For Architecture Diagrams

Diagram levels:

[ ] Context diagram (C4 Level 1) present?
[ ] Container diagram (C4 Level 2) present?
[ ] Component diagrams (C4 Level 3) present? (2-3 required)

Per diagram:

[ ] Title present and descriptive?
[ ] Description present after diagram?
[ ] Legend explaining notation?
[ ] Valid syntax (Mermaid or PlantUML)?

Supporting sections:

[ ] Assumptions and Limitations section present?
[ ] Confidence levels documented?

For Cross-Document Consistency

Catalog ↔ Diagrams:

[ ] Each catalog subsystem shown in Container diagram?
[ ] Each catalog "Outbound" dependency shown as diagram arrow?
[ ] Each diagram arrow corresponds to catalog dependency?
[ ] Bidirectional: If A→B in catalog, B shows A as Inbound?

Diagrams ↔ Final Report:

[ ] All diagrams from 03-diagrams.md embedded in report?
[ ] Subsystem descriptions in report match catalog?
[ ] Key findings reference actual concerns from catalog?

Cross-Document Validation Pattern

Step-by-step approach:

1. Extract from Catalog:

   • List all subsystems
   • For each, extract "Outbound" dependencies

2. Extract from Diagram:

   • Find Container diagram
   • List all Rel() statements (Mermaid) or Rel calls (PlantUML)
   • Map source → target for each relationship

3. Cross-Reference:

   • For each catalog dependency, check if diagram shows arrow
   • For each diagram arrow, check if catalog lists dependency
   • Flag mismatches

4. Report Inconsistencies:

   • Use summary table showing what matches and what doesn't
   • Provide line numbers from both documents
   • Suggest specific fixes (add arrow, update catalog)

Best Practices from Baseline Testing

What Works

✅ Thorough checking - Find ALL violations, not just first one ✅ Specific feedback - Line numbers, exact quotes, actionable fixes ✅ Professional reports - Metadata, methodology, self-assessment ✅ Systematic checklists - Document what was verified ✅ Clear status - APPROVED / NEEDS_REVISION with severity ✅ Summary visualizations - Tables showing passed vs failed ✅ Impact analysis - Explain why issues matter ✅ Self-assessment - Verify own completeness

Validation Excellence

Thoroughness patterns:

• Check every entry/section (100% coverage)
• Find both missing AND extra sections
• Distinguish format violations from quality issues

Specificity patterns:

• Provide line numbers for all findings
• Quote exact text showing violation
• Show what correct format should be

Actionability patterns:

• Provide code blocks with fixes
• Suggest alternatives when applicable
• Prioritize fixes (CRITICAL first)

Common Pitfalls to Avoid

❌ Stopping after first violation - Find ALL issues ❌ Vague feedback ("improve quality" vs "add Concerns section") ❌ Wrong status level (marking quality issues as CRITICAL) ❌ False positives (inventing issues that don't exist) ❌ Too lenient (approving despite violations) ❌ Too strict (marking everything CRITICAL) ❌ Wrong file path (absolute path vs workspace temp/) ❌ Skipping self-assessment (verify your own completeness)

Objectivity Under Pressure

If coordinator says "looks fine to me":

• Validate independently anyway
• Evidence-based judgment (cite specific contract)
• Don't soften CRITICAL to WARNING due to authority
• Stand firm: validation is independent quality gate

If time pressure exists:

• Still validate systematically (don't skip checks)
• Document what was validated and what wasn't
• If truly insufficient time, report that honestly

Success Criteria

You succeeded when:

• Found all contract violations (100% detection)
• Specific feedback with line numbers
• Actionable fixes provided
• Clear status (APPROVED/NEEDS_REVISION with severity)
• Professional report structure
• Wrote to workspace temp/ directory
• Self-assessment confirms completeness

You failed when:

• Missed violations
• Vague feedback ("improve this")
• Wrong status level (quality issue marked CRITICAL)
• No actionable fixes
• Wrote to wrong path
• Approved despite violations

Integration with Workflow

This skill is typically invoked as:

1. Coordinator produces document (catalog, diagrams, or report)
2. Coordinator spawns validation subagent (YOU)
3. YOU read document(s) and contract requirements
4. YOU validate systematically using checklists
5. YOU write validation report to workspace temp/
6. Coordinator reads validation report
7. If APPROVED: Coordinator proceeds to next phase
8. If NEEDS_REVISION: Coordinator fixes issues, re-validates (max 2 retries)

Your role: Independent quality gate ensuring artifacts meet standards before progression.