Initial commit

skills/markdown-optimizer/USAGE_GUIDE.md

@@ -0,0 +1,148 @@
+# Markdown Optimizer Skill - Usage Guide
+
+## What This Skill Does
+
+The markdown-optimizer skill transforms markdown documents to maximize their utility as LLM reference material by:
+
+1. **Adding structured metadata** - YAML front-matter with title, token count, key concepts, TOC, and diagram suggestions
+2. **Normalizing structure** - Ensures logical heading hierarchy
+3. **Identifying optimization opportunities** - Flags sections that could benefit from diagrams or restructuring
+4. **Removing noise** - Strips redundant formatting and empty lines
+5. **Enabling manual optimization** - Provides patterns for converting verbose prose to structured formats
+
+## Installation
+
+The skill includes everything needed:
+- `scripts/optimize_markdown.py` - Automated optimization script
+- `references/optimization-patterns.md` - Manual optimization patterns and best practices
+- `SKILL.md` - Complete usage instructions
+
+## Quick Start
+
+```bash
+# Run automated optimization
+python scripts/optimize_markdown.py input.md output.md
+
+# Review the output front-matter for optimization suggestions
+# Apply manual optimizations using patterns from references/
+```
+
+## Real-World Example
+
+### Original Document Stats
+- **Tokens**: ~701
+- **Redundant examples**: 3 similar code blocks
+- **Verbose prose**: Multiple sequential steps described in paragraphs
+- **Missing structure**: No metadata or navigation aids
+- **Unclear relationships**: Component architecture described in prose
+
+### After Automated Optimization
+- **Tokens**: ~946 (metadata adds ~245 tokens initially)
+- **Added**: Complete front-matter with TOC and key concepts
+- **Identified**: 6 sections flagged for potential diagrams
+- **Normalized**: Heading hierarchy corrected
+- **Cleaned**: Noise patterns removed
+
+### After Manual Optimization (Using Skill Patterns)
+- **Tokens**: ~420 (**40% reduction** from original)
+- **Improvements**:
+  - Consolidated 3 redundant examples into 1 comprehensive example
+  - Converted verbose step lists to definition lists
+  - Replaced prose architecture description with Mermaid diagram
+  - Created table for command reference
+  - Removed filler phrases ("it's very important", "make sure to")
+  - Added workflow diagram for setup process
+
+## Optimization Results Comparison
+
+| Metric | Original | Auto-Optimized | Fully Optimized |
+|--------|----------|----------------|-----------------|
+| Tokens | 701 | 946 | 420 |
+| Has Metadata | No | Yes | Yes |
+| Has TOC | No | Yes | Yes |
+| Diagrams | 0 | 0 (suggested 6) | 2 (implemented) |
+| Structure | Prose-heavy | Same content | Tables + Lists |
+| Redundancy | High | High | Eliminated |
+
+**Key Insight**: The automated optimizer adds metadata (~35% token increase) but identifies the opportunities that, when manually applied, yield a net 40% reduction.
+
+## Typical Workflow
+
+1. **Run automated optimizer** to add metadata and identify opportunities
+2. **Review `suggested_diagrams`** in front-matter - these often reveal unclear sections
+3. **Apply manual optimizations** using patterns from `references/optimization-patterns.md`:
+   - Convert verbose lists to tables or definition lists
+   - Consolidate redundant examples
+   - Create diagrams for flagged sections
+   - Remove filler phrases and excessive emphasis
+4. **Verify quality** - ensure all key information preserved
+5. **Update token count** in front-matter if desired
+
+## When to Use This Skill
+
+**Ideal for**:
+- Technical documentation being used as skill references
+- Knowledge base articles for LLM ingestion
+- API documentation
+- Process guides and workflows
+- Research notes for prompt construction
+
+**Not recommended for**:
+- Creative writing (stories, poetry)
+- Legal documents (precision required)
+- Already-concise technical specifications
+- Marketing content (different optimization goals)
+
+## Integration with Other Skills
+
+Optimized markdown works particularly well as:
+
+1. **Reference material in custom skills** - Place in `references/` directory
+2. **Knowledge base entries** - Structured metadata aids retrieval
+3. **Prompt components** - Reduced token count allows more context
+4. **Documentation libraries** - Cross-linking via front-matter relationships
+
+Example front-matter for skill integration:
+```yaml
+---
+title: "API Authentication Guide"
+related_docs:
+  - api-reference.md
+  - security-best-practices.md
+dependencies:
+  - python>=3.8
+  - requests
+---
+```
+
+## Advanced Patterns
+
+See `references/optimization-patterns.md` for detailed guidance on:
+- Converting prose to structured formats (tables, definition lists)
+- Diagram patterns for different content types (flowcharts, graphs, architecture)
+- Content compression techniques
+- When NOT to optimize
+- Quality verification checklists
+
+## Example Files
+
+The skill includes example files showing the transformation:
+
+- `example_before.md` - Original verbose document (701 tokens)
+- `example_after.md` - Auto-optimized with metadata (946 tokens)
+- `example_fully_optimized.md` - Manual optimizations applied (420 tokens, 40% reduction)
+
+## Tips
+
+1. **Always run automated optimization first** - it establishes the baseline and identifies opportunities
+2. **Pay attention to suggested diagrams** - they often highlight sections that are hard to understand
+3. **Test iteratively** - optimize one section at a time and verify clarity
+4. **Preserve semantics** - never sacrifice accuracy for brevity
+5. **Update front-matter** - add `related_docs` and `dependencies` for context
+
+## Support
+
+For questions or issues with the skill, refer to:
+- `SKILL.md` - Complete usage instructions
+- `references/optimization-patterns.md` - Detailed optimization patterns
+- Example files - Real-world before/after demonstrations