9.2 KiB
9.2 KiB
Operation: Validate Example Quality
Validate example code quality, detecting placeholders and ensuring examples are complete and runnable.
Parameters from $ARGUMENTS
- path: Target plugin/marketplace path (required)
- no-placeholders: Strict placeholder enforcement (optional, default: true)
- recursive: Check all markdown and code files recursively (optional, default: true)
- extensions: File extensions to check (optional, default: "md,txt,json,sh,py,js")
Example Quality Requirements
Complete Examples:
- Concrete, runnable code or commands
- Real values, not placeholder text
- Proper syntax and formatting
- Context and explanations
- Expected output or results
No Placeholder Patterns:
- TODO:
TODO,@TODO,// TODO: - FIXME:
FIXME,@FIXME,// FIXME: - XXX:
XXX,@XXX,// XXX: - Placeholders:
placeholder,PLACEHOLDER,your-value-here,<your-value>,[YOUR-VALUE] - Generic:
example,sample,test,dummy,foo,bar,baz - User substitution:
<username>,<your-email>,your-api-key,INSERT-HERE
Acceptable Patterns (not placeholders):
- Template variables:
{{variable}},${variable},$VARIABLE - Documentation examples:
<name>,[optional]in usage syntax - Actual values: Real plugin names, real commands, concrete examples
Workflow
-
Identify Files to Validate
Scan plugin directory for documentation files: - README.md (primary source) - CONTRIBUTING.md - docs/**/*.md - examples/**/* - *.sh, *.py, *.js (example scripts) If recursive:false, only check README.md -
Execute Example Validator
Execute .scripts/example-validator.sh with parameters: - Path to plugin directory - No-placeholders flag - Recursive flag - File extensions to check Script returns: - files_checked: Count of files analyzed - placeholders_found: Array of placeholder instances - files_with_issues: Array of files containing placeholders - example_count: Number of code examples found - quality_score: 0-100 -
Detect Placeholder Patterns
Search for patterns (case-insensitive): # TODO/FIXME/XXX markers grep -iE '(TODO|FIXME|XXX|HACK)[:)]' <files> # Placeholder text grep -iE '(placeholder|your-.*-here|<your-|INSERT.?HERE)' <files> # Generic dummy values grep -iE '\b(foo|bar|baz|dummy|sample|test)\b' <files> # User substitution patterns grep -iE '(<username>|<email>|<api-key>|YOUR_[A-Z_]+)' <files> # Exclude: - Comments explaining placeholders - Documentation of template syntax - Proper template variables ({{x}}, ${x}) -
Analyze Code Blocks
For each code block in markdown: - Extract language and content - Check for placeholder patterns - Verify syntax highlighting specified - Ensure examples are complete Example extraction: ```bash /plugin install my-plugin@marketplace ✅ Concrete /plugin install <plugin-name> ⚠️ Documentation (acceptable) /plugin install YOUR_PLUGIN ❌ Placeholder -
Count and Categorize Examples
Count examples by type: - Command examples: /plugin install ... - Configuration examples: JSON snippets - Code examples: Script samples - Usage examples: Real-world scenarios Quality criteria: - At least 2-3 concrete examples - Examples cover primary use cases - Examples are copy-pasteable -
Calculate Quality Score
score = 100 score -= (placeholder_instances × 10) # -10 per placeholder score -= (todo_markers × 5) # -5 per TODO/FIXME score -= (example_count < 2) ? 20 : 0 # -20 if < 2 examples score -= (incomplete_examples × 15) # -15 per incomplete example score = max(0, score) -
Format Output
Display: - Files checked count - Examples found count - Placeholders detected - Quality score - Specific issues with file/line references - Improvement recommendations
Examples
# Validate examples with strict placeholder checking (default)
/documentation-validation examples path:.
# Check only README.md (non-recursive)
/documentation-validation examples path:. recursive:false
# Allow placeholders (lenient mode)
/documentation-validation examples path:. no-placeholders:false
# Check specific file extensions
/documentation-validation examples path:. extensions:"md,sh,py"
# Strict validation of examples directory
/documentation-validation examples path:./examples no-placeholders:true recursive:true
Error Handling
Error: Placeholders detected
⚠️ WARNING: Placeholder patterns detected in examples
Placeholders found: <N> instances across <M> files
README.md:
- Line 45: /plugin install YOUR_PLUGIN_NAME
^ Should be concrete plugin name
- Line 67: API_KEY=your-api-key-here
^ Should be removed or use template syntax
examples/usage.sh:
- Line 12: # TODO: Add authentication example
^ Complete example or remove TODO
Remediation:
1. Replace "YOUR_PLUGIN_NAME" with actual plugin name
2. Use template syntax for user-provided values: ${API_KEY}
3. Remove TODO markers - complete examples or remove them
4. Provide concrete, copy-pasteable examples
Acceptable patterns:
- Template variables: ${VARIABLE}, {{variable}}
- Documentation syntax: <name> in usage descriptions
- Generic placeholders in template explanations
Error: Too few examples
⚠️ WARNING: Insufficient examples in documentation
Examples found: <N> (minimum recommended: 3)
README.md contains <N> code examples:
- Installation example ✅
- Basic usage ❌ Missing
- Advanced usage ❌ Missing
Remediation:
Add at least 2-3 concrete usage examples showing:
1. Basic usage (most common scenario)
2. Common configuration options
3. Advanced or specialized use case
Example structure:
```bash
# Basic usage
/my-plugin action param:value
# With options
/my-plugin action param:value option:true
# Advanced example
/my-plugin complex-action config:custom nested:value
Good examples are copy-pasteable and use real values.
**Error: Incomplete examples**
⚠️ WARNING: Incomplete or broken examples detected
Incomplete examples:
README.md:
- Line 34: Code block with syntax error
- Line 56: Example missing expected output
- Line 78: Example truncated with "..."
Remediation:
- Ensure all code examples are syntactically valid
- Show expected output or results after examples
- Complete truncated examples (no "..." placeholders)
- Test examples before including in documentation
Example format:
# Command with description
/plugin install example-plugin@marketplace
# Expected output:
# ✓ Installing example-plugin@marketplace
# ✓ Plugin installed successfully
**Error: Generic dummy values**
⚠️ WARNING: Generic placeholder values detected
Generic values found:
- README.md:45 - "foo", "bar" used as example values
- examples/config.json:12 - "sample" as placeholder
While "foo/bar" are common in documentation, concrete examples are more helpful for users.
Remediation: Replace generic values with realistic examples:
- Instead of "foo", use actual plugin name
- Instead of "bar", use real parameter value
- Instead of "sample", use concrete example
Good: /my-plugin process file:README.md Bad: /my-plugin process file:foo.txt
### Output Format
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ EXAMPLE QUALITY VALIDATION ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Files Checked: Code Examples Found:
Example Count by Type:
- Command examples: ✅
- Configuration examples: ✅
- Usage examples: ⚠️ (recommend 3+)
Placeholder Detection: TODO/FIXME markers: ❌ Placeholder patterns: ❌ Generic values (foo/bar): ⚠️
Quality Score: <0-100>/100
Issues by File: README.md: issues ├─ Line 45: YOUR_PLUGIN_NAME (placeholder) ├─ Line 67: TODO marker └─ Line 89: Generic "foo" value
examples/usage.sh: issues └─ Line 12: Incomplete example
Recommendations:
- Replace placeholder patterns with concrete values [+10 pts]
- Complete or remove TODO markers [+5 pts]
- Add more usage examples [+15 pts]
Overall: <PASS|WARNINGS|FAIL>
### Integration
This operation is invoked by:
- `/documentation-validation examples path:.` (direct)
- `/documentation-validation full-docs path:.` (as part of complete validation)
- `/validation-orchestrator comprehensive path:.` (via orchestrator)
Results contribute to documentation quality score:
- High-quality examples (90+): +10 points
- Some issues (60-89): +5 points
- Poor quality (<60): 0 points
- Missing examples: -10 points
### Special Cases
**Template Documentation**:
If the plugin provides templates or scaffolding, some placeholders
are acceptable when properly documented as template variables.
Example:
```markdown
The generated code includes template variables:
- {{PROJECT_NAME}} - Will be replaced with actual project name
- {{AUTHOR}} - Will be replaced with author information
This is acceptable because the placeholders are documented as intentional template syntax.
Request: $ARGUMENTS