Initial commit
This commit is contained in:
317
references/documentation-coverage-analysis.md
Normal file
317
references/documentation-coverage-analysis.md
Normal file
@@ -0,0 +1,317 @@
|
||||
# Documentation Coverage Analysis
|
||||
|
||||
## Purpose
|
||||
|
||||
Provide context-aware documentation coverage assessment based on actual extension features rather than static file count thresholds.
|
||||
|
||||
## Problem with Static Thresholds
|
||||
|
||||
Traditional conformance scoring uses fixed RST file count thresholds:
|
||||
- 50-99 files: +1 point
|
||||
- 100-149 files: +2 points
|
||||
- 150+ files: +3 points
|
||||
|
||||
**This approach is flawed** because:
|
||||
1. Extension scope varies dramatically (focused vs. comprehensive)
|
||||
2. File count doesn't reflect feature coverage quality
|
||||
3. Penalizes well-scoped extensions with complete user documentation
|
||||
4. Based on large CMS extensions (georgringer/news) as "excellence" reference
|
||||
|
||||
## Improved Methodology: Feature Coverage Analysis
|
||||
|
||||
### Step 1: Identify Extension Features
|
||||
|
||||
Categorize features into two groups:
|
||||
|
||||
**User-Facing Features:**
|
||||
- Installation & configuration
|
||||
- Backend modules & dashboards
|
||||
- End-user workflows
|
||||
- Integration guides
|
||||
- Troubleshooting
|
||||
|
||||
**Developer Features:**
|
||||
- CLI commands
|
||||
- Scheduler tasks
|
||||
- Reports/widgets
|
||||
- Event listeners
|
||||
- PHP API (public methods)
|
||||
- Extension points
|
||||
|
||||
### Step 2: Assess Documentation Coverage
|
||||
|
||||
For each feature category, calculate coverage percentage:
|
||||
|
||||
```
|
||||
User Coverage = Documented User Features / Total User Features
|
||||
Developer Coverage = Documented Developer Features / Total Developer Features
|
||||
Overall Coverage = (User + Developer) / (Total User + Total Developer)
|
||||
```
|
||||
|
||||
### Step 3: Quality Assessment
|
||||
|
||||
Evaluate documentation quality beyond mere existence:
|
||||
|
||||
**Quality Indicators:**
|
||||
- ✅ Uses TYPO3 directives (confval, versionadded, php:method)
|
||||
- ✅ Includes code examples and integration patterns
|
||||
- ✅ Modern tooling (guides.xml, card-grid navigation)
|
||||
- ✅ Proper cross-references and interlinking
|
||||
- ✅ Screenshots or visual aids where appropriate
|
||||
|
||||
### Step 4: Context-Aware Scoring
|
||||
|
||||
Score based on extension scope and feature coverage:
|
||||
|
||||
**Small/Focused Extensions (10-30 classes):**
|
||||
- User coverage 100% + modern tooling = EXCELLENT (3-4 points)
|
||||
- User coverage 80-99% = GOOD (2-3 points)
|
||||
- User coverage 60-79% = ADEQUATE (1-2 points)
|
||||
- User coverage <60% = INSUFFICIENT (0-1 points)
|
||||
|
||||
**Medium Extensions (31-100 classes):**
|
||||
- User coverage 100% + developer coverage 80%+ = EXCELLENT (3-4 points)
|
||||
- User coverage 100% + developer coverage 40-79% = GOOD (2-3 points)
|
||||
- User coverage 80-99% = ADEQUATE (1-2 points)
|
||||
- User coverage <80% = INSUFFICIENT (0-1 points)
|
||||
|
||||
**Large Extensions (100+ classes):**
|
||||
- Comprehensive documentation (user + developer 90%+) = EXCELLENT (3-4 points)
|
||||
- Good user docs + partial developer docs = GOOD (2-3 points)
|
||||
- User docs 80%+ = ADEQUATE (1-2 points)
|
||||
- User docs <80% = INSUFFICIENT (0-1 points)
|
||||
|
||||
## Analysis Workflow
|
||||
|
||||
### 1. Count Extension Features
|
||||
|
||||
```bash
|
||||
# User-facing features
|
||||
echo "User Features:"
|
||||
echo " - Installation: $(ls -1 Documentation/Installation/*.rst 2>/dev/null | wc -l)"
|
||||
echo " - Configuration: $(ls -1 Documentation/Configuration/*.rst 2>/dev/null | wc -l)"
|
||||
echo " - Backend Module: $(ls -1 Documentation/Backend/*.rst 2>/dev/null | wc -l)"
|
||||
echo " - Guides: $(ls -1 Documentation/Guides/*.rst 2>/dev/null | wc -l)"
|
||||
|
||||
# Developer features
|
||||
echo "Developer Features:"
|
||||
echo " - CLI Commands: $(find Classes/Command -name "*.php" 2>/dev/null | wc -l)"
|
||||
echo " - Scheduler Tasks: $(find Classes/Task -name "*.php" 2>/dev/null | wc -l)"
|
||||
echo " - Reports: $(find Classes/Report -name "*.php" 2>/dev/null | wc -l)"
|
||||
echo " - Event Listeners: $(find Classes/EventListener -name "*.php" 2>/dev/null | wc -l)"
|
||||
```
|
||||
|
||||
### 2. Map Documentation to Features
|
||||
|
||||
Create feature-to-documentation mapping:
|
||||
|
||||
```
|
||||
User Features:
|
||||
✓ Installation & Setup → Documentation/Installation/Index.rst
|
||||
✓ Configuration → Documentation/Configuration/*.rst
|
||||
✓ Backend Module → Documentation/Backend/*.rst
|
||||
⚠️ Troubleshooting → Documentation/Troubleshooting/Index.rst (partial)
|
||||
|
||||
Developer Features:
|
||||
⚠️ CLI: FlushCachesCommand → mentioned but no API reference
|
||||
⚠️ CLI: ShowTransitionCommand → mentioned but no API reference
|
||||
❌ Task: FlushExpiredCachesTask → not documented
|
||||
❌ Report: CacheStatusReport → not documented
|
||||
❌ EventListener: CacheLifetimeListener → not documented
|
||||
```
|
||||
|
||||
### 3. Calculate Coverage Scores
|
||||
|
||||
```
|
||||
User Coverage: 3/4 features = 75%
|
||||
Developer Coverage: 0/5 features = 0%
|
||||
Overall Coverage: 3/9 features = 33%
|
||||
```
|
||||
|
||||
### 4. Assess Quality
|
||||
|
||||
```
|
||||
Quality Indicators:
|
||||
✅ Modern tooling (guides.xml)
|
||||
✅ TYPO3 directives (confval)
|
||||
✅ Card-grid navigation
|
||||
✅ Code examples
|
||||
⚠️ Screenshots mentioned but not included
|
||||
|
||||
Quality Score: 4/5 = 80%
|
||||
```
|
||||
|
||||
### 5. Determine Final Rating
|
||||
|
||||
For a **focused extension** (30 classes):
|
||||
- User coverage: 75% (3/4) → ADEQUATE
|
||||
- Developer coverage: 0% (0/5) → gap acceptable for user-focused extensions
|
||||
- Quality: 80% → HIGH
|
||||
- Modern tooling: YES
|
||||
|
||||
**Final Rating: GOOD (2-3 points)**
|
||||
- User documentation is COMPREHENSIVE for critical features
|
||||
- Developer API documentation is a nice-to-have
|
||||
- Quality is HIGH with modern TYPO3 13.x patterns
|
||||
|
||||
## Comparison: Static vs. Feature-Based Scoring
|
||||
|
||||
### Example Extension Analysis
|
||||
|
||||
**Extension:** Temporal Cache (focused, 30 classes)
|
||||
|
||||
**Static Threshold Scoring:**
|
||||
```
|
||||
RST Files: 22
|
||||
Threshold: Need 50+ for points
|
||||
Score: 1/4 (modern tooling bonus only)
|
||||
Rating: INSUFFICIENT
|
||||
```
|
||||
|
||||
**Feature-Based Scoring:**
|
||||
```
|
||||
User Features: 6/6 = 100% coverage ✅
|
||||
Developer Features: 0/5 = 0% coverage (acceptable for scope)
|
||||
Quality: 4/5 = 80% (modern tooling, directives, examples)
|
||||
Extension Scope: Focused (30 classes)
|
||||
|
||||
Score: 3/4 points
|
||||
Rating: EXCELLENT for scope
|
||||
```
|
||||
|
||||
### Key Insight
|
||||
|
||||
**22 RST files can be EXCELLENT** for a focused extension with:
|
||||
- 100% user feature coverage
|
||||
- High-quality modern documentation
|
||||
- Proper TYPO3 directives and examples
|
||||
- Appropriate scope matching
|
||||
|
||||
**150+ RST files may be INSUFFICIENT** for a large CMS extension with:
|
||||
- Incomplete feature coverage
|
||||
- Missing integration guides
|
||||
- No API reference
|
||||
- Outdated patterns
|
||||
|
||||
## Recommendations for TYPO3 Conformance Skill
|
||||
|
||||
### Update Scoring Logic
|
||||
|
||||
Replace static thresholds with feature-based analysis:
|
||||
|
||||
```python
|
||||
def calculate_documentation_score(extension):
|
||||
"""
|
||||
Calculate documentation excellence score based on feature coverage.
|
||||
|
||||
Returns: 0-4 points
|
||||
"""
|
||||
# Determine extension scope
|
||||
class_count = count_php_classes(extension)
|
||||
scope = classify_scope(class_count) # small/medium/large
|
||||
|
||||
# Calculate feature coverage
|
||||
user_coverage = calculate_user_feature_coverage(extension)
|
||||
dev_coverage = calculate_developer_feature_coverage(extension)
|
||||
|
||||
# Assess quality
|
||||
quality_score = assess_documentation_quality(extension)
|
||||
|
||||
# Score based on scope
|
||||
if scope == "small":
|
||||
if user_coverage >= 0.90 and quality_score >= 0.80:
|
||||
return 3 # EXCELLENT
|
||||
elif user_coverage >= 0.75:
|
||||
return 2 # GOOD
|
||||
elif user_coverage >= 0.60:
|
||||
return 1 # ADEQUATE
|
||||
else:
|
||||
return 0 # INSUFFICIENT
|
||||
|
||||
elif scope == "medium":
|
||||
if user_coverage >= 0.90 and dev_coverage >= 0.80:
|
||||
return 4 # OUTSTANDING
|
||||
elif user_coverage >= 0.90 and dev_coverage >= 0.40:
|
||||
return 3 # EXCELLENT
|
||||
elif user_coverage >= 0.80:
|
||||
return 2 # GOOD
|
||||
else:
|
||||
return 1 if user_coverage >= 0.60 else 0
|
||||
|
||||
else: # large
|
||||
total_coverage = (user_coverage + dev_coverage) / 2
|
||||
if total_coverage >= 0.90:
|
||||
return 4 # OUTSTANDING
|
||||
elif total_coverage >= 0.75:
|
||||
return 3 # EXCELLENT
|
||||
elif total_coverage >= 0.60:
|
||||
return 2 # GOOD
|
||||
else:
|
||||
return 1 if total_coverage >= 0.40 else 0
|
||||
```
|
||||
|
||||
### Provide Clear Feedback
|
||||
|
||||
Documentation assessment should include:
|
||||
1. Feature coverage breakdown (user vs. developer)
|
||||
2. Quality assessment (directives, examples, tooling)
|
||||
3. Scope-appropriate recommendations
|
||||
4. Specific missing documentation items
|
||||
|
||||
### Example Output
|
||||
|
||||
```
|
||||
## Documentation Excellence Assessment
|
||||
|
||||
**Extension Scope:** Small/Focused (30 classes)
|
||||
|
||||
**User Feature Coverage:** 6/6 (100%) ✅
|
||||
✓ Installation & Setup
|
||||
✓ Configuration (3 strategies)
|
||||
✓ Backend Module
|
||||
✓ Performance Guide
|
||||
✓ Architecture
|
||||
✓ Phases Roadmap
|
||||
|
||||
**Developer Feature Coverage:** 0/5 (0%) ⚠️
|
||||
❌ CLI Commands API reference (2 commands)
|
||||
❌ Scheduler Tasks reference (1 task)
|
||||
❌ Reports reference (1 report)
|
||||
❌ EventListener docs
|
||||
❌ PHP API method-level docs
|
||||
|
||||
**Quality Assessment:** 4/5 (80%) ✅
|
||||
✅ Modern tooling (guides.xml, card-grid)
|
||||
✅ TYPO3 directives (confval throughout)
|
||||
✅ Code examples extensive
|
||||
✅ Cross-references proper
|
||||
⚠️ Screenshots mentioned but not included
|
||||
|
||||
**Score:** 3/4 points (EXCELLENT for extension scope)
|
||||
|
||||
**Recommendation:**
|
||||
Your user documentation is COMPREHENSIVE (100% coverage). Developer API
|
||||
documentation is optional for this extension's scope. Consider adding
|
||||
API reference if you expect other developers to extend your extension.
|
||||
|
||||
**Impact of Adding Developer Docs:**
|
||||
- Current: 22 RST files (100% user coverage, 0% developer)
|
||||
- With API reference: ~30 RST files (100% user + 100% developer)
|
||||
- Score change: 3/4 → 4/4 (OUTSTANDING)
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
**Key Principles:**
|
||||
1. Documentation quality > file quantity
|
||||
2. Feature coverage > arbitrary thresholds
|
||||
3. Scope-appropriate expectations
|
||||
4. User-facing docs prioritized over developer API docs
|
||||
5. Context-aware scoring avoids penalizing focused extensions
|
||||
|
||||
**Benefits:**
|
||||
- Accurate assessment of documentation completeness
|
||||
- Fair scoring across extension scopes
|
||||
- Actionable recommendations
|
||||
- Recognizes excellent documentation regardless of file count
|
||||
- Aligns with TYPO3 documentation best practices
|
||||
Reference in New Issue
Block a user