422 lines
12 KiB
Markdown
422 lines
12 KiB
Markdown
# runTests.sh Validation Guide
|
|
|
|
**Purpose:** Validate Build/Scripts/runTests.sh against TYPO3 Best Practices (Tea extension reference)
|
|
|
|
## Why Validate runTests.sh?
|
|
|
|
The `runTests.sh` script is the **central orchestration tool** for TYPO3 extension quality workflows. An outdated or misconfigured script can lead to:
|
|
|
|
- ❌ Testing with wrong PHP/TYPO3 versions (false positives/negatives)
|
|
- ❌ Missing database compatibility issues
|
|
- ❌ Inconsistent local vs CI environments
|
|
- ❌ Developer confusion with incorrect defaults
|
|
|
|
## Reference Implementation
|
|
|
|
**Source of Truth:** https://github.com/TYPO3BestPractices/tea/blob/main/Build/Scripts/runTests.sh
|
|
|
|
The Tea extension maintains the canonical runTests.sh implementation, updated for latest TYPO3 standards.
|
|
|
|
## Critical Validation Points
|
|
|
|
### 1. PHP Version Configuration
|
|
|
|
**Check Lines ~318 and ~365:**
|
|
|
|
```bash
|
|
# Default PHP version
|
|
PHP_VERSION="X.X"
|
|
|
|
# PHP version validation regex
|
|
if ! [[ ${PHP_VERSION} =~ ^(X.X|X.X|X.X)$ ]]; then
|
|
```
|
|
|
|
**Validation:**
|
|
1. Read extension's composer.json `require.php` constraint
|
|
2. Extract minimum PHP version (e.g., `^8.2` → minimum 8.2)
|
|
3. Verify runTests.sh default matches minimum
|
|
4. Verify version regex includes all supported versions
|
|
|
|
**Example Check:**
|
|
|
|
```bash
|
|
# Extension composer.json
|
|
"require": {
|
|
"php": "^8.2 || ^8.3 || ^8.4"
|
|
}
|
|
|
|
# runTests.sh SHOULD have:
|
|
PHP_VERSION="8.2" # ✅ Matches minimum
|
|
if ! [[ ${PHP_VERSION} =~ ^(8.2|8.3|8.4)$ ]]; then # ✅ All supported
|
|
|
|
# runTests.sh SHOULD NOT have:
|
|
PHP_VERSION="7.4" # ❌ Below minimum
|
|
if ! [[ ${PHP_VERSION} =~ ^(7.4|8.0|8.1|8.2|8.3)$ ]]; then # ❌ Includes unsupported
|
|
```
|
|
|
|
**Severity:** 🔴 **High** - Testing with wrong PHP version invalidates results
|
|
|
|
### 2. TYPO3 Version Configuration
|
|
|
|
**Check Lines ~315 and ~374:**
|
|
|
|
```bash
|
|
# Default TYPO3 version
|
|
TYPO3_VERSION="XX"
|
|
|
|
# TYPO3 version validation
|
|
if ! [[ ${TYPO3_VERSION} =~ ^(11|12|13)$ ]]; then
|
|
```
|
|
|
|
**Validation:**
|
|
1. Read extension's composer.json TYPO3 core dependency
|
|
2. Extract target TYPO3 version (e.g., `^13.4` → TYPO3 13)
|
|
3. Verify runTests.sh default matches target
|
|
4. Check composerInstallHighest/Lowest version constraints
|
|
|
|
**Example Check:**
|
|
|
|
```bash
|
|
# Extension composer.json
|
|
"require": {
|
|
"typo3/cms-core": "^13.4"
|
|
}
|
|
|
|
# runTests.sh SHOULD have:
|
|
TYPO3_VERSION="13" # ✅ Matches target
|
|
|
|
# In composerInstallHighest (line ~530):
|
|
if [ ${TYPO3_VERSION} -eq 13 ]; then
|
|
composer require --no-ansi --no-interaction --no-progress --no-install \
|
|
typo3/cms-core:^13.4 # ✅ Matches composer.json
|
|
|
|
# runTests.sh SHOULD NOT have:
|
|
TYPO3_VERSION="11" # ❌ Below target
|
|
```
|
|
|
|
**Severity:** 🔴 **High** - Testing against wrong TYPO3 version
|
|
|
|
### 3. Database Version Support
|
|
|
|
**Check Lines ~48-107 (handleDbmsOptions function):**
|
|
|
|
```bash
|
|
mariadb)
|
|
[ -z "${DBMS_VERSION}" ] && DBMS_VERSION="X.X"
|
|
if ! [[ ${DBMS_VERSION} =~ ^(10.2|10.3|...|11.1)$ ]]; then
|
|
```
|
|
|
|
**Validation:**
|
|
1. Check MariaDB, MySQL, PostgreSQL version lists are current
|
|
2. Verify default versions are maintained (not EOL)
|
|
3. Cross-reference with TYPO3 core database support matrix
|
|
|
|
**Current Database Support (TYPO3 13):**
|
|
|
|
| DBMS | Supported Versions | Default | EOL Status |
|
|
|------|-------------------|---------|------------|
|
|
| MariaDB | 10.4-10.11, 11.0-11.4 | 10.11 | 10.4+ maintained |
|
|
| MySQL | 8.0, 8.1, 8.2, 8.3, 8.4 | 8.0 | 8.0 maintained until 2026 |
|
|
| PostgreSQL | 10-16 | 16 | 10-11 EOL, 12+ maintained |
|
|
| SQLite | 3.x | 3.x | Always latest |
|
|
|
|
**Example Check:**
|
|
|
|
```bash
|
|
# runTests.sh MariaDB (line ~48)
|
|
[ -z "${DBMS_VERSION}" ] && DBMS_VERSION="10.11" # ✅ LTS version
|
|
if ! [[ ${DBMS_VERSION} =~ ^(10.4|10.5|10.6|10.11|11.0|11.1|11.2|11.3|11.4)$ ]]; then
|
|
|
|
# ❌ BAD - EOL version as default:
|
|
[ -z "${DBMS_VERSION}" ] && DBMS_VERSION="10.2" # EOL 2023
|
|
|
|
# runTests.sh PostgreSQL (line ~79)
|
|
[ -z "${DBMS_VERSION}" ] && DBMS_VERSION="16" # ✅ Latest stable
|
|
if ! [[ ${DBMS_VERSION} =~ ^(10|11|12|13|14|15|16)$ ]]; then
|
|
```
|
|
|
|
**Severity:** 🟡 **Medium** - May miss database-specific compatibility issues
|
|
|
|
### 4. Network Name Configuration
|
|
|
|
**Check Line ~331:**
|
|
|
|
```bash
|
|
NETWORK="extension-name-${SUFFIX}"
|
|
```
|
|
|
|
**Validation:**
|
|
1. Should match extension key or composer package name
|
|
2. Should NOT be hardcoded to "friendsoftypo3-tea" (copy-paste artifact)
|
|
|
|
**Example Check:**
|
|
|
|
```bash
|
|
# Extension key: rte_ckeditor_image
|
|
# Composer package: netresearch/rte-ckeditor-image
|
|
|
|
# ✅ Good options:
|
|
NETWORK="rte-ckeditor-image-${SUFFIX}"
|
|
NETWORK="netresearch-rte-ckeditor-image-${SUFFIX}"
|
|
|
|
# ❌ Bad (copy-paste from Tea):
|
|
NETWORK="friendsoftypo3-tea-${SUFFIX}"
|
|
```
|
|
|
|
**Severity:** 🟢 **Low** - Cosmetic, but indicates lack of customization
|
|
|
|
### 5. Test Suite Commands
|
|
|
|
**Check Lines ~580, ~620 (functional and unit test commands):**
|
|
|
|
```bash
|
|
functional)
|
|
COMMAND=(.Build/bin/phpunit -c Build/phpunit/FunctionalTests.xml ...)
|
|
|
|
unit)
|
|
COMMAND=(.Build/bin/phpunit -c Build/phpunit/UnitTests.xml ...)
|
|
```
|
|
|
|
**Validation:**
|
|
1. Paths match actual PHPUnit config locations
|
|
2. Config files exist and are properly named
|
|
3. Exclude groups match available database types
|
|
|
|
**Example Check:**
|
|
|
|
```bash
|
|
# Verify config files exist:
|
|
ls -la Build/phpunit/UnitTests.xml # Must exist
|
|
ls -la Build/phpunit/FunctionalTests.xml # Must exist
|
|
|
|
# Check command paths:
|
|
COMMAND=(.Build/bin/phpunit -c Build/phpunit/FunctionalTests.xml ...)
|
|
└─────┬─────┘ └──────────┬───────────┘
|
|
✅ Matches ✅ Matches actual
|
|
.Build/bin/ Build/phpunit/
|
|
from composer.json directory structure
|
|
```
|
|
|
|
**Severity:** 🔴 **High** - Tests won't run if paths are wrong
|
|
|
|
### 6. Container Image Versions
|
|
|
|
**Check Lines ~446-451:**
|
|
|
|
```bash
|
|
IMAGE_PHP="ghcr.io/typo3/core-testing-$(echo "php${PHP_VERSION}" | sed -e 's/\.//'):latest"
|
|
IMAGE_ALPINE="docker.io/alpine:3.8"
|
|
IMAGE_DOCS="ghcr.io/typo3-documentation/render-guides:latest"
|
|
IMAGE_MARIADB="docker.io/mariadb:${DBMS_VERSION}"
|
|
IMAGE_MYSQL="docker.io/mysql:${DBMS_VERSION}"
|
|
IMAGE_POSTGRES="docker.io/postgres:${DBMS_VERSION}-alpine"
|
|
```
|
|
|
|
**Validation:**
|
|
1. PHP testing image uses official TYPO3 images
|
|
2. Alpine version is reasonably current (not ancient)
|
|
3. Documentation renderer is latest official TYPO3 image
|
|
|
|
**Severity:** 🟢 **Low** - Usually works, but outdated Alpine may have issues
|
|
|
|
## Conformance Evaluation Workflow
|
|
|
|
### Step 1: Extract Extension Requirements
|
|
|
|
```bash
|
|
# Read composer.json
|
|
cat composer.json | jq -r '.require.php' # e.g., "^8.2 || ^8.3 || ^8.4"
|
|
cat composer.json | jq -r '.require."typo3/cms-core"' # e.g., "^13.4"
|
|
|
|
# Parse minimum versions
|
|
MIN_PHP=$(echo "^8.2 || ^8.3" | grep -oE '[0-9]+\.[0-9]+' | head -1) # 8.2
|
|
TARGET_TYPO3=$(echo "^13.4" | grep -oE '[0-9]+') # 13
|
|
```
|
|
|
|
### Step 2: Validate runTests.sh Defaults
|
|
|
|
```bash
|
|
# Check PHP version default (line ~318)
|
|
grep '^PHP_VERSION=' Build/Scripts/runTests.sh
|
|
# Expected: PHP_VERSION="8.2" (matches MIN_PHP)
|
|
|
|
# Check TYPO3 version default (line ~315)
|
|
grep '^TYPO3_VERSION=' Build/Scripts/runTests.sh
|
|
# Expected: TYPO3_VERSION="13" (matches TARGET_TYPO3)
|
|
```
|
|
|
|
### Step 3: Validate PHP Version Regex
|
|
|
|
```bash
|
|
# Extract PHP version regex (line ~365)
|
|
grep -A 2 'if ! \[\[ \${PHP_VERSION}' Build/Scripts/runTests.sh
|
|
|
|
# Expected pattern for "^8.2 || ^8.3 || ^8.4":
|
|
# ^(8.2|8.3|8.4)$
|
|
|
|
# ❌ Outdated pattern:
|
|
# ^(7.4|8.0|8.1|8.2|8.3)$
|
|
```
|
|
|
|
### Step 4: Validate TYPO3 Version Constraints in Composer Install
|
|
|
|
```bash
|
|
# Check composerInstallHighest TYPO3 13 block (line ~530)
|
|
sed -n '/if \[ \${TYPO3_VERSION} -eq 13 \];/,/fi/p' Build/Scripts/runTests.sh
|
|
|
|
# Should match composer.json requirements:
|
|
# typo3/cms-core:^13.4
|
|
# typo3/cms-backend:^13.4
|
|
# etc.
|
|
```
|
|
|
|
### Step 5: Validate Network Name
|
|
|
|
```bash
|
|
# Check network name (line ~331)
|
|
grep '^NETWORK=' Build/Scripts/runTests.sh
|
|
|
|
# Extract extension key from composer.json or ext_emconf.php
|
|
EXT_KEY=$(jq -r '.extra.typo3.cms."extension-key"' composer.json)
|
|
|
|
# Expected: NETWORK="${EXT_KEY}-${SUFFIX}" or similar
|
|
# ❌ Wrong: NETWORK="friendsoftypo3-tea-${SUFFIX}"
|
|
```
|
|
|
|
## Automated Validation Script
|
|
|
|
Create `scripts/validate-runtests.sh`:
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
|
|
set -e
|
|
|
|
echo "🔍 Validating Build/Scripts/runTests.sh against extension requirements..."
|
|
|
|
# Extract requirements
|
|
MIN_PHP=$(jq -r '.require.php' composer.json | grep -oE '[0-9]+\.[0-9]+' | head -1)
|
|
TARGET_TYPO3=$(jq -r '.require."typo3/cms-core"' composer.json | grep -oE '^[0-9]+' | head -1)
|
|
EXT_KEY=$(jq -r '.extra.typo3.cms."extension-key"' composer.json)
|
|
|
|
echo "📋 Extension Requirements:"
|
|
echo " PHP: ${MIN_PHP}+"
|
|
echo " TYPO3: ${TARGET_TYPO3}"
|
|
echo " Extension Key: ${EXT_KEY}"
|
|
echo ""
|
|
|
|
# Validate PHP version default
|
|
RUNTESTS_PHP=$(grep '^PHP_VERSION=' Build/Scripts/runTests.sh | cut -d'"' -f2)
|
|
if [ "${RUNTESTS_PHP}" != "${MIN_PHP}" ]; then
|
|
echo "❌ PHP version mismatch: runTests.sh uses ${RUNTESTS_PHP}, should be ${MIN_PHP}"
|
|
exit 1
|
|
else
|
|
echo "✅ PHP version default: ${RUNTESTS_PHP}"
|
|
fi
|
|
|
|
# Validate TYPO3 version default
|
|
RUNTESTS_TYPO3=$(grep '^TYPO3_VERSION=' Build/Scripts/runTests.sh | cut -d'"' -f2)
|
|
if [ "${RUNTESTS_TYPO3}" != "${TARGET_TYPO3}" ]; then
|
|
echo "❌ TYPO3 version mismatch: runTests.sh uses ${RUNTESTS_TYPO3}, should be ${TARGET_TYPO3}"
|
|
exit 1
|
|
else
|
|
echo "✅ TYPO3 version default: ${RUNTESTS_TYPO3}"
|
|
fi
|
|
|
|
# Validate network name
|
|
NETWORK_NAME=$(grep '^NETWORK=' Build/Scripts/runTests.sh | cut -d'"' -f2 | sed 's/-${SUFFIX}$//')
|
|
if [[ "${NETWORK_NAME}" == "friendsoftypo3-tea" ]]; then
|
|
echo "⚠️ Network name is copy-paste from Tea extension: ${NETWORK_NAME}"
|
|
echo " Should be: ${EXT_KEY}-\${SUFFIX}"
|
|
else
|
|
echo "✅ Network name: ${NETWORK_NAME}-\${SUFFIX}"
|
|
fi
|
|
|
|
echo ""
|
|
echo "✅ runTests.sh validation complete"
|
|
```
|
|
|
|
## Conformance Report Integration
|
|
|
|
### When evaluating runTests.sh:
|
|
|
|
**In "Best Practices" Section:**
|
|
|
|
```markdown
|
|
### Build Scripts
|
|
|
|
**runTests.sh Analysis:**
|
|
|
|
- ✅ Script present and executable
|
|
- ✅ PHP version default matches composer.json minimum (8.2)
|
|
- ✅ TYPO3 version default matches target (13)
|
|
- ✅ PHP version regex includes all supported versions (8.2, 8.3, 8.4)
|
|
- ⚠️ Network name uses Tea extension default (cosmetic issue)
|
|
- ✅ Test suite commands match actual file structure
|
|
- ✅ Database version support is current
|
|
|
|
**Or with issues:**
|
|
|
|
- ❌ PHP version default (7.4) below extension minimum (8.2)
|
|
- File: Build/Scripts/runTests.sh:318
|
|
- Severity: High
|
|
- Fix: Change `PHP_VERSION="7.4"` to `PHP_VERSION="8.2"`
|
|
|
|
- ❌ TYPO3 version default (11) below extension target (13)
|
|
- File: Build/Scripts/runTests.sh:315
|
|
- Severity: High
|
|
- Fix: Change `TYPO3_VERSION="11"` to `TYPO3_VERSION="13"`
|
|
|
|
- ❌ PHP version regex includes unsupported versions
|
|
- File: Build/Scripts/runTests.sh:365
|
|
- Current: `^(7.4|8.0|8.1|8.2|8.3)$`
|
|
- Expected: `^(8.2|8.3|8.4)$`
|
|
- Severity: Medium
|
|
- Fix: Remove unsupported versions from regex
|
|
```
|
|
|
|
## Scoring Impact
|
|
|
|
**Best Practices Score Deductions:**
|
|
|
|
| Issue | Severity | Score Impact |
|
|
|-------|----------|--------------|
|
|
| PHP version default outdated | High | -3 points |
|
|
| TYPO3 version default outdated | High | -3 points |
|
|
| PHP version regex includes unsupported | Medium | -2 points |
|
|
| Database versions EOL | Medium | -2 points |
|
|
| Network name copy-paste | Low | -1 point |
|
|
| Missing runTests.sh | Critical | -10 points |
|
|
|
|
**Maximum deduction for runTests.sh issues:** -6 points (out of 20 for Best Practices)
|
|
|
|
## Quick Reference Checklist
|
|
|
|
**When evaluating Build/Scripts/runTests.sh:**
|
|
|
|
```
|
|
□ File exists and is executable
|
|
□ PHP_VERSION default matches composer.json minimum
|
|
□ TYPO3_VERSION default matches composer.json target
|
|
□ PHP version regex matches composer.json constraint exactly
|
|
□ TYPO3_VERSION regex includes supported versions only
|
|
□ Database version lists are current (not EOL)
|
|
□ Database version defaults are maintained LTS versions
|
|
□ Network name is customized (not "friendsoftypo3-tea")
|
|
□ Test suite paths match actual directory structure
|
|
□ Container images use official TYPO3 testing images
|
|
```
|
|
|
|
**Comparison Strategy:**
|
|
|
|
1. Download latest Tea runTests.sh as reference
|
|
2. Compare line-by-line for structural differences
|
|
3. Validate version-specific values against extension requirements
|
|
4. Flag any outdated patterns or hardcoded Tea-specific values
|
|
|
|
## Resources
|
|
|
|
- **Tea Extension runTests.sh:** https://github.com/TYPO3BestPractices/tea/blob/main/Build/Scripts/runTests.sh
|
|
- **TYPO3 Testing Documentation:** https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/Testing/
|
|
- **Database Compatibility:** https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Database/
|