gh-netresearch-claude-code-marketplace-netresearch-skills-bundle/skills/typo3-testing/references/test-runners.md

Test Runners and Orchestration

The runTests.sh script is the standard TYPO3 pattern for orchestrating all quality checks and test suites.

Purpose

• Single entry point for all testing and quality checks
• Consistent environment across local and CI/CD
• Handles Docker, database setup, and test execution
• Based on TYPO3 Best Practices tea extension

Script Location

Build/Scripts/runTests.sh

Basic Usage

# Show help
./Build/Scripts/runTests.sh -h

# Run specific test suite
./Build/Scripts/runTests.sh -s unit
./Build/Scripts/runTests.sh -s functional
./Build/Scripts/runTests.sh -s acceptance

# Run quality tools
./Build/Scripts/runTests.sh -s lint
./Build/Scripts/runTests.sh -s phpstan
./Build/Scripts/runTests.sh -s cgl
./Build/Scripts/runTests.sh -s rector

Script Options

-s <suite>     Test suite to run (required)
               unit, functional, acceptance, lint, phpstan, cgl, rector

-d <driver>    Database driver for functional tests
               mysqli (default), pdo_mysql, postgres, sqlite

-p <version>   PHP version (7.4, 8.1, 8.2, 8.3)

-e <command>   Execute specific command in container

-n             Don't pull Docker images

-u             Update composer dependencies

-v             Enable verbose output

-x             Stop on first error (PHPUnit --stop-on-error)

Examples

Run Unit Tests

# Default PHP version
./Build/Scripts/runTests.sh -s unit

# Specific PHP version
./Build/Scripts/runTests.sh -s unit -p 8.3

# Stop on first error
./Build/Scripts/runTests.sh -s unit -x

Run Functional Tests

# Default database (mysqli)
./Build/Scripts/runTests.sh -s functional

# PostgreSQL
./Build/Scripts/runTests.sh -s functional -d postgres

# SQLite (fastest for local development)
./Build/Scripts/runTests.sh -s functional -d sqlite

Run Quality Tools

# Lint all PHP files
./Build/Scripts/runTests.sh -s lint

# PHPStan static analysis
./Build/Scripts/runTests.sh -s phpstan

# Code style check
./Build/Scripts/runTests.sh -s cgl

# Rector automated refactoring
./Build/Scripts/runTests.sh -s rector

Custom Commands

# Run specific test file
./Build/Scripts/runTests.sh -s unit -e "bin/phpunit Tests/Unit/Domain/Model/ProductTest.php"

# Run with coverage
./Build/Scripts/runTests.sh -s unit -e "bin/phpunit --coverage-html coverage/"

Composer Integration

Integrate runTests.sh into composer.json:

{
    "scripts": {
        "ci:test": [
            "@ci:test:php:lint",
            "@ci:test:php:phpstan",
            "@ci:test:php:cgl",
            "@ci:test:php:rector",
            "@ci:test:php:unit",
            "@ci:test:php:functional"
        ],
        "ci:test:php:lint": "Build/Scripts/runTests.sh -s lint",
        "ci:test:php:phpstan": "Build/Scripts/runTests.sh -s phpstan",
        "ci:test:php:cgl": "Build/Scripts/runTests.sh -s cgl",
        "ci:test:php:rector": "Build/Scripts/runTests.sh -s rector",
        "ci:test:php:unit": "Build/Scripts/runTests.sh -s unit",
        "ci:test:php:functional": "Build/Scripts/runTests.sh -s functional"
    }
}

Then run via composer:

composer ci:test              # All checks
composer ci:test:php:unit     # Just unit tests
composer ci:test:php:phpstan  # Just PHPStan

Script Structure

Basic Template

#!/usr/bin/env bash

# Script configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"

# Default values
TEST_SUITE=""
DATABASE_DRIVER="mysqli"
PHP_VERSION="8.4"
VERBOSE=""

# Parse arguments
while getopts ":s:d:p:e:nuvx" opt; do
    case ${opt} in
        s) TEST_SUITE=${OPTARG} ;;
        d) DATABASE_DRIVER=${OPTARG} ;;
        p) PHP_VERSION=${OPTARG} ;;
        *) showHelp; exit 1 ;;
    esac
done

# Validate required arguments
if [ -z "${TEST_SUITE}" ]; then
    echo "Error: -s parameter (test suite) is required"
    showHelp
    exit 1
fi

# Execute test suite
case ${TEST_SUITE} in
    unit)
        runUnitTests
        ;;
    functional)
        runFunctionalTests
        ;;
    lint)
        runLint
        ;;
    *)
        echo "Error: Unknown test suite: ${TEST_SUITE}"
        showHelp
        exit 1
        ;;
esac

Docker Integration

runUnitTests() {
    CONTAINER_PATH="/app"

    docker run \
        --rm \
        -v "${PROJECT_DIR}:${CONTAINER_PATH}" \
        -w "${CONTAINER_PATH}" \
        php:${PHP_VERSION}-cli \
        bin/phpunit -c Build/phpunit/UnitTests.xml
}

runFunctionalTests() {
    CONTAINER_PATH="/app"

    docker run \
        --rm \
        -v "${PROJECT_DIR}:${CONTAINER_PATH}" \
        -w "${CONTAINER_PATH}" \
        -e typo3DatabaseDriver="${DATABASE_DRIVER}" \
        -e typo3DatabaseHost="localhost" \
        -e typo3DatabaseName="typo3_test" \
        php:${PHP_VERSION}-cli \
        bin/phpunit -c Build/phpunit/FunctionalTests.xml
}

Quality Tool Functions

runLint() {
    docker run \
        --rm \
        -v "${PROJECT_DIR}:/app" \
        -w /app \
        php:${PHP_VERSION}-cli \
        vendor/bin/phplint
}

runPhpstan() {
    docker run \
        --rm \
        -v "${PROJECT_DIR}:/app" \
        -w /app \
        php:${PHP_VERSION}-cli \
        vendor/bin/phpstan analyze --configuration Build/phpstan.neon
}

runCgl() {
    docker run \
        --rm \
        -v "${PROJECT_DIR}:/app" \
        -w /app \
        php:${PHP_VERSION}-cli \
        vendor/bin/php-cs-fixer fix --config Build/php-cs-fixer.php --dry-run --diff
}

Environment Variables

Configure via environment variables:

# Database configuration
export typo3DatabaseDriver=pdo_mysql
export typo3DatabaseHost=db
export typo3DatabasePort=3306
export typo3DatabaseName=typo3_test
export typo3DatabaseUsername=root
export typo3DatabasePassword=root

# TYPO3 context
export TYPO3_CONTEXT=Testing

# Run tests
./Build/Scripts/runTests.sh -s functional

CI/CD Integration

GitHub Actions

name: Tests

on: [push, pull_request]

jobs:
  tests:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        php: '8.2', '8.2', '8.3']
        suite: ['unit', 'functional', 'lint', 'phpstan']

    steps:
      - uses: actions/checkout@v4

      - name: Run ${{ matrix.suite }} tests
        run: Build/Scripts/runTests.sh -s ${{ matrix.suite }} -p ${{ matrix.php }}

GitLab CI

.test:
  image: php:${PHP_VERSION}-cli
  script:
    - Build/Scripts/runTests.sh -s ${TEST_SUITE} -p ${PHP_VERSION}

unit:8.2:
  extends: .test
  variables:
    PHP_VERSION: "8.2"
    TEST_SUITE: "unit"

functional:8.2:
  extends: .test
  variables:
    PHP_VERSION: "8.2"
    TEST_SUITE: "functional"

Performance Optimization

Parallel Execution

# Run linting in parallel (fast)
find . -name '*.php' -print0 | xargs -0 -n1 -P8 php -l

# PHPUnit parallel execution
vendor/bin/paratest -c Build/phpunit/UnitTests.xml --processes=4

Caching

# Cache Composer dependencies
if [ ! -d "${PROJECT_DIR}/.cache/composer" ]; then
    mkdir -p "${PROJECT_DIR}/.cache/composer"
fi

docker run \
    --rm \
    -v "${PROJECT_DIR}:/app" \
    -v "${PROJECT_DIR}/.cache/composer:/tmp/composer-cache" \
    php:${PHP_VERSION}-cli \
    composer install --no-progress --no-suggest

Best Practices

1. Single Source of Truth: Use runTests.sh for all test execution
2. CI/CD Alignment: CI should use same script as local development
3. Docker Isolation: Run tests in containers for consistency
4. Fast Feedback: Run lint and unit tests first (fastest)
5. Matrix Testing: Test multiple PHP versions and databases
6. Caching: Cache dependencies to speed up execution
7. Verbose Mode: Use -v flag for debugging test failures

Troubleshooting

Docker Permission Issues

# Run with current user
docker run \
    --rm \
    --user $(id -u):$(id -g) \
    -v "${PROJECT_DIR}:/app" \
    php:${PHP_VERSION}-cli \
    bin/phpunit

Database Connection Errors

# Verify database is accessible
docker run --rm --network host mysql:8.0 \
    mysql -h localhost -u root -p -e "SELECT 1"

# Use SQLite for simple tests
./Build/Scripts/runTests.sh -s functional -d sqlite

Missing Dependencies

# Update dependencies
./Build/Scripts/runTests.sh -s unit -u

Resources

• TYPO3 Tea Extension runTests.sh
• TYPO3 Testing Documentation
• PHPUnit Documentation

Multi-PHP Version Testing

TYPO3 extensions should support multiple PHP versions to maximize compatibility. This section covers testing strategies for all supported PHP versions.

Standard Approach: Using Build/Scripts/runTests.sh

The runTests.sh script with Docker containers is the recommended TYPO3 testing approach. It provides isolated environments and consistent results.

Testing Across PHP Versions with Docker

# Test with PHP 8.2
./Build/Scripts/runTests.sh -s unit -p 8.2
./Build/Scripts/runTests.sh -s functional -p 8.2 -d sqlite

# Test with PHP 8.3
./Build/Scripts/runTests.sh -s unit -p 8.3
./Build/Scripts/runTests.sh -s functional -p 8.3 -d sqlite

# Test with PHP 8.4
./Build/Scripts/runTests.sh -s unit -p 8.4
./Build/Scripts/runTests.sh -s functional -p 8.4 -d sqlite

Advantages:

• Isolated Docker containers per PHP version
• Consistent environment across local and CI/CD
• No need to install multiple PHP versions locally
• Handles database setup automatically
• Based on official TYPO3 best practices

Complete Test Matrix Example

#!/bin/bash
# Test all supported PHP versions and databases

PHP_VERSIONS=("8.2" "8.3" "8.4")
DATABASES=("sqlite" "mysql" "postgres")

for PHP in "${PHP_VERSIONS[@]}"; do
    echo "Testing PHP ${PHP}..."
    
    # Unit tests
    ./Build/Scripts/runTests.sh -s unit -p "${PHP}"
    
    # Functional tests with different databases
    for DB in "${DATABASES[@]}"; do
        echo "  Functional tests with ${DB}..."
        ./Build/Scripts/runTests.sh -s functional -p "${PHP}" -d "${DB}"
    done
    
    # Quality tools (run once per PHP version)
    ./Build/Scripts/runTests.sh -s lint -p "${PHP}"
    ./Build/Scripts/runTests.sh -s phpstan -p "${PHP}"
done

Alternative Approach: Native PHP Versions (Without Docker)

For CI/CD environments or when Docker is unavailable, use locally installed PHP versions.

Testing with Native PHP Installations

# Test with PHP 8.2
php8.2 /usr/local/bin/composer update --no-interaction
php8.2 .Build/bin/phpunit --configuration=Build/phpunit/UnitTests.xml
typo3DatabaseDriver=pdo_sqlite php8.2 .Build/bin/phpunit --configuration=Build/phpunit/FunctionalTests.xml

# Test with PHP 8.3
php8.3 /usr/local/bin/composer update --no-interaction
php8.3 .Build/bin/phpunit --configuration=Build/phpunit/UnitTests.xml
typo3DatabaseDriver=pdo_sqlite php8.3 .Build/bin/phpunit --configuration=Build/phpunit/FunctionalTests.xml

# Test with PHP 8.4
php8.4 /usr/local/bin/composer update --no-interaction
php8.4 .Build/bin/phpunit --configuration=Build/phpunit/UnitTests.xml
typo3DatabaseDriver=pdo_sqlite php8.4 .Build/bin/phpunit --configuration=Build/phpunit/FunctionalTests.xml

CRITICAL: Always run composer update with the target PHP version FIRST. This ensures:

• Correct PHPUnit version selection (PHPUnit 11 for PHP 8.2, PHPUnit 12 for PHP 8.3+)
• Proper dependency resolution for the PHP version
• Compatible autoloader generation

PHPUnit Version Compatibility

The typo3/testing-framework supports both PHPUnit 11 and 12, allowing Composer to automatically select the compatible version:

PHP Version	PHPUnit Version	Auto-Selected by Composer
8.2	11.x	✅ Yes
8.3	11.x or 12.x	✅ Yes (12.x preferred)
8.4	11.x or 12.x	✅ Yes (12.x preferred)

Example: Automatic PHPUnit Selection

# PHP 8.2 automatically gets PHPUnit 11
$ php8.2 /usr/local/bin/composer update
# Installing typo3/testing-framework (v8.0.14)
# Installing phpunit/phpunit (11.5.42)

# PHP 8.3 automatically gets PHPUnit 12
$ php8.3 /usr/local/bin/composer update
# Installing typo3/testing-framework (v8.0.14)
# Installing phpunit/phpunit (12.4.1)

GitHub Actions CI/CD Integration

Using runTests.sh (Recommended)

name: CI

on: [push, pull_request]

jobs:
  tests:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        php: ['8.2', '8.3', '8.4']
        suite: ['unit', 'functional']
        database: ['sqlite', 'mysql', 'postgres']
        exclude:
          # Only test sqlite for unit tests
          - suite: unit
            database: mysql
          - suite: unit
            database: postgres

    steps:
      - uses: actions/checkout@v4

      - name: Run ${{ matrix.suite }} tests on PHP ${{ matrix.php }}
        run: |
          Build/Scripts/runTests.sh \
            -s ${{ matrix.suite }} \
            -p ${{ matrix.php }} \
            -d ${{ matrix.database }}

Using Native PHP (Alternative)

name: CI

on: [push, pull_request]

jobs:
  tests:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        php: ['8.2', '8.3', '8.4']

    steps:
      - uses: actions/checkout@v4

      - name: Set up PHP ${{ matrix.php }}
        uses: shivammathur/setup-php@v2
        with:
          php-version: ${{ matrix.php }}
          extensions: dom, libxml, sqlite3
          coverage: none

      - name: Install dependencies
        run: composer update --no-interaction --no-progress

      - name: Run unit tests
        run: .Build/bin/phpunit -c Build/phpunit/UnitTests.xml

      - name: Run functional tests
        env:
          typo3DatabaseDriver: pdo_sqlite
        run: .Build/bin/phpunit -c Build/phpunit/FunctionalTests.xml

Common Pitfalls

❌ Wrong: Testing Without Updating Dependencies

# This will use wrong PHPUnit version
php8.2 .Build/bin/phpunit -c Build/phpunit/UnitTests.xml

Problem: Uses PHPUnit version from previous composer update, may be incompatible.

✅ Right: Update Dependencies First

# This ensures correct PHPUnit version
php8.2 /usr/local/bin/composer update --no-interaction
php8.2 .Build/bin/phpunit -c Build/phpunit/UnitTests.xml

❌ Wrong: Removing PHP Version Support Due to Test Failures

# Don't do this!
# composer.json: "php": "^8.3 || ^8.4"  # Removed 8.2

Problem: Unnecessarily reduces compatibility. Fix the testing approach instead.

✅ Right: Fix Testing Process

# Run composer update with the problematic PHP version
php8.2 /usr/local/bin/composer update
# Composer automatically selects compatible dependencies
# Now tests work correctly

Best Practices

1. Standard Approach First: Use Build/Scripts/runTests.sh with Docker whenever possible
2. Update Before Testing: Always run composer update with target PHP version first (native approach)
3. Trust Composer: Let Composer select compatible PHPUnit versions automatically
4. Test All Versions: Run full test suite on all supported PHP versions in CI
5. SQLite for Speed: Use SQLite for local functional testing (fastest)
6. Matrix Testing: Use CI matrix to test all PHP versions in parallel
7. Don't Remove Support: Fix testing process, don't remove PHP version support

Troubleshooting

Docker Issues

# Update Docker images
./Build/Scripts/runTests.sh -u

# Check Docker is running
docker ps

# Clean up old containers
docker system prune -a

Native PHP Issues

# Verify PHP version
php8.2 --version

# Check installed extensions
php8.2 -m | grep -E 'dom|libxml|sqlite'

# Clean and reinstall dependencies
rm -rf .Build vendor composer.lock
php8.2 /usr/local/bin/composer install

PHPUnit Version Conflicts

# Clear Composer cache
php8.2 /usr/local/bin/composer clear-cache

# Show why PHPUnit version was selected
php8.2 /usr/local/bin/composer why phpunit/phpunit

# Force dependency resolution
rm composer.lock
php8.2 /usr/local/bin/composer update --with-all-dependencies

Summary

Aspect	Docker/runTests.sh	Native PHP
Use Case	Standard TYPO3 development	CI/CD, no Docker available
Isolation	✅ Full container isolation	⚠️ System-wide PHP
Setup	Docker required	Multiple PHP versions required
Speed	Slower (container overhead)	Faster (native execution)
Consistency	✅ Guaranteed environment	⚠️ Depends on system config
Recommendation	Recommended	Alternative when needed

Choose Docker/runTests.sh for:

• Local development and testing
• Consistent environment across team
• Official TYPO3 best practices

Choose Native PHP for:

• GitHub Actions/GitLab CI environments
• Systems without Docker
• Performance-critical testing pipelines