Initial commit

This commit is contained in:
Zhongwei Li
2025-11-29 18:35:18 +08:00
commit 72b4217943
8 changed files with 2268 additions and 0 deletions

View File

@@ -0,0 +1,533 @@
---
name: bash-defensive-patterns
description: Master defensive Bash programming techniques for production-grade scripts. Use when writing robust shell scripts, CI/CD pipelines, or system utilities requiring fault tolerance and safety.
---
# Bash Defensive Patterns
Comprehensive guidance for writing production-ready Bash scripts using defensive programming techniques, error handling, and safety best practices to prevent common pitfalls and ensure reliability.
## When to Use This Skill
- Writing production automation scripts
- Building CI/CD pipeline scripts
- Creating system administration utilities
- Developing error-resilient deployment automation
- Writing scripts that must handle edge cases safely
- Building maintainable shell script libraries
- Implementing comprehensive logging and monitoring
- Creating scripts that must work across different platforms
## Core Defensive Principles
### 1. Strict Mode
Enable bash strict mode at the start of every script to catch errors early.
```bash
#!/bin/bash
set -Eeuo pipefail # Exit on error, unset variables, pipe failures
```
**Key flags:**
- `set -E`: Inherit ERR trap in functions
- `set -e`: Exit on any error (command returns non-zero)
- `set -u`: Exit on undefined variable reference
- `set -o pipefail`: Pipe fails if any command fails (not just last)
### 2. Error Trapping and Cleanup
Implement proper cleanup on script exit or error.
```bash
#!/bin/bash
set -Eeuo pipefail
trap 'echo "Error on line $LINENO"' ERR
trap 'echo "Cleaning up..."; rm -rf "$TMPDIR"' EXIT
TMPDIR=$(mktemp -d)
# Script code here
```
### 3. Variable Safety
Always quote variables to prevent word splitting and globbing issues.
```bash
# Wrong - unsafe
cp $source $dest
# Correct - safe
cp "$source" "$dest"
# Required variables - fail with message if unset
: "${REQUIRED_VAR:?REQUIRED_VAR is not set}"
```
### 4. Array Handling
Use arrays safely for complex data handling.
```bash
# Safe array iteration
declare -a items=("item 1" "item 2" "item 3")
for item in "${items[@]}"; do
echo "Processing: $item"
done
# Reading output into array safely
mapfile -t lines < <(some_command)
readarray -t numbers < <(seq 1 10)
```
### 5. Conditional Safety
Use `[[ ]]` for Bash-specific features, `[ ]` for POSIX.
```bash
# Bash - safer
if [[ -f "$file" && -r "$file" ]]; then
content=$(<"$file")
fi
# POSIX - portable
if [ -f "$file" ] && [ -r "$file" ]; then
content=$(cat "$file")
fi
# Test for existence before operations
if [[ -z "${VAR:-}" ]]; then
echo "VAR is not set or is empty"
fi
```
## Fundamental Patterns
### Pattern 1: Safe Script Directory Detection
```bash
#!/bin/bash
set -Eeuo pipefail
# Correctly determine script directory
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
SCRIPT_NAME="$(basename -- "${BASH_SOURCE[0]}")"
echo "Script location: $SCRIPT_DIR/$SCRIPT_NAME"
```
### Pattern 2: Comprehensive Function Templat
```bash
#!/bin/bash
set -Eeuo pipefail
# Prefix for functions: handle_*, process_*, check_*, validate_*
# Include documentation and error handling
validate_file() {
local -r file="$1"
local -r message="${2:-File not found: $file}"
if [[ ! -f "$file" ]]; then
echo "ERROR: $message" >&2
return 1
fi
return 0
}
process_files() {
local -r input_dir="$1"
local -r output_dir="$2"
# Validate inputs
[[ -d "$input_dir" ]] || { echo "ERROR: input_dir not a directory" >&2; return 1; }
# Create output directory if needed
mkdir -p "$output_dir" || { echo "ERROR: Cannot create output_dir" >&2; return 1; }
# Process files safely
while IFS= read -r -d '' file; do
echo "Processing: $file"
# Do work
done < <(find "$input_dir" -maxdepth 1 -type f -print0)
return 0
}
```
### Pattern 3: Safe Temporary File Handling
```bash
#!/bin/bash
set -Eeuo pipefail
trap 'rm -rf -- "$TMPDIR"' EXIT
# Create temporary directory
TMPDIR=$(mktemp -d) || { echo "ERROR: Failed to create temp directory" >&2; exit 1; }
# Create temporary files in directory
TMPFILE1="$TMPDIR/temp1.txt"
TMPFILE2="$TMPDIR/temp2.txt"
# Use temporary files
touch "$TMPFILE1" "$TMPFILE2"
echo "Temp files created in: $TMPDIR"
```
### Pattern 4: Robust Argument Parsing
```bash
#!/bin/bash
set -Eeuo pipefail
# Default values
VERBOSE=false
DRY_RUN=false
OUTPUT_FILE=""
THREADS=4
usage() {
cat <<EOF
Usage: $0 [OPTIONS]
Options:
-v, --verbose Enable verbose output
-d, --dry-run Run without making changes
-o, --output FILE Output file path
-j, --jobs NUM Number of parallel jobs
-h, --help Show this help message
EOF
exit "${1:-0}"
}
# Parse arguments
while [[ $# -gt 0 ]]; do
case "$1" in
-v|--verbose)
VERBOSE=true
shift
;;
-d|--dry-run)
DRY_RUN=true
shift
;;
-o|--output)
OUTPUT_FILE="$2"
shift 2
;;
-j|--jobs)
THREADS="$2"
shift 2
;;
-h|--help)
usage 0
;;
--)
shift
break
;;
*)
echo "ERROR: Unknown option: $1" >&2
usage 1
;;
esac
done
# Validate required arguments
[[ -n "$OUTPUT_FILE" ]] || { echo "ERROR: -o/--output is required" >&2; usage 1; }
```
### Pattern 5: Structured Logging
```bash
#!/bin/bash
set -Eeuo pipefail
# Logging functions
log_info() {
echo "[$(date +'%Y-%m-%d %H:%M:%S')] INFO: $*" >&2
}
log_warn() {
echo "[$(date +'%Y-%m-%d %H:%M:%S')] WARN: $*" >&2
}
log_error() {
echo "[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $*" >&2
}
log_debug() {
if [[ "${DEBUG:-0}" == "1" ]]; then
echo "[$(date +'%Y-%m-%d %H:%M:%S')] DEBUG: $*" >&2
fi
}
# Usage
log_info "Starting script"
log_debug "Debug information"
log_warn "Warning message"
log_error "Error occurred"
```
### Pattern 6: Process Orchestration with Signals
```bash
#!/bin/bash
set -Eeuo pipefail
# Track background processes
PIDS=()
cleanup() {
log_info "Shutting down..."
# Terminate all background processes
for pid in "${PIDS[@]}"; do
if kill -0 "$pid" 2>/dev/null; then
kill -TERM "$pid" 2>/dev/null || true
fi
done
# Wait for graceful shutdown
for pid in "${PIDS[@]}"; do
wait "$pid" 2>/dev/null || true
done
}
trap cleanup SIGTERM SIGINT
# Start background tasks
background_task &
PIDS+=($!)
another_task &
PIDS+=($!)
# Wait for all background processes
wait
```
### Pattern 7: Safe File Operations
```bash
#!/bin/bash
set -Eeuo pipefail
# Use -i flag to move safely without overwriting
safe_move() {
local -r source="$1"
local -r dest="$2"
if [[ ! -e "$source" ]]; then
echo "ERROR: Source does not exist: $source" >&2
return 1
fi
if [[ -e "$dest" ]]; then
echo "ERROR: Destination already exists: $dest" >&2
return 1
fi
mv "$source" "$dest"
}
# Safe directory cleanup
safe_rmdir() {
local -r dir="$1"
if [[ ! -d "$dir" ]]; then
echo "ERROR: Not a directory: $dir" >&2
return 1
fi
# Use -I flag to prompt before rm (BSD/GNU compatible)
rm -rI -- "$dir"
}
# Atomic file writes
atomic_write() {
local -r target="$1"
local -r tmpfile
tmpfile=$(mktemp) || return 1
# Write to temp file first
cat > "$tmpfile"
# Atomic rename
mv "$tmpfile" "$target"
}
```
### Pattern 8: Idempotent Script Design
```bash
#!/bin/bash
set -Eeuo pipefail
# Check if resource already exists
ensure_directory() {
local -r dir="$1"
if [[ -d "$dir" ]]; then
log_info "Directory already exists: $dir"
return 0
fi
mkdir -p "$dir" || {
log_error "Failed to create directory: $dir"
return 1
}
log_info "Created directory: $dir"
}
# Ensure configuration state
ensure_config() {
local -r config_file="$1"
local -r default_value="$2"
if [[ ! -f "$config_file" ]]; then
echo "$default_value" > "$config_file"
log_info "Created config: $config_file"
fi
}
# Rerunning script multiple times should be safe
ensure_directory "/var/cache/myapp"
ensure_config "/etc/myapp/config" "DEBUG=false"
```
### Pattern 9: Safe Command Substitution
```bash
#!/bin/bash
set -Eeuo pipefail
# Use $() instead of backticks
name=$(<"$file") # Modern, safe variable assignment from file
output=$(command -v python3) # Get command location safely
# Handle command substitution with error checking
result=$(command -v node) || {
log_error "node command not found"
return 1
}
# For multiple lines
mapfile -t lines < <(grep "pattern" "$file")
# NUL-safe iteration
while IFS= read -r -d '' file; do
echo "Processing: $file"
done < <(find /path -type f -print0)
```
### Pattern 10: Dry-Run Support
```bash
#!/bin/bash
set -Eeuo pipefail
DRY_RUN="${DRY_RUN:-false}"
run_cmd() {
if [[ "$DRY_RUN" == "true" ]]; then
echo "[DRY RUN] Would execute: $*"
return 0
fi
"$@"
}
# Usage
run_cmd cp "$source" "$dest"
run_cmd rm "$file"
run_cmd chown "$owner" "$target"
```
## Advanced Defensive Techniques
### Named Parameters Pattern
```bash
#!/bin/bash
set -Eeuo pipefail
process_data() {
local input_file=""
local output_dir=""
local format="json"
# Parse named parameters
while [[ $# -gt 0 ]]; do
case "$1" in
--input=*)
input_file="${1#*=}"
;;
--output=*)
output_dir="${1#*=}"
;;
--format=*)
format="${1#*=}"
;;
*)
echo "ERROR: Unknown parameter: $1" >&2
return 1
;;
esac
shift
done
# Validate required parameters
[[ -n "$input_file" ]] || { echo "ERROR: --input is required" >&2; return 1; }
[[ -n "$output_dir" ]] || { echo "ERROR: --output is required" >&2; return 1; }
}
```
### Dependency Checking
```bash
#!/bin/bash
set -Eeuo pipefail
check_dependencies() {
local -a missing_deps=()
local -a required=("jq" "curl" "git")
for cmd in "${required[@]}"; do
if ! command -v "$cmd" &>/dev/null; then
missing_deps+=("$cmd")
fi
done
if [[ ${#missing_deps[@]} -gt 0 ]]; then
echo "ERROR: Missing required commands: ${missing_deps[*]}" >&2
return 1
fi
}
check_dependencies
```
## Best Practices Summary
1. **Always use strict mode** - `set -Eeuo pipefail`
2. **Quote all variables** - `"$variable"` prevents word splitting
3. **Use [[ ]] conditionals** - More robust than [ ]
4. **Implement error trapping** - Catch and handle errors gracefully
5. **Validate all inputs** - Check file existence, permissions, formats
6. **Use functions for reusability** - Prefix with meaningful names
7. **Implement structured logging** - Include timestamps and levels
8. **Support dry-run mode** - Allow users to preview changes
9. **Handle temporary files safely** - Use mktemp, cleanup with trap
10. **Design for idempotency** - Scripts should be safe to rerun
11. **Document requirements** - List dependencies and minimum versions
12. **Test error paths** - Ensure error handling works correctly
13. **Use `command -v`** - Safer than `which` for checking executables
14. **Prefer printf over echo** - More predictable across systems
## Resources
- **Bash Strict Mode**: http://redsymbol.net/articles/unofficial-bash-strict-mode/
- **Google Shell Style Guide**: https://google.github.io/styleguide/shellguide.html
- **Defensive BASH Programming**: https://www.lifepipe.net/

View File

@@ -0,0 +1,630 @@
---
name: bats-testing-patterns
description: Master Bash Automated Testing System (Bats) for comprehensive shell script testing. Use when writing tests for shell scripts, CI/CD pipelines, or requiring test-driven development of shell utilities.
---
# Bats Testing Patterns
Comprehensive guidance for writing comprehensive unit tests for shell scripts using Bats (Bash Automated Testing System), including test patterns, fixtures, and best practices for production-grade shell testing.
## When to Use This Skill
- Writing unit tests for shell scripts
- Implementing test-driven development (TDD) for scripts
- Setting up automated testing in CI/CD pipelines
- Testing edge cases and error conditions
- Validating behavior across different shell environments
- Building maintainable test suites for scripts
- Creating fixtures for complex test scenarios
- Testing multiple shell dialects (bash, sh, dash)
## Bats Fundamentals
### What is Bats?
Bats (Bash Automated Testing System) is a TAP (Test Anything Protocol) compliant testing framework for shell scripts that provides:
- Simple, natural test syntax
- TAP output format compatible with CI systems
- Fixtures and setup/teardown support
- Assertion helpers
- Parallel test execution
### Installation
```bash
# macOS with Homebrew
brew install bats-core
# Ubuntu/Debian
git clone https://github.com/bats-core/bats-core.git
cd bats-core
./install.sh /usr/local
# From npm (Node.js)
npm install --global bats
# Verify installation
bats --version
```
### File Structure
```
project/
├── bin/
│ ├── script.sh
│ └── helper.sh
├── tests/
│ ├── test_script.bats
│ ├── test_helper.sh
│ ├── fixtures/
│ │ ├── input.txt
│ │ └── expected_output.txt
│ └── helpers/
│ └── mocks.bash
└── README.md
```
## Basic Test Structure
### Simple Test File
```bash
#!/usr/bin/env bats
# Load test helper if present
load test_helper
# Setup runs before each test
setup() {
export TMPDIR=$(mktemp -d)
}
# Teardown runs after each test
teardown() {
rm -rf "$TMPDIR"
}
# Test: simple assertion
@test "Function returns 0 on success" {
run my_function "input"
[ "$status" -eq 0 ]
}
# Test: output verification
@test "Function outputs correct result" {
run my_function "test"
[ "$output" = "expected output" ]
}
# Test: error handling
@test "Function returns 1 on missing argument" {
run my_function
[ "$status" -eq 1 ]
}
```
## Assertion Patterns
### Exit Code Assertions
```bash
#!/usr/bin/env bats
@test "Command succeeds" {
run true
[ "$status" -eq 0 ]
}
@test "Command fails as expected" {
run false
[ "$status" -ne 0 ]
}
@test "Command returns specific exit code" {
run my_function --invalid
[ "$status" -eq 127 ]
}
@test "Can capture command result" {
run echo "hello"
[ $status -eq 0 ]
[ "$output" = "hello" ]
}
```
### Output Assertions
```bash
#!/usr/bin/env bats
@test "Output matches string" {
result=$(echo "hello world")
[ "$result" = "hello world" ]
}
@test "Output contains substring" {
result=$(echo "hello world")
[[ "$result" == *"world"* ]]
}
@test "Output matches pattern" {
result=$(date +%Y)
[[ "$result" =~ ^[0-9]{4}$ ]]
}
@test "Multi-line output" {
run printf "line1\nline2\nline3"
[ "$output" = "line1
line2
line3" ]
}
@test "Lines variable contains output" {
run printf "line1\nline2\nline3"
[ "${lines[0]}" = "line1" ]
[ "${lines[1]}" = "line2" ]
[ "${lines[2]}" = "line3" ]
}
```
### File Assertions
```bash
#!/usr/bin/env bats
@test "File is created" {
[ ! -f "$TMPDIR/output.txt" ]
my_function > "$TMPDIR/output.txt"
[ -f "$TMPDIR/output.txt" ]
}
@test "File contents match expected" {
my_function > "$TMPDIR/output.txt"
[ "$(cat "$TMPDIR/output.txt")" = "expected content" ]
}
@test "File is readable" {
touch "$TMPDIR/test.txt"
[ -r "$TMPDIR/test.txt" ]
}
@test "File has correct permissions" {
touch "$TMPDIR/test.txt"
chmod 644 "$TMPDIR/test.txt"
[ "$(stat -f %OLp "$TMPDIR/test.txt")" = "644" ]
}
@test "File size is correct" {
echo -n "12345" > "$TMPDIR/test.txt"
[ "$(wc -c < "$TMPDIR/test.txt")" -eq 5 ]
}
```
## Setup and Teardown Patterns
### Basic Setup and Teardown
```bash
#!/usr/bin/env bats
setup() {
# Create test directory
TEST_DIR=$(mktemp -d)
export TEST_DIR
# Source script under test
source "${BATS_TEST_DIRNAME}/../bin/script.sh"
}
teardown() {
# Clean up temporary directory
rm -rf "$TEST_DIR"
}
@test "Test using TEST_DIR" {
touch "$TEST_DIR/file.txt"
[ -f "$TEST_DIR/file.txt" ]
}
```
### Setup with Resources
```bash
#!/usr/bin/env bats
setup() {
# Create directory structure
mkdir -p "$TMPDIR/data/input"
mkdir -p "$TMPDIR/data/output"
# Create test fixtures
echo "line1" > "$TMPDIR/data/input/file1.txt"
echo "line2" > "$TMPDIR/data/input/file2.txt"
# Initialize environment
export DATA_DIR="$TMPDIR/data"
export INPUT_DIR="$DATA_DIR/input"
export OUTPUT_DIR="$DATA_DIR/output"
}
teardown() {
rm -rf "$TMPDIR/data"
}
@test "Processes input files" {
run my_process_script "$INPUT_DIR" "$OUTPUT_DIR"
[ "$status" -eq 0 ]
[ -f "$OUTPUT_DIR/file1.txt" ]
}
```
### Global Setup/Teardown
```bash
#!/usr/bin/env bats
# Load shared setup from test_helper.sh
load test_helper
# setup_file runs once before all tests
setup_file() {
export SHARED_RESOURCE=$(mktemp -d)
echo "Expensive setup" > "$SHARED_RESOURCE/data.txt"
}
# teardown_file runs once after all tests
teardown_file() {
rm -rf "$SHARED_RESOURCE"
}
@test "First test uses shared resource" {
[ -f "$SHARED_RESOURCE/data.txt" ]
}
@test "Second test uses shared resource" {
[ -d "$SHARED_RESOURCE" ]
}
```
## Mocking and Stubbing Patterns
### Function Mocking
```bash
#!/usr/bin/env bats
# Mock external command
my_external_tool() {
echo "mocked output"
return 0
}
@test "Function uses mocked tool" {
export -f my_external_tool
run my_function
[[ "$output" == *"mocked output"* ]]
}
```
### Command Stubbing
```bash
#!/usr/bin/env bats
setup() {
# Create stub directory
STUBS_DIR="$TMPDIR/stubs"
mkdir -p "$STUBS_DIR"
# Add to PATH
export PATH="$STUBS_DIR:$PATH"
}
create_stub() {
local cmd="$1"
local output="$2"
local code="${3:-0}"
cat > "$STUBS_DIR/$cmd" <<EOF
#!/bin/bash
echo "$output"
exit $code
EOF
chmod +x "$STUBS_DIR/$cmd"
}
@test "Function works with stubbed curl" {
create_stub curl "{ \"status\": \"ok\" }" 0
run my_api_function
[ "$status" -eq 0 ]
}
```
### Variable Stubbing
```bash
#!/usr/bin/env bats
@test "Function handles environment override" {
export MY_SETTING="override_value"
run my_function
[ "$status" -eq 0 ]
[[ "$output" == *"override_value"* ]]
}
@test "Function uses default when var unset" {
unset MY_SETTING
run my_function
[ "$status" -eq 0 ]
[[ "$output" == *"default"* ]]
}
```
## Fixture Management
### Using Fixture Files
```bash
#!/usr/bin/env bats
# Fixture directory: tests/fixtures/
setup() {
FIXTURES_DIR="${BATS_TEST_DIRNAME}/fixtures"
WORK_DIR=$(mktemp -d)
export WORK_DIR
}
teardown() {
rm -rf "$WORK_DIR"
}
@test "Process fixture file" {
# Copy fixture to work directory
cp "$FIXTURES_DIR/input.txt" "$WORK_DIR/input.txt"
# Run function
run my_process_function "$WORK_DIR/input.txt"
# Compare output
diff "$WORK_DIR/output.txt" "$FIXTURES_DIR/expected_output.txt"
}
```
### Dynamic Fixture Generation
```bash
#!/usr/bin/env bats
generate_fixture() {
local lines="$1"
local file="$2"
for i in $(seq 1 "$lines"); do
echo "Line $i content" >> "$file"
done
}
@test "Handle large input file" {
generate_fixture 1000 "$TMPDIR/large.txt"
run my_function "$TMPDIR/large.txt"
[ "$status" -eq 0 ]
[ "$(wc -l < "$TMPDIR/large.txt")" -eq 1000 ]
}
```
## Advanced Patterns
### Testing Error Conditions
```bash
#!/usr/bin/env bats
@test "Function fails with missing file" {
run my_function "/nonexistent/file.txt"
[ "$status" -ne 0 ]
[[ "$output" == *"not found"* ]]
}
@test "Function fails with invalid input" {
run my_function ""
[ "$status" -ne 0 ]
}
@test "Function fails with permission denied" {
touch "$TMPDIR/readonly.txt"
chmod 000 "$TMPDIR/readonly.txt"
run my_function "$TMPDIR/readonly.txt"
[ "$status" -ne 0 ]
chmod 644 "$TMPDIR/readonly.txt" # Cleanup
}
@test "Function provides helpful error message" {
run my_function --invalid-option
[ "$status" -ne 0 ]
[[ "$output" == *"Usage:"* ]]
}
```
### Testing with Dependencies
```bash
#!/usr/bin/env bats
setup() {
# Check for required tools
if ! command -v jq &>/dev/null; then
skip "jq is not installed"
fi
export SCRIPT="${BATS_TEST_DIRNAME}/../bin/script.sh"
}
@test "JSON parsing works" {
skip_if ! command -v jq &>/dev/null
run my_json_parser '{"key": "value"}'
[ "$status" -eq 0 ]
}
```
### Testing Shell Compatibility
```bash
#!/usr/bin/env bats
@test "Script works in bash" {
bash "${BATS_TEST_DIRNAME}/../bin/script.sh" arg1
}
@test "Script works in sh (POSIX)" {
sh "${BATS_TEST_DIRNAME}/../bin/script.sh" arg1
}
@test "Script works in dash" {
if command -v dash &>/dev/null; then
dash "${BATS_TEST_DIRNAME}/../bin/script.sh" arg1
else
skip "dash not installed"
fi
}
```
### Parallel Execution
```bash
#!/usr/bin/env bats
@test "Multiple independent operations" {
run bash -c 'for i in {1..10}; do
my_operation "$i" &
done
wait'
[ "$status" -eq 0 ]
}
@test "Concurrent file operations" {
for i in {1..5}; do
my_function "$TMPDIR/file$i" &
done
wait
[ -f "$TMPDIR/file1" ]
[ -f "$TMPDIR/file5" ]
}
```
## Test Helper Pattern
### test_helper.sh
```bash
#!/usr/bin/env bash
# Source script under test
export SCRIPT_DIR="${BATS_TEST_DIRNAME%/*}/bin"
# Common test utilities
assert_file_exists() {
if [ ! -f "$1" ]; then
echo "Expected file to exist: $1"
return 1
fi
}
assert_file_equals() {
local file="$1"
local expected="$2"
if [ ! -f "$file" ]; then
echo "File does not exist: $file"
return 1
fi
local actual=$(cat "$file")
if [ "$actual" != "$expected" ]; then
echo "File contents do not match"
echo "Expected: $expected"
echo "Actual: $actual"
return 1
fi
}
# Create temporary test directory
setup_test_dir() {
export TEST_DIR=$(mktemp -d)
}
cleanup_test_dir() {
rm -rf "$TEST_DIR"
}
```
## Integration with CI/CD
### GitHub Actions Workflow
```yaml
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Bats
run: |
npm install --global bats
- name: Run Tests
run: |
bats tests/*.bats
- name: Run Tests with Tap Reporter
run: |
bats tests/*.bats --tap | tee test_output.tap
```
### Makefile Integration
```makefile
.PHONY: test test-verbose test-tap
test:
bats tests/*.bats
test-verbose:
bats tests/*.bats --verbose
test-tap:
bats tests/*.bats --tap
test-parallel:
bats tests/*.bats --parallel 4
coverage: test
# Optional: Generate coverage reports
```
## Best Practices
1. **Test one thing per test** - Single responsibility principle
2. **Use descriptive test names** - Clearly states what is being tested
3. **Clean up after tests** - Always remove temporary files in teardown
4. **Test both success and failure paths** - Don't just test happy path
5. **Mock external dependencies** - Isolate unit under test
6. **Use fixtures for complex data** - Makes tests more readable
7. **Run tests in CI/CD** - Catch regressions early
8. **Test across shell dialects** - Ensure portability
9. **Keep tests fast** - Run in parallel when possible
10. **Document complex test setup** - Explain unusual patterns
## Resources
- **Bats GitHub**: https://github.com/bats-core/bats-core
- **Bats Documentation**: https://bats-core.readthedocs.io/
- **TAP Protocol**: https://testanything.org/
- **Test-Driven Development**: https://en.wikipedia.org/wiki/Test-driven_development

View File

@@ -0,0 +1,454 @@
---
name: shellcheck-configuration
description: Master ShellCheck static analysis configuration and usage for shell script quality. Use when setting up linting infrastructure, fixing code issues, or ensuring script portability.
---
# ShellCheck Configuration and Static Analysis
Comprehensive guidance for configuring and using ShellCheck to improve shell script quality, catch common pitfalls, and enforce best practices through static code analysis.
## When to Use This Skill
- Setting up linting for shell scripts in CI/CD pipelines
- Analyzing existing shell scripts for issues
- Understanding ShellCheck error codes and warnings
- Configuring ShellCheck for specific project requirements
- Integrating ShellCheck into development workflows
- Suppressing false positives and configuring rule sets
- Enforcing consistent code quality standards
- Migrating scripts to meet quality gates
## ShellCheck Fundamentals
### What is ShellCheck?
ShellCheck is a static analysis tool that analyzes shell scripts and detects problematic patterns. It supports:
- Bash, sh, dash, ksh, and other POSIX shells
- Over 100 different warnings and errors
- Configuration for target shell and flags
- Integration with editors and CI/CD systems
### Installation
```bash
# macOS with Homebrew
brew install shellcheck
# Ubuntu/Debian
apt-get install shellcheck
# From source
git clone https://github.com/koalaman/shellcheck.git
cd shellcheck
make build
make install
# Verify installation
shellcheck --version
```
## Configuration Files
### .shellcheckrc (Project Level)
Create `.shellcheckrc` in your project root:
```
# Specify target shell
shell=bash
# Enable optional checks
enable=avoid-nullary-conditions
enable=require-variable-braces
# Disable specific warnings
disable=SC1091
disable=SC2086
```
### Environment Variables
```bash
# Set default shell target
export SHELLCHECK_SHELL=bash
# Enable strict mode
export SHELLCHECK_STRICT=true
# Specify configuration file location
export SHELLCHECK_CONFIG=~/.shellcheckrc
```
## Common ShellCheck Error Codes
### SC1000-1099: Parser Errors
```bash
# SC1004: Backslash continuation not followed by newline
echo hello\
world # Error - needs line continuation
# SC1008: Invalid data for operator `=='
if [[ $var = "value" ]]; then # Space before ==
true
fi
```
### SC2000-2099: Shell Issues
```bash
# SC2009: Consider using pgrep or pidof instead of grep|grep
ps aux | grep -v grep | grep myprocess # Use pgrep instead
# SC2012: Use `ls` only for viewing. Use `find` for reliable output
for file in $(ls -la) # Better: use find or globbing
# SC2015: Avoid using && and || instead of if-then-else
[[ -f "$file" ]] && echo "found" || echo "not found" # Less clear
# SC2016: Expressions don't expand in single quotes
echo '$VAR' # Literal $VAR, not variable expansion
# SC2026: This word is non-standard. Set POSIXLY_CORRECT
# when using with scripts for other shells
```
### SC2100-2199: Quoting Issues
```bash
# SC2086: Double quote to prevent globbing and word splitting
for i in $list; do # Should be: for i in $list or for i in "$list"
echo "$i"
done
# SC2115: Literal tilde in path not expanded. Use $HOME instead
~/.bashrc # In strings, use "$HOME/.bashrc"
# SC2181: Check exit code directly with `if`, not indirectly in a list
some_command
if [ $? -eq 0 ]; then # Better: if some_command; then
# SC2206: Quote to prevent word splitting or set IFS
array=( $items ) # Should use: array=( $items )
```
### SC3000-3999: POSIX Compliance Issues
```bash
# SC3010: In POSIX sh, use 'case' instead of 'cond && foo'
[[ $var == "value" ]] && do_something # Not POSIX
# SC3043: In POSIX sh, use 'local' is undefined
function my_func() {
local var=value # Not POSIX in some shells
}
```
## Practical Configuration Examples
### Minimal Configuration (Strict POSIX)
```bash
#!/bin/bash
# Configure for maximum portability
shellcheck \
--shell=sh \
--external-sources \
--check-sourced \
script.sh
```
### Development Configuration (Bash with Relaxed Rules)
```bash
#!/bin/bash
# Configure for Bash development
shellcheck \
--shell=bash \
--exclude=SC1091,SC2119 \
--enable=all \
script.sh
```
### CI/CD Integration Configuration
```bash
#!/bin/bash
set -Eeuo pipefail
# Analyze all shell scripts and fail on issues
find . -type f -name "*.sh" | while read -r script; do
echo "Checking: $script"
shellcheck \
--shell=bash \
--format=gcc \
--exclude=SC1091 \
"$script" || exit 1
done
```
### .shellcheckrc for Project
```
# Shell dialect to analyze against
shell=bash
# Enable optional checks
enable=avoid-nullary-conditions,require-variable-braces,check-unassigned-uppercase
# Disable specific warnings
# SC1091: Not following sourced files (many false positives)
disable=SC1091
# SC2119: Use function_name instead of function_name -- (arguments)
disable=SC2119
# External files to source for context
external-sources=true
```
## Integration Patterns
### Pre-commit Hook Configuration
```bash
#!/bin/bash
# .git/hooks/pre-commit
#!/bin/bash
set -e
# Find all shell scripts changed in this commit
git diff --cached --name-only | grep '\.sh$' | while read -r script; do
echo "Linting: $script"
if ! shellcheck "$script"; then
echo "ShellCheck failed on $script"
exit 1
fi
done
```
### GitHub Actions Workflow
```yaml
name: ShellCheck
on: [push, pull_request]
jobs:
shellcheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Run ShellCheck
run: |
sudo apt-get install shellcheck
find . -type f -name "*.sh" -exec shellcheck {} \;
```
### GitLab CI Pipeline
```yaml
shellcheck:
stage: lint
image: koalaman/shellcheck-alpine
script:
- find . -type f -name "*.sh" -exec shellcheck {} \;
allow_failure: false
```
## Handling ShellCheck Violations
### Suppressing Specific Warnings
```bash
#!/bin/bash
# Disable warning for entire line
# shellcheck disable=SC2086
for file in $(ls -la); do
echo "$file"
done
# Disable for entire script
# shellcheck disable=SC1091,SC2119
# Disable multiple warnings (format varies)
command_that_fails() {
# shellcheck disable=SC2015
[ -f "$1" ] && echo "found" || echo "not found"
}
# Disable specific check for source directive
# shellcheck source=./helper.sh
source helper.sh
```
### Common Violations and Fixes
#### SC2086: Double quote to prevent word splitting
```bash
# Problem
for i in $list; do done
# Solution
for i in $list; do done # If $list is already quoted, or
for i in "${list[@]}"; do done # If list is an array
```
#### SC2181: Check exit code directly
```bash
# Problem
some_command
if [ $? -eq 0 ]; then
echo "success"
fi
# Solution
if some_command; then
echo "success"
fi
```
#### SC2015: Use if-then instead of && ||
```bash
# Problem
[ -f "$file" ] && echo "exists" || echo "not found"
# Solution - clearer intent
if [ -f "$file" ]; then
echo "exists"
else
echo "not found"
fi
```
#### SC2016: Expressions don't expand in single quotes
```bash
# Problem
echo 'Variable value: $VAR'
# Solution
echo "Variable value: $VAR"
```
#### SC2009: Use pgrep instead of grep
```bash
# Problem
ps aux | grep -v grep | grep myprocess
# Solution
pgrep -f myprocess
```
## Performance Optimization
### Checking Multiple Files
```bash
#!/bin/bash
# Sequential checking
for script in *.sh; do
shellcheck "$script"
done
# Parallel checking (faster)
find . -name "*.sh" -print0 | \
xargs -0 -P 4 -n 1 shellcheck
```
### Caching Results
```bash
#!/bin/bash
CACHE_DIR=".shellcheck_cache"
mkdir -p "$CACHE_DIR"
check_script() {
local script="$1"
local hash
local cache_file
hash=$(sha256sum "$script" | cut -d' ' -f1)
cache_file="$CACHE_DIR/$hash"
if [[ ! -f "$cache_file" ]]; then
if shellcheck "$script" > "$cache_file" 2>&1; then
touch "$cache_file.ok"
else
return 1
fi
fi
[[ -f "$cache_file.ok" ]]
}
find . -name "*.sh" | while read -r script; do
check_script "$script" || exit 1
done
```
## Output Formats
### Default Format
```bash
shellcheck script.sh
# Output:
# script.sh:1:3: warning: foo is referenced but not assigned. [SC2154]
```
### GCC Format (for CI/CD)
```bash
shellcheck --format=gcc script.sh
# Output:
# script.sh:1:3: warning: foo is referenced but not assigned.
```
### JSON Format (for parsing)
```bash
shellcheck --format=json script.sh
# Output:
# [{"file": "script.sh", "line": 1, "column": 3, "level": "warning", "code": 2154, "message": "..."}]
```
### Quiet Format
```bash
shellcheck --format=quiet script.sh
# Returns non-zero if issues found, no output otherwise
```
## Best Practices
1. **Run ShellCheck in CI/CD** - Catch issues before merging
2. **Configure for your target shell** - Don't analyze bash as sh
3. **Document exclusions** - Explain why violations are suppressed
4. **Address violations** - Don't just disable warnings
5. **Enable strict mode** - Use `--enable=all` with careful exclusions
6. **Update regularly** - Keep ShellCheck current for new checks
7. **Use pre-commit hooks** - Catch issues locally before pushing
8. **Integrate with editors** - Get real-time feedback during development
## Resources
- **ShellCheck GitHub**: https://github.com/koalaman/shellcheck
- **ShellCheck Wiki**: https://www.shellcheck.net/wiki/
- **Error Code Reference**: https://www.shellcheck.net/