Initial commit

This commit is contained in:
Zhongwei Li
2025-11-29 18:00:18 +08:00
commit 765529cd13
69 changed files with 18291 additions and 0 deletions

View File

@@ -0,0 +1,545 @@
---
name: python-uv-scripts
description: >
Python single-file script development using uv and PEP 723 inline metadata.
Prevents invalid patterns like [tool.uv.metadata].
Use when creating standalone Python utilities, converting scripts to uv format,
managing script dependencies, implementing script testing,
or establishing team standards for script development.
---
# Python Single-File Scripts with uv
Expert guidance for creating production-ready, self-contained Python scripts using uv's inline dependency management
(PEP 723).
## Quick Start
### Create Your First uv Script
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# "rich>=13.0.0",
# ]
# ///
import httpx
from rich import print
response = httpx.get("https://api.github.com")
print(f"[green]Status: {response.status_code}[/green]")
```
Make it executable:
```bash
chmod +x script.py
./script.py # uv automatically installs dependencies
```
### Convert Existing Script
```bash
# Add inline metadata to existing script
./tools/convert_to_uv.py existing_script.py
# Validate PEP 723 metadata
./tools/validate_script.py script.py
```
## Core Concepts
### What is PEP 723?
**PEP 723** defines inline script metadata for Python files:
```python
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "package>=1.0.0",
# ]
# ///
```
**Benefits:**
- ✅ Dependencies live with the code
- ✅ No separate `requirements.txt`
- ✅ Reproducible execution
- ✅ Version constraints included
- ✅ Self-documenting
### uv Script Execution Modes
**Mode 1: Inline Dependencies** (Recommended for utilities)
```python
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = ["httpx"]
# ///
```
**Mode 2: Project Mode** (For larger scripts)
```bash
uv run script.py # Uses pyproject.toml
```
### Mode 3: No Dependencies
```python
#!/usr/bin/env -S uv run
# Standard library only
```
## Critical Anti-Patterns: What NOT to Do
### ❌ NEVER Use [tool.uv.metadata]
**WRONG** - This will cause errors:
```python
# /// script
# requires-python = ">=3.11"
# [tool.uv.metadata] # ❌ THIS DOES NOT WORK
# purpose = "testing"
# ///
```
**Error**:
```text
error: TOML parse error at line 3, column 7
unknown field `metadata`
```
**Why**: `[tool.uv.metadata]` is not part of PEP 723 and is not supported by uv.
**CORRECT** - Use Python docstrings for metadata:
```python
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""
Purpose: Testing automation
Team: DevOps
Author: team@example.com
"""
```
**Valid `tool.uv` fields** (if needed):
```python
# /// script
# requires-python = ">=3.11"
# dependencies = []
# [tool.uv]
# exclude-newer = "2025-01-01T00:00:00Z" # For reproducibility
# ///
```
## Real-World Examples from This Repository
### Example 1: Cluster Health Checker
See [examples/03-production-ready/check_cluster_health_enhanced.py](examples/03-production-ready/check_cluster_health_enhanced.py)
**Current version** (basic):
```python
#!/usr/bin/env python3
import subprocess
# Manual dependency installation required
```
**Enhanced with uv** (production-ready):
```python
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "rich>=13.0.0",
# "typer>=0.9.0",
# ]
# ///
"""
Purpose: Cluster health monitoring
Team: Infrastructure
"""
import typer
from rich.console import Console
from rich.table import Table
```
### Example 2: CEPH Health Monitor
See [examples/03-production-ready/ceph_health.py](examples/03-production-ready/ceph_health.py)
Pattern: JSON API interaction with structured output
## Best Practices from This Repository
### 1. Security Patterns
See [reference/security-patterns.md](reference/security-patterns.md) for complete security guide including:
- Secrets management (environment variables, keyring, Infisical)
- Input validation
- Dependency security
- File operations security
- Command execution security
### 2. Version Pinning Strategy
Following this repository's approach (from `pyproject.toml`):
```python
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0", # Minimum version for compatibility
# "rich>=13.0.0", # Known good version
# "ansible>=11.1.0", # Match project requirements
# ]
# ///
```
**Pinning levels:**
- `>=X.Y.Z` - Minimum version (most flexible)
- `~=X.Y.Z` - Compatible release (patch updates only)
- `==X.Y.Z` - Exact version (most strict)
See [reference/dependency-management.md](reference/dependency-management.md).
### 3. Team Standards
**File naming:**
```bash
check_cluster_health.py # ✅ Descriptive, snake_case
validate_template.py # ✅ Action-oriented
cluster.py # ❌ Too generic
```
**Shebang pattern:**
```python
#!/usr/bin/env -S uv run --script --quiet
# --quiet suppresses uv's own output
```
**Documentation template:**
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""
Check Proxmox cluster health
Purpose: cluster-monitoring
Team: infrastructure
Author: devops@spaceships.work
Usage:
python check_cluster_health.py [--node NODE] [--json]
Examples:
python check_cluster_health.py --node foxtrot
python check_cluster_health.py --json
"""
```
### 4. Error Handling Patterns
Following Ansible best practices from this repository:
```python
import sys
import subprocess
def run_command(cmd: str) -> str:
"""Execute command with proper error handling"""
try:
result = subprocess.run(
cmd.split(),
capture_output=True,
text=True,
check=True
)
return result.stdout
except subprocess.CalledProcessError as e:
print(f"Error: Command failed: {cmd}", file=sys.stderr)
print(f" {e.stderr}", file=sys.stderr)
sys.exit(1)
except FileNotFoundError:
print(f"Error: Command not found: {cmd.split()[0]}", file=sys.stderr)
sys.exit(1)
```
See [patterns/error-handling.md](patterns/error-handling.md).
### 5. Testing Patterns
**Inline testing** (for simple scripts):
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
def validate_ip(ip: str) -> bool:
"""Validate IP address format"""
import re
pattern = r'^(\d{1,3}\.){3}\d{1,3}$'
return bool(re.match(pattern, ip))
# Inline tests
if __name__ == "__main__":
import sys
# Run tests if --test flag provided
if len(sys.argv) > 1 and sys.argv[1] == "--test":
assert validate_ip("192.168.1.1") == True
assert validate_ip("256.1.1.1") == False
print("All tests passed!")
sys.exit(0)
# Normal execution
print(validate_ip("192.168.3.5"))
```
See [workflows/testing-strategies.md](workflows/testing-strategies.md).
## When NOT to Use Single-File Scripts
See [anti-patterns/when-not-to-use.md](anti-patterns/when-not-to-use.md) for details.
**Use a proper project instead when:**
- ❌ Script exceeds 500 lines
- ❌ Multiple modules/files needed
- ❌ Complex configuration management
- ❌ Requires packaging/distribution
- ❌ Shared library code across multiple scripts
- ❌ Web applications or long-running services
**Example - Too Complex for Single File:**
```python
# This should be a uv project, not a script:
# - 15+ dependencies
# - Database models
# - API routes
# - Background workers
# - Configuration management
# - Multiple environments
```
## Common Patterns
See pattern guides for complete examples:
- [CLI Applications](patterns/cli-applications.md) - Typer, Click, argparse patterns
- [API Clients](patterns/api-clients.md) - httpx, requests, authentication
- [Data Processing](patterns/data-processing.md) - Polars, pandas, analysis
- [System Automation](patterns/system-automation.md) - psutil, subprocess, system admin
## CI/CD Integration
### GitHub Actions
```yaml
name: Run Health Checks
on:
schedule:
- cron: '0 */6 * * *' # Every 6 hours
jobs:
health-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v3
- name: Check cluster health
run: |
uv run --script tools/check_cluster_health.py --json
env:
PROXMOX_TOKEN: ${{ secrets.PROXMOX_TOKEN }}
```
### GitLab CI
```yaml
cluster-health:
image: ghcr.io/astral-sh/uv:python3.11-bookworm-slim
script:
- uv run --script tools/check_cluster_health.py
only:
- schedules
```
See [workflows/ci-cd-integration.md](workflows/ci-cd-integration.md).
## Tools Available
### Script Validation
```bash
# Validate PEP 723 metadata
./tools/validate_script.py script.py
# Output:
# ✓ Valid PEP 723 metadata
# ✓ Python version specified
# ✓ Dependencies properly formatted
```
### Script Conversion
```bash
# Convert requirements.txt-based script to uv
./tools/convert_to_uv.py old_script.py
# Creates:
# - old_script_uv.py with inline dependencies
# - Preserves original script
```
## Progressive Disclosure
For deeper knowledge:
### Reference Documentation
- [PEP 723 Specification](reference/pep-723-spec.md) - Complete inline metadata spec
- [Dependency Management](reference/dependency-management.md) - Version pinning strategies
- [Security Patterns](reference/security-patterns.md) - Secrets, validation, input sanitization
### Pattern Guides
- [CLI Applications](patterns/cli-applications.md) - Typer, Click, argparse patterns
- [API Clients](patterns/api-clients.md) - httpx, requests, authentication
- [Data Processing](patterns/data-processing.md) - Polars, pandas, analysis
- [System Automation](patterns/system-automation.md) - psutil, subprocess, system admin
- [Error Handling](patterns/error-handling.md) - Exception handling, logging
> **Note:** See [Common Patterns](#common-patterns) section above for quick access to these guides.
### Working Examples
- [NetBox API Client](examples/04-api-clients/netbox_client.py) - Production-ready API client with Infisical, validation, error handling, and Rich output
- [Cluster Health Checker](examples/03-production-ready/check_cluster_health_enhanced.py) - Production-ready monitoring script with Typer, Rich, and JSON output
### Anti-Patterns
- [When NOT to Use](anti-patterns/when-not-to-use.md) - Signs you need a proper project
- [Common Mistakes](anti-patterns/common-mistakes.md) - Pitfalls and how to avoid them
### Workflows
- [Team Adoption](workflows/team-adoption.md) - Rolling out uv scripts across teams
- [CI/CD Integration](workflows/ci-cd-integration.md) - GitHub Actions, GitLab CI
- [Testing Strategies](workflows/testing-strategies.md) - Inline tests, pytest integration
## Related Skills
- **Ansible Best Practices** - Many Ansible modules could be standalone uv scripts
- **Proxmox Infrastructure** - Validation tools use this pattern
- **NetBox + PowerDNS Integration** - API interaction scripts
## Quick Reference
### Shebang Options
```python
# Standard script execution
#!/usr/bin/env -S uv run --script
# Quiet mode (suppress uv output)
#!/usr/bin/env -S uv run --script --quiet
# With Python version
#!/usr/bin/env -S uv run --script --python 3.11
```
### Common Dependencies
```python
# CLI applications
"typer>=0.9.0" # Modern CLI framework
"click>=8.0.0" # Alternative CLI framework
"rich>=13.0.0" # Rich text and formatting
# API clients
"httpx>=0.27.0" # Modern async HTTP client
"requests>=2.31.0" # Traditional HTTP client
# Data processing
"polars>=0.20.0" # Fast dataframe library
"pandas>=2.0.0" # Traditional dataframe library
# Infrastructure
"ansible>=11.1.0" # Automation (from this repo)
"infisical-python>=2.3.3" # Secrets (from this repo)
# System automation
"psutil>=5.9.0" # System monitoring
```
### Metadata Template
```python
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = [
# # Add dependencies here
# ]
# ///
"""
One-line description
Purpose: describe-purpose
Team: team-name
Author: email@example.com
Usage:
python script.py [OPTIONS]
Examples:
python script.py --help
"""
```
## Best Practices Summary
1. **Always specify Python version** - `requires-python = ">=3.11"`
2. **Pin dependencies appropriately** - Use `>=X.Y.Z` for utilities
3. **Add metadata in docstrings** - Put team info, purpose, and author in module docstring
4. **Include comprehensive docstrings** - Document purpose, usage, and examples
5. **Handle errors gracefully** - Use try/except with clear messages
6. **Validate inputs** - Check arguments before processing
7. **Use quiet mode** - `--quiet` flag for production scripts
8. **Keep it focused** - Single file, single purpose
9. **Test inline** - Add `--test` flag for simple validation
10. **Secure secrets** - Never hardcode, use env vars or keyring

View File

@@ -0,0 +1,373 @@
# Anti-Patterns: Common Mistakes with uv Scripts
This document lists common mistakes when creating uv scripts and how to avoid them.
## Critical Mistakes
### ❌ Using [tool.uv.metadata]
**WRONG**:
```python
# /// script
# requires-python = ">=3.11"
# dependencies = []
# [tool.uv.metadata] # ❌ THIS DOES NOT WORK
# purpose = "testing"
# team = "devops"
# ///
```
**Error**:
```text
error: TOML parse error at line 4, column 7
|
4 | [tool.uv.metadata]
| ^^
unknown field `metadata`
```
**Why**: `[tool.uv.metadata]` is not part of PEP 723 and is not supported by uv.
**CORRECT**: Use docstrings for metadata:
```python
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""
Purpose: Testing automation
Team: DevOps
Author: team@example.com
"""
```
**NOTE**: `[tool.uv]` (without `.metadata`) IS valid for fields like `exclude-newer`. See reference/pep-723-spec.md.
### ❌ Adding Custom TOML Fields
**WRONG**:
```python
# /// script
# requires-python = ">=3.11"
# author = "me" # ❌ INVALID
# version = "1.0.0" # ❌ INVALID
# description = "test" # ❌ INVALID
# dependencies = []
# ///
```
**CORRECT**: Only use `requires-python` and `dependencies` at the top level:
```python
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""
Author: me
Version: 1.0.0
Description: test
"""
```
### ❌ Hardcoded Secrets
**WRONG**:
```python
API_KEY = "sk-1234567890abcdef" # ❌ NEVER DO THIS
PASSWORD = "super_secret_password" # ❌ NEVER DO THIS
```
**CORRECT**: Use environment variables:
```python
import os
import sys
API_KEY = os.getenv("API_KEY")
if not API_KEY:
print("Error: API_KEY environment variable not set", file=sys.stderr)
sys.exit(1)
```
**BETTER**: Use Infisical (following repo pattern):
```python
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "infisical-python>=2.3.3",
# ]
# ///
from infisical import InfisicalClient
client = InfisicalClient()
api_key = client.get_secret("API_KEY", path="/production")
```
### ❌ Missing Error Handling
**WRONG**:
```python
import httpx
response = httpx.get("https://api.example.com") # ❌ No error handling
data = response.json()
```
**CORRECT**:
```python
import httpx
import sys
try:
response = httpx.get("https://api.example.com", timeout=10.0)
response.raise_for_status()
data = response.json()
except httpx.HTTPStatusError as e:
print(f"HTTP error: {e.response.status_code}", file=sys.stderr)
sys.exit(1)
except httpx.RequestError as e:
print(f"Request failed: {e}", file=sys.stderr)
sys.exit(1)
```
## When NOT to Use Single-File Scripts
### ❌ Complex Applications
Don't use single-file scripts when:
- **Script exceeds 500 lines** → Use a proper uv project
- **Multiple modules needed** → Use a proper uv project
- **Shared code across scripts** → Use a proper uv project with shared library
- **Web applications** → Use a proper uv project (Flask/FastAPI/Django)
- **Long-running services** → Use a proper uv project
- **Complex configuration** → Use a proper uv project with config files
**Example - Too Complex**:
```python
# This should be a uv project, not a script:
# - 15+ dependencies
# - Database models
# - API routes
# - Background workers
# - Multiple configuration files
# - 1000+ lines of code
```
### ❌ Heavy Dependencies
**WRONG**: Using heavy ML/data libraries in scripts:
```python
# /// script
# dependencies = [
# "tensorflow>=2.15.0", # ❌ Too heavy for script
# "torch>=2.1.0", # ❌ Too heavy for script
# "transformers>=4.35.0", # ❌ Too heavy for script
# ]
# ///
```
**Why**: These create very large environments. Use a proper project instead.
**OK**: Lightweight data processing:
```python
# /// script
# dependencies = [
# "polars>=0.20.0", # ✓ Reasonable for scripts
# "httpx>=0.27.0", # ✓ Reasonable for scripts
# ]
# ///
```
## Common Syntax Mistakes
### ❌ Missing # on TOML Lines
**WRONG**:
```python
# /// script
requires-python = ">=3.11" # ❌ Missing # at start
dependencies = [] # ❌ Missing # at start
# ///
```
**CORRECT**:
```python
# /// script
# requires-python = ">=3.11" # ✓ Each line starts with #
# dependencies = [] # ✓ Each line starts with #
# ///
```
### ❌ Wrong Marker Format
**WRONG**:
```python
# /// scripts # ❌ Wrong: "scripts" not "script"
# requires-python = ">=3.11"
# ///
# // script # ❌ Wrong: "//" not "///"
# requires-python = ">=3.11"
# //
# /// script # ❌ Wrong: No closing marker
# requires-python = ">=3.11"
```
**CORRECT**:
```python
# /// script # ✓ Exactly "/// script"
# requires-python = ">=3.11"
# /// # ✓ Exactly "///"
```
## Dependency Management Mistakes
### ❌ No Version Constraints
**WRONG**:
```python
# /// script
# dependencies = [
# "httpx", # ❌ No version specified
# "requests", # ❌ Could break unexpectedly
# ]
# ///
```
**CORRECT**:
```python
# /// script
# dependencies = [
# "httpx>=0.27.0", # ✓ Minimum version specified
# "requests>=2.31.0", # ✓ Prevents breaking changes
# ]
# ///
```
### ❌ Overly Strict Pinning
**WRONG** (for utility scripts):
```python
# /// script
# dependencies = [
# "httpx==0.27.0", # ❌ Too strict, prevents updates
# "rich==13.7.0", # ❌ Blocks security fixes
# ]
# ///
```
**CORRECT**:
```python
# /// script
# dependencies = [
# "httpx>=0.27.0", # ✓ Allows updates
# "rich>=13.0.0", # ✓ Allows security fixes
# ]
# ///
```
**NOTE**: Exact pinning (`==`) is appropriate for deployment scripts where reproducibility is critical.
## Additional Pitfalls
### ❌ Missing Shebang
**WRONG**:
```python
# No shebang - script won't be directly executable
# /// script
# dependencies = ["requests"]
# ///
```
**CORRECT**:
```python
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = ["requests"]
# ///
```
### ❌ Broad Exception Handling
**WRONG**:
```python
try:
do_something()
except Exception: # Too broad!
pass
```
**CORRECT**:
```python
try:
do_something()
except (FileNotFoundError, PermissionError) as e:
print(f"Error: {e}", file=sys.stderr)
sys.exit(1)
```
### ❌ Platform-Specific Code Without Guards
**WRONG**:
```python
import pwd # Unix-only, crashes on Windows
user = pwd.getpwuid(os.getuid())
```
**CORRECT**:
```python
import sys
if sys.platform != "win32":
import pwd
user = pwd.getpwuid(os.getuid())
else:
user = os.environ.get("USERNAME")
```
## Summary
**Never use**:
- `[tool.uv.metadata]` (invalid field)
- Custom fields in PEP 723 metadata (only `requires-python` and `dependencies`)
- Hardcoded secrets
- Scripts for complex applications (>500 lines, web apps, services)
**Always use**:
- Valid PEP 723 format only
- `[tool.uv]` (without `.metadata`) for valid fields like `exclude-newer`
- Environment variables or Infisical for secrets
- Error handling for external calls
- Version constraints on dependencies
- Proper projects when scripts become too complex

View File

@@ -0,0 +1,363 @@
# When NOT to Use Single-File Scripts
This document helps determine when to use a single-file uv script vs. when to use a proper uv project.
## Decision Tree
```text
Is this a Python program?
├─ No → Use appropriate language/tool
└─ Yes
├─ Is it a one-time task or simple automation?
│ ├─ Yes → Consider single-file script
│ └─ No → Use proper uv project
└─ Does it meet ANY of these criteria?
├─ >500 lines of code → Use proper uv project
├─ Multiple Python files needed → Use proper uv project
├─ Web application or API → Use proper uv project
├─ Long-running service → Use proper uv project
├─ Complex configuration → Use proper uv project
├─ Shared library code → Use proper uv project
├─ Heavy ML/data dependencies → Use proper uv project
└─ None of the above → Single-file script is appropriate
```
## Use Proper uv Project When
### 1. Code Complexity
**Use project if**:
- Script exceeds 500 lines
- Logic is spread across multiple functions/classes
- Code would benefit from splitting into modules
- Complex data models or class hierarchies
**Example - Too Complex for Script**:
```python
# This needs a proper project structure:
# - 800 lines of code
# - Multiple classes (User, Database, API, Config)
# - Would be clearer as separate modules
# - Needs tests
```
### 2. Multiple Files Needed
**Use project if**:
- Shared utilities across multiple scripts
- Common data models used by different tools
- Reusable library code
- Multiple entry points
**Example**:
```text
# This needs a project:
my-tool/
├── src/
│ ├── __init__.py
│ ├── database.py # Shared by multiple tools
│ ├── models.py # Common data structures
│ └── utils.py # Utility functions
├── scripts/
│ ├── import_data.py # Uses shared code
│ └── export_data.py # Uses shared code
└── pyproject.toml
```
### 3. Web Applications
**Use project for**:
- Flask/FastAPI/Django applications
- REST APIs
- Web services
- Applications with routes/controllers
**Example - Needs Project**:
```python
# Don't use single-file script for web apps:
from flask import Flask, request, jsonify
app = Flask(__name__)
# 50+ routes
# Database models
# Authentication
# Background tasks
# Configuration management
# This should be a proper project structure
```
### 4. Long-Running Services
**Use project for**:
- Daemons
- Background workers
- Queue consumers
- Services that run continuously
**Example**:
```python
# Don't use script for services:
# - Runs 24/7
# - Monitors message queue
# - Complex retry logic
# - Logging configuration
# - Health checks
# - Graceful shutdown
# Needs proper project structure with:
# - Proper logging setup
# - Configuration management
# - Process management (systemd/supervisor)
```
### 5. Complex Configuration
**Use project if**:
- Multiple environment configs (dev/staging/prod)
- YAML/JSON configuration files
- Feature flags
- Database connection pools
**Example**:
```text
# This needs a project:
config/
├── dev.yaml
├── staging.yaml
└── production.yaml
# Single-file scripts should use simple env vars instead
```
### 6. Heavy Dependencies
**Use project for**:
- Machine learning frameworks (TensorFlow, PyTorch)
- Large data processing (PySpark)
- Complex scientific computing
- GUI frameworks
**Example - Too Heavy**:
```python
# /// script
# dependencies = [
# "tensorflow>=2.15.0", # ❌ ~500MB download
# "torch>=2.1.0", # ❌ ~800MB download
# "transformers>=4.35.0", # ❌ Complex dependency tree
# ]
# ///
# Use proper project with managed virtual environment
```
### 7. Testing Requirements
**Use project if**:
- Comprehensive test suite needed
- Multiple test files
- Fixtures and mocking
- CI/CD integration
**Example**:
```text
# This needs a project:
tests/
├── unit/
│ ├── test_models.py
│ ├── test_utils.py
│ └── test_api.py
├── integration/
│ └── test_database.py
└── conftest.py
# Can't reasonably organize this with single-file script
```
### 8. Team Collaboration
**Use project if**:
- Multiple developers working on code
- Code review processes
- Versioning and releases
- Documentation requirements
## Single-File Scripts ARE Appropriate For
### ✅ Good Use Cases
**One-time tasks**:
```python
# Convert CSV format
# Migrate data between systems
# Clean up old files
# Generate reports
```
**Simple automation**:
```python
# Check server health
# Send notifications
# Backup files
# Parse logs
```
**CLI utilities**:
```python
# Format files
# Validate data
# Query APIs
# Process input
```
**Prototyping**:
```python
# Test API endpoints
# Experiment with libraries
# Quick data analysis
# Proof of concept
```
### ✅ Characteristics of Good Single-File Scripts
- **<500 lines of code**
- **Self-contained logic**
- **Simple, clear purpose**
- **Minimal dependencies (1-5 packages)**
- **Standalone execution**
- **Quick to understand**
## Examples of Good Single-File Scripts
### Example 1: Health Check
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "psutil>=5.9.0",
# "rich>=13.0.0",
# ]
# ///
"""Check system health and display metrics."""
import psutil
from rich import print
def main():
cpu = psutil.cpu_percent()
mem = psutil.virtual_memory().percent
disk = psutil.disk_usage('/').percent
print(f"CPU: {cpu}% | Memory: {mem}% | Disk: {disk}%")
if cpu > 80 or mem > 80 or disk > 80:
print("[red]⚠ High resource usage![/red]")
exit(1)
if __name__ == "__main__":
main()
```
### Example 2: API Query
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# ]
# ///
"""Query API and display results."""
import httpx
import sys
def main():
response = httpx.get("https://api.github.com/users/octocat")
response.raise_for_status()
data = response.json()
print(f"Name: {data['name']}")
print(f"Public repos: {data['public_repos']}")
if __name__ == "__main__":
main()
```
## Migration Path
**When a script outgrows single-file format**:
1. Create proper uv project:
```bash
uv init my-tool
cd my-tool
```
2. Move script logic to `src/`:
```bash
mv script.py src/my_tool/main.py
```
3. Add dependencies to `pyproject.toml`:
```toml
[project]
dependencies = [
"httpx>=0.27.0",
"rich>=13.0.0",
]
```
4. Create entry point in `pyproject.toml`:
```toml
[project.scripts]
my-tool = "my_tool.main:main"
```
## Summary
**Use single-file scripts for**:
- Simple automation (<500 lines)
- One-off tasks
- CLI utilities
- Prototypes
- Standalone tools
**Use proper uv projects for**:
- Complex applications (>500 lines)
- Multiple files/modules
- Web applications
- Long-running services
- Heavy dependencies
- Team collaboration
- Comprehensive testing
**When in doubt**: Start with a script. If it grows too complex, migrate to a project.

View File

@@ -0,0 +1,72 @@
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# "rich>=13.0.0",
# ]
# ///
"""
API client template using httpx.
Demonstrates HTTP requests, error handling, and structured output.
Includes environment variable usage for API credentials.
Usage:
export API_URL="https://api.example.com"
export API_TOKEN="your-token"
python api-client.py
"""
import httpx
import os
import sys
from rich import print
from rich.console import Console
console = Console()
def get_env_var(name: str) -> str:
"""Get required environment variable or exit with error."""
value = os.getenv(name)
if not value:
console.print(f"[red]Error: {name} environment variable not set[/red]")
sys.exit(1)
return value
def fetch_data(api_url: str, token: str):
"""Fetch data from API with error handling."""
headers = {"Authorization": f"Bearer {token}"}
try:
with httpx.Client() as client:
response = client.get(api_url, headers=headers, timeout=10.0)
response.raise_for_status()
return response.json()
except httpx.HTTPStatusError as e:
console.print(f"[red]HTTP error {e.response.status_code}[/red]")
console.print(f"Response: {e.response.text}")
sys.exit(1)
except httpx.RequestError as e:
console.print(f"[red]Request failed: {e}[/red]")
sys.exit(1)
def main():
"""Main entry point."""
api_url = get_env_var("API_URL")
api_token = get_env_var("API_TOKEN")
console.print(f"[cyan]Fetching data from {api_url}...[/cyan]")
data = fetch_data(api_url, api_token)
# Process and display data
print(data)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,24 @@
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""
Basic Python script template using uv.
This is a minimal working example showing the correct PEP 723 format.
No external dependencies - uses only Python standard library.
Usage:
python basic-script.py
"""
def main():
"""Main entry point for the script."""
print("Hello from uv!")
# Your code here
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,62 @@
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "typer>=0.9.0",
# "rich>=13.0.0",
# ]
# ///
"""
CLI application template using Typer and Rich.
Demonstrates command-line argument parsing, subcommands, and formatted output.
Usage:
python cli-app.py --help
python cli-app.py greet "World"
python cli-app.py process input.txt --output output.txt
"""
import typer
from rich import print
from rich.console import Console
from pathlib import Path
app = typer.Typer()
console = Console()
@app.command()
def greet(name: str):
"""Greet someone by name."""
print(f"[green]Hello, {name}![/green]")
@app.command()
def process(
input_file: Path = typer.Argument(..., help="Input file to process"),
output: Path = typer.Option(None, "--output", "-o", help="Output file path"),
):
"""Process a file and optionally write results."""
if not input_file.exists():
console.print(f"[red]Error: {input_file} not found[/red]")
raise typer.Exit(code=1)
# Process file
with open(input_file) as f:
content = f.read()
console.print(f"[cyan]Processing {input_file}...[/cyan]")
# Your processing logic here
result = content.upper() # Example transformation
if output:
output.write_text(result)
console.print(f"[green]✓ Written to {output}[/green]")
else:
print(result)
if __name__ == "__main__":
app()

View File

@@ -0,0 +1,84 @@
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "polars>=0.20.0",
# "rich>=13.0.0",
# ]
# ///
"""
Data processing template using Polars.
Demonstrates reading, transforming, and analyzing data with
modern data processing libraries.
Usage:
python data-processor.py input.csv
python data-processor.py data/*.csv --output results/
"""
import sys
from pathlib import Path
import polars as pl
from rich.console import Console
from rich.table import Table
console = Console()
def process_csv(file_path: Path) -> pl.DataFrame:
"""Read and process a CSV file."""
try:
df = pl.read_csv(file_path)
console.print(f"[cyan]Loaded {len(df)} rows from {file_path.name}[/cyan]")
# Example transformations
# df = df.filter(pl.col("status") == "active")
# df = df.with_columns(pl.col("amount").cast(pl.Float64))
return df
except Exception as e:
console.print(f"[red]Error processing {file_path}: {e}[/red]")
sys.exit(1)
def display_summary(df: pl.DataFrame):
"""Display data summary using Rich tables."""
table = Table(title="Data Summary")
table.add_column("Metric", style="cyan")
table.add_column("Value", style="green")
table.add_row("Total Rows", str(len(df)))
table.add_row("Columns", str(len(df.columns)))
# Example statistics
# if "amount" in df.columns:
# table.add_row("Total Amount", f"${df['amount'].sum():,.2f}")
console.print(table)
def main():
"""Main entry point."""
if len(sys.argv) < 2:
console.print("[red]Usage: python data-processor.py <input.csv>[/red]")
sys.exit(1)
input_path = Path(sys.argv[1])
if not input_path.exists():
console.print(f"[red]Error: {input_path} not found[/red]")
sys.exit(1)
df = process_csv(input_path)
display_summary(df)
# Optional: Save results
# output_path = Path("output.csv")
# df.write_csv(output_path)
# console.print(f"[green]✓ Saved to {output_path}[/green]")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,257 @@
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "rich>=13.0.0",
# "typer>=0.9.0",
# ]
# ///
"""
Proxmox Cluster Health Checker - Production Ready Example
Purpose: cluster-monitoring
Team: infrastructure
Author: devops@spaceships.work
This is an enhanced version of the basic cluster health checker,
demonstrating all best practices for production uv scripts.
Features:
- Rich CLI with Typer
- Structured output with Rich
- Proper error handling
- Input validation
- Security best practices
- Comprehensive logging
- Exit codes for automation
Usage:
# Interactive mode with rich output
./check_cluster_health_enhanced.py --node foxtrot
# JSON output for automation
./check_cluster_health_enhanced.py --node foxtrot --json
# Quiet mode (only errors)
./check_cluster_health_enhanced.py --node foxtrot --quiet
Examples:
# Check specific cluster node
./check_cluster_health_enhanced.py --node golf
# CI/CD integration
./check_cluster_health_enhanced.py --json | jq '.is_healthy'
"""
import json
import re
import subprocess
import sys
from dataclasses import dataclass, asdict
from typing import Optional
import typer
from rich.console import Console
from rich.table import Table
from rich import print as rprint
app = typer.Typer(help="Check Proxmox cluster health")
console = Console()
@dataclass
class NodeStatus:
"""Cluster node status"""
name: str
online: bool
node_id: int
ip: str
@dataclass
class ClusterHealth:
"""Overall cluster health status"""
cluster_name: str
quorate: bool
node_count: int
expected_votes: int
total_votes: int
nodes: list[NodeStatus]
warnings: list[str]
errors: list[str]
@property
def is_healthy(self) -> bool:
"""Check if cluster is healthy"""
return self.quorate and len(self.errors) == 0
def validate_hostname(hostname: str) -> bool:
"""Validate hostname format"""
pattern = r'^[a-zA-Z0-9][a-zA-Z0-9.-]*[a-zA-Z0-9]$'
return bool(re.match(pattern, hostname)) and len(hostname) <= 253
def run_ssh_command(node: str, command: str) -> str:
"""Execute command on remote node via SSH"""
if not validate_hostname(node):
console.print(f"[red]Error: Invalid hostname: {node}[/red]", file=sys.stderr)
raise typer.Exit(code=1)
try:
result = subprocess.run(
["ssh", f"root@{node}", command],
capture_output=True,
text=True,
check=True,
timeout=30
)
return result.stdout
except subprocess.CalledProcessError as e:
console.print(f"[red]Command failed: {command}[/red]", file=sys.stderr)
console.print(f" {e.stderr}", file=sys.stderr)
raise typer.Exit(code=1)
except subprocess.TimeoutExpired:
console.print(f"[red]Command timed out: {command}[/red]", file=sys.stderr)
raise typer.Exit(code=1)
def check_cluster_status(node: str) -> ClusterHealth:
"""Check cluster status and return health object"""
health = ClusterHealth(
cluster_name="",
quorate=False,
node_count=0,
expected_votes=0,
total_votes=0,
nodes=[],
warnings=[],
errors=[]
)
# Get cluster status
output = run_ssh_command(node, "pvecm status")
# Parse cluster name
if match := re.search(r'Cluster name:\s+(\S+)', output):
health.cluster_name = match.group(1)
# Parse quorum
if match := re.search(r'Quorate:\s+(\w+)', output):
health.quorate = match.group(1).lower() == 'yes'
if not health.quorate:
health.errors.append("Cluster does not have quorum!")
# Parse votes
if match := re.search(r'Nodes:\s+(\d+)', output):
health.node_count = int(match.group(1))
if match := re.search(r'Expected votes:\s+(\d+)', output):
health.expected_votes = int(match.group(1))
if match := re.search(r'Total votes:\s+(\d+)', output):
health.total_votes = int(match.group(1))
# Get node list
output = run_ssh_command(node, "pvecm nodes")
for line in output.strip().split('\n')[1:]: # Skip header
if not line.strip():
continue
parts = line.split()
if len(parts) >= 3:
try:
health.nodes.append(NodeStatus(
name=parts[2] if len(parts) >= 3 else "unknown",
online=True,
node_id=int(parts[0]),
ip=parts[3] if len(parts) >= 4 else "unknown"
))
except (ValueError, IndexError):
health.warnings.append(f"Failed to parse node line: {line}")
return health
@app.command()
def main(
node: str = typer.Option("foxtrot", help="Cluster node to check"),
json_output: bool = typer.Option(False, "--json", help="Output as JSON"),
quiet: bool = typer.Option(False, help="Only show errors"),
):
"""
Check Proxmox cluster health
Connects to a cluster node via SSH and checks:
- Quorum status
- Node membership
- Vote distribution
"""
try:
health = check_cluster_status(node)
if json_output:
# JSON output for automation
print(json.dumps(asdict(health), indent=2))
sys.exit(0 if health.is_healthy else 1)
if not quiet:
# Rich table output
console.print(f"\n[bold]Cluster Health: {health.cluster_name}[/bold]")
console.print("=" * 60)
# Status
quorum_icon = "" if health.quorate else ""
quorum_color = "green" if health.quorate else "red"
console.print(f"Quorum: [{quorum_color}]{quorum_icon} {'YES' if health.quorate else 'NO'}[/{quorum_color}]")
console.print(f"Nodes: {health.node_count} ({health.total_votes}/{health.expected_votes} votes)")
# Nodes table
table = Table(title="\nCluster Nodes")
table.add_column("Node", style="cyan")
table.add_column("Node ID", style="magenta")
table.add_column("IP Address", style="yellow")
table.add_column("Status", style="green")
for node_status in health.nodes:
status_icon = "" if node_status.online else ""
table.add_row(
node_status.name,
str(node_status.node_id),
node_status.ip,
status_icon
)
console.print(table)
# Warnings
if health.warnings:
console.print("\n[yellow]Warnings:[/yellow]")
for warning in health.warnings:
console.print(f"{warning}")
# Errors
if health.errors:
console.print("\n[red]Errors:[/red]")
for error in health.errors:
console.print(f"{error}")
console.print("\n" + "=" * 60)
# Final status
if health.is_healthy:
if not quiet:
console.print("[green]Status: ✓ HEALTHY[/green]\n")
sys.exit(0)
else:
console.print("[red]Status: ✗ UNHEALTHY[/red]\n", file=sys.stderr)
sys.exit(1)
except Exception as e:
console.print(f"[red]Unexpected error: {e}[/red]", file=sys.stderr)
sys.exit(2)
if __name__ == "__main__":
app()

View File

@@ -0,0 +1,358 @@
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "pynetbox>=7.0.0",
# "infisical-python>=2.3.3",
# "rich>=13.0.0",
# ]
# ///
"""
NetBox API Client Example
Purpose: api-client-example
Team: infrastructure
Author: devops@spaceships.work
Demonstrates best practices for API client scripts using uv:
- PEP 723 inline dependencies
- Infisical for secrets management
- Error handling and validation
- Rich output formatting
- Type hints and documentation
This is a production-ready example showing all patterns together.
Usage:
# List Matrix cluster VMs
./netbox_client.py
# Query specific VM
./netbox_client.py --vm docker-01
Example output:
Matrix Cluster Virtual Machines
┌────┬───────────┬──────┬─────────┬──────────────┬──────────────────────────────────┐
│ ID │ Name │vCPUs │Memory MB│ Status │ Primary IP │
├────┼───────────┼──────┼─────────┼──────────────┼──────────────────────────────────┤
│ 1 │ docker-01 │ 4 │ 8192 │ active │ 192.168.3.10/24 │
│ 2 │ k8s-01 │ 8 │ 16384 │ active │ 192.168.3.20/24 │
└────┴───────────┴──────┴─────────┴──────────────┴──────────────────────────────────┘
"""
import re
import sys
from typing import Optional
from dataclasses import dataclass
import pynetbox
from infisical import InfisicalClient
from rich.console import Console
from rich.table import Table
from rich.panel import Panel
console = Console()
# ============================================================================
# Configuration
# ============================================================================
@dataclass
class NetBoxConfig:
"""NetBox connection configuration."""
url: str = "https://netbox.spaceships.work"
project_id: str = "7b832220-24c0-45bc-a5f1-ce9794a31259"
environment: str = "prod"
path: str = "/matrix"
# ============================================================================
# Authentication
# ============================================================================
def get_netbox_client(config: NetBoxConfig) -> Optional[pynetbox.api]:
"""
Get authenticated NetBox API client.
Uses Infisical to securely retrieve API token (Virgo-Core security pattern).
Args:
config: NetBox configuration
Returns:
Authenticated pynetbox client or None on error
Raises:
Exception: If token cannot be retrieved or connection fails
"""
try:
# Get token from Infisical (never hardcoded)
client = InfisicalClient()
token = client.get_secret(
secret_name="NETBOX_API_TOKEN",
project_id=config.project_id,
environment=config.environment,
path=config.path
).secret_value
if not token:
raise ValueError("NETBOX_API_TOKEN is empty")
# Connect to NetBox
nb = pynetbox.api(config.url, token=token)
# Test connection
_ = nb.status()
return nb
except Exception as e:
console.print(f"[red]Failed to connect to NetBox: {e}[/red]")
return None
# ============================================================================
# Data Validation
# ============================================================================
def validate_vm_name(name: str) -> bool:
"""
Validate VM name format.
Pattern: lowercase letters, numbers, hyphens only
Example: docker-01, k8s-01-master
Args:
name: VM name to validate
Returns:
True if valid, False otherwise
"""
pattern = r'^[a-z0-9-]+$'
return bool(re.match(pattern, name))
# ============================================================================
# NetBox Queries
# ============================================================================
def get_cluster_vms(nb: pynetbox.api, cluster_name: str = "Matrix") -> list:
"""
Get all VMs in a cluster.
Args:
nb: NetBox API client
cluster_name: Cluster name to query
Returns:
List of VM objects
"""
try:
vms = nb.virtualization.virtual_machines.filter(cluster=cluster_name.lower())
return list(vms)
except Exception as e:
console.print(f"[red]Error querying VMs: {e}[/red]")
return []
def get_vm_details(nb: pynetbox.api, vm_name: str) -> Optional[dict]:
"""
Get detailed information about a VM.
Args:
nb: NetBox API client
vm_name: VM name
Returns:
VM details dict or None if not found
"""
try:
# Validate name
if not validate_vm_name(vm_name):
console.print(f"[red]Invalid VM name format: {vm_name}[/red]")
return None
vm = nb.virtualization.virtual_machines.get(name=vm_name)
if not vm:
console.print(f"[yellow]VM '{vm_name}' not found[/yellow]")
return None
# Get interfaces
interfaces = nb.virtualization.interfaces.filter(virtual_machine_id=vm.id)
# Get IPs
ips = []
for iface in interfaces:
iface_ips = nb.ipam.ip_addresses.filter(vminterface_id=iface.id)
ips.extend(iface_ips)
return {
"vm": vm,
"interfaces": list(interfaces),
"ips": ips
}
except Exception as e:
console.print(f"[red]Error getting VM details: {e}[/red]")
return None
# ============================================================================
# Output Formatting
# ============================================================================
def display_vms_table(vms: list) -> None:
"""
Display VMs in a formatted table.
Args:
vms: List of VM objects
"""
if not vms:
console.print("[yellow]No VMs found[/yellow]")
return
table = Table(title="Matrix Cluster Virtual Machines")
table.add_column("ID", style="cyan", justify="right")
table.add_column("Name", style="green")
table.add_column("vCPUs", justify="right")
table.add_column("Memory (MB)", justify="right")
table.add_column("Status")
table.add_column("Primary IP", style="yellow")
for vm in vms:
table.add_row(
str(vm.id),
vm.name,
str(vm.vcpus) if vm.vcpus else "N/A",
str(vm.memory) if vm.memory else "N/A",
vm.status.value if hasattr(vm.status, 'value') else str(vm.status),
str(vm.primary_ip4.address) if vm.primary_ip4 else "N/A"
)
console.print(table)
console.print(f"\n[green]Total VMs: {len(vms)}[/green]")
def display_vm_details(details: dict) -> None:
"""
Display detailed VM information.
Args:
details: VM details dict from get_vm_details()
"""
vm = details["vm"]
interfaces = details["interfaces"]
ips = details["ips"]
# VM info panel
info = (
f"[green]ID:[/green] {vm.id}\n"
f"[green]Name:[/green] {vm.name}\n"
f"[green]Cluster:[/green] {vm.cluster.name if vm.cluster else 'N/A'}\n"
f"[green]Status:[/green] {vm.status}\n"
f"[green]vCPUs:[/green] {vm.vcpus or 'N/A'}\n"
f"[green]Memory:[/green] {vm.memory or 'N/A'} MB\n"
f"[green]Disk:[/green] {vm.disk or 'N/A'} GB\n"
f"[green]Primary IP:[/green] {vm.primary_ip4.address if vm.primary_ip4 else 'N/A'}"
)
console.print(Panel(info, title=f"VM: {vm.name}", border_style="cyan"))
# Interfaces table
if interfaces:
iface_table = Table(title="Network Interfaces")
iface_table.add_column("Name", style="cyan")
iface_table.add_column("Type")
iface_table.add_column("Enabled")
iface_table.add_column("MTU")
for iface in interfaces:
iface_table.add_row(
iface.name,
iface.type.value if hasattr(iface.type, 'value') else str(iface.type),
"" if iface.enabled else "",
str(iface.mtu) if iface.mtu else "default"
)
console.print("\n", iface_table)
# IPs table
if ips:
ip_table = Table(title="IP Addresses")
ip_table.add_column("Address", style="yellow")
ip_table.add_column("DNS Name", style="green")
ip_table.add_column("Status")
for ip in ips:
ip_table.add_row(
str(ip.address),
ip.dns_name or "",
ip.status.value if hasattr(ip.status, 'value') else str(ip.status)
)
console.print("\n", ip_table)
# ============================================================================
# Main
# ============================================================================
def main(vm_name: Optional[str] = None) -> int:
"""
Main entry point.
Args:
vm_name: Optional specific VM to query
Returns:
Exit code (0 = success, 1 = error)
"""
config = NetBoxConfig()
# Get NetBox client
nb = get_netbox_client(config)
if not nb:
return 1
try:
if vm_name:
# Query specific VM
details = get_vm_details(nb, vm_name)
if details:
display_vm_details(details)
return 0
return 1
else:
# List all VMs in Matrix cluster
vms = get_cluster_vms(nb, cluster_name="Matrix")
display_vms_table(vms)
return 0
except KeyboardInterrupt:
console.print("\n[yellow]Interrupted by user[/yellow]")
return 1
except Exception as e:
console.print(f"[red]Unexpected error: {e}[/red]")
return 1
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(
description="NetBox API client example",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__
)
parser.add_argument(
"--vm",
help="Specific VM to query (default: list all)"
)
args = parser.parse_args()
sys.exit(main(vm_name=args.vm))

View File

@@ -0,0 +1,209 @@
# API Client Patterns
Patterns for building API clients with uv scripts.
## Basic GET Request
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# ]
# ///
import httpx
response = httpx.get("https://api.github.com/users/octocat")
response.raise_for_status()
data = response.json()
print(f"Name: {data['name']}")
```
## Authenticated Requests
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# ]
# ///
import httpx
import os
import sys
api_token = os.getenv("API_TOKEN")
if not api_token:
print("Error: API_TOKEN not set", file=sys.stderr)
sys.exit(1)
headers = {"Authorization": f"Bearer {api_token}"}
response = httpx.get(
"https://api.example.com/data",
headers=headers,
timeout=10.0
)
response.raise_for_status()
print(response.json())
```
## POST with JSON
```python
import httpx
data = {
"name": "example",
"status": "active"
}
response = httpx.post(
"https://api.example.com/resources",
json=data,
headers={"Authorization": f"Bearer {token}"},
timeout=10.0
)
response.raise_for_status()
result = response.json()
print(f"Created: {result['id']}")
```
## Query Parameters
```python
import httpx
params = {
"q": "python",
"sort": "stars",
"order": "desc"
}
response = httpx.get(
"https://api.github.com/search/repositories",
params=params
)
response.raise_for_status()
repos = response.json()
```
## Error Handling
```python
import httpx
import sys
try:
with httpx.Client(timeout=10.0) as client:
response = client.get("https://api.example.com/data")
response.raise_for_status()
data = response.json()
except httpx.HTTPStatusError as e:
status = e.response.status_code
if status == 401:
print("Error: Unauthorized - check API key", file=sys.stderr)
elif status == 404:
print("Error: Resource not found", file=sys.stderr)
elif status >= 500:
print(f"Error: Server error ({status})", file=sys.stderr)
else:
print(f"Error: HTTP {status}", file=sys.stderr)
sys.exit(1)
except httpx.RequestError as e:
print(f"Error: Request failed - {type(e).__name__}", file=sys.stderr)
sys.exit(1)
```
## Retry Logic
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# "tenacity>=8.2.0",
# ]
# ///
import httpx
from tenacity import retry, stop_after_attempt, wait_exponential
@retry(
stop=stop_after_attempt(3),
wait=wait_exponential(multiplier=1, min=2, max=10)
)
def fetch_data(url: str):
"""Fetch data with automatic retry."""
response = httpx.get(url, timeout=10.0)
response.raise_for_status()
return response.json()
data = fetch_data("https://api.example.com/data")
```
## Pagination
```python
import httpx
def fetch_all_pages(base_url: str, headers: dict):
"""Fetch all pages from paginated API."""
all_results = []
next_url = base_url
with httpx.Client(headers=headers, timeout=10.0) as client:
while next_url:
response = client.get(next_url)
response.raise_for_status()
data = response.json()
all_results.extend(data["results"])
# Get next page URL
next_url = data.get("next")
return all_results
```
## Rate Limiting
```python
import httpx
import time
def fetch_with_rate_limit(urls: list[str], requests_per_second: int = 2):
"""Fetch URLs respecting rate limit."""
delay = 1.0 / requests_per_second
results = []
for url in urls:
response = httpx.get(url)
response.raise_for_status()
results.append(response.json())
time.sleep(delay)
return results
```
## Complete Example
For a complete API client template, see: `assets/templates/api-client.py`
## Best Practices
- Always set timeouts (default: 10 seconds)
- Use `with httpx.Client()` for multiple requests
- Handle specific HTTP status codes (401, 404, 500)
- Don't log sensitive data (tokens, responses)
- Use environment variables for credentials
- Implement retry logic for transient failures
- Respect rate limits

View File

@@ -0,0 +1,129 @@
# CLI Application Patterns
Patterns for building command-line applications with uv scripts.
## Basic CLI with Typer
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "typer>=0.9.0",
# "rich>=13.0.0",
# ]
# ///
import typer
from rich import print
app = typer.Typer()
@app.command()
def main(
name: str = typer.Argument(..., help="Your name"),
greeting: str = typer.Option("Hello", "--greeting", "-g"),
):
"""Greet someone."""
print(f"[green]{greeting}, {name}![/green]")
if __name__ == "__main__":
app()
```
## Multiple Subcommands
```python
import typer
app = typer.Typer()
@app.command()
def create(name: str):
"""Create a new resource."""
print(f"Creating: {name}")
@app.command()
def delete(name: str):
"""Delete a resource."""
print(f"Deleting: {name}")
if __name__ == "__main__":
app()
```
Usage:
```bash
./script.py create foo
./script.py delete bar
```
## File Input/Output
```python
import typer
from pathlib import Path
def process_file(
input_file: Path = typer.Argument(..., exists=True),
output: Path = typer.Option(None, "--output", "-o"),
):
"""Process a file."""
content = input_file.read_text()
# Process
result = content.upper()
if output:
output.write_text(result)
print(f"Written to: {output}")
else:
print(result)
```
## Progress Bars
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "rich>=13.0.0",
# ]
# ///
from rich.progress import track
import time
for item in track(range(100), description="Processing..."):
time.sleep(0.01) # Simulate work
```
## Formatted Tables
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "rich>=13.0.0",
# ]
# ///
from rich.console import Console
from rich.table import Table
console = Console()
table = Table(title="Results")
table.add_column("Name", style="cyan")
table.add_column("Status", style="green")
table.add_row("Task 1", "✓ Complete")
table.add_row("Task 2", "⏳ Pending")
console.print(table)
```
For complete template, see: `assets/templates/cli-app.py`

View File

@@ -0,0 +1,54 @@
# Data Processing Patterns
> **Status**: 🚧 Placeholder - Content in development
## Overview
Patterns for data analysis, ETL, and processing using Polars, pandas, and other data libraries in UV single-file
scripts.
## Topics to Cover
- [ ] Polars patterns (recommended for performance)
- [ ] Pandas alternatives
- [ ] CSV/Excel processing
- [ ] JSON data manipulation
- [ ] Data validation and cleaning
- [ ] Aggregation and transformation
- [ ] Memory-efficient processing
## Quick Example
```python
#!/usr/bin/env -S uv run
# /// script
# requires-python = ">=3.11"
# dependencies = ["polars>=0.20.0"]
# ///
import polars as pl
def analyze_csv(file_path: str):
df = pl.read_csv(file_path)
# Basic analysis
summary = df.describe()
print(summary)
# Filter and aggregate
result = (
df.filter(pl.col("value") > 100)
.groupby("category")
.agg(pl.col("value").mean())
)
print(result)
```
## TODO
This file will be expanded to include:
- Complete Polars patterns
- Performance optimization techniques
- Large file processing strategies
- Data validation patterns
- Export formats and options

View File

@@ -0,0 +1,198 @@
# Error Handling Patterns
Best practices for error handling in uv scripts.
## Exit Codes
Always use appropriate exit codes:
- `0` - Success
- `1` - General error
- `2` - Invalid usage
```python
import sys
if len(sys.argv) < 2:
print("Usage: script.py <input>", file=sys.stderr)
sys.exit(2) # Invalid usage
try:
result = process()
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
sys.exit(1) # General error
# Success
sys.exit(0)
```
## Try-Except Pattern
```python
import sys
def main():
try:
# Operations that might fail
data = fetch_data()
result = process(data)
save(result)
except FileNotFoundError as e:
print(f"File not found: {e}", file=sys.stderr)
sys.exit(1)
except ValueError as e:
print(f"Invalid data: {e}", file=sys.stderr)
sys.exit(1)
except Exception as e:
print(f"Unexpected error: {e}", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
```
## HTTP Error Handling
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# ]
# ///
import httpx
import sys
try:
response = httpx.get("https://api.example.com", timeout=10.0)
response.raise_for_status()
data = response.json()
except httpx.HTTPStatusError as e:
print(f"HTTP {e.response.status_code} error", file=sys.stderr)
sys.exit(1)
except httpx.RequestError as e:
print(f"Request failed: {type(e).__name__}", file=sys.stderr)
sys.exit(1)
except httpx.TimeoutException:
print("Request timed out", file=sys.stderr)
sys.exit(1)
```
## File Operation Errors
```python
from pathlib import Path
import sys
def read_config(path: Path):
"""Read configuration file with error handling."""
try:
if not path.exists():
raise FileNotFoundError(f"Config not found: {path}")
if not path.is_file():
raise ValueError(f"Not a file: {path}")
return path.read_text()
except PermissionError:
print(f"Permission denied: {path}", file=sys.stderr)
sys.exit(1)
except Exception as e:
print(f"Error reading config: {e}", file=sys.stderr)
sys.exit(1)
```
## Subprocess Errors
```python
import subprocess
import sys
try:
result = subprocess.run(
["command", "arg"],
capture_output=True,
text=True,
check=True
)
print(result.stdout)
except subprocess.CalledProcessError as e:
print(f"Command failed with exit code {e.returncode}", file=sys.stderr)
print(f"Error output: {e.stderr}", file=sys.stderr)
sys.exit(1)
except FileNotFoundError:
print("Command not found", file=sys.stderr)
sys.exit(1)
```
## Validation Errors
```python
import sys
def validate_input(value: str) -> int:
"""Validate and convert input."""
if not value:
print("Error: Empty value", file=sys.stderr)
sys.exit(2)
try:
number = int(value)
except ValueError:
print(f"Error: Not a number: {value}", file=sys.stderr)
sys.exit(2)
if number < 0:
print("Error: Must be non-negative", file=sys.stderr)
sys.exit(2)
return number
```
## Graceful Cleanup
```python
import sys
from pathlib import Path
def main():
temp_file = Path("/tmp/data.tmp")
try:
# Create temporary file
temp_file.write_text("data")
# Process
result = process(temp_file)
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
sys.exit(1)
finally:
# Always cleanup
if temp_file.exists():
temp_file.unlink()
```
## Summary
- Always write errors to `stderr`
- Use specific exception types
- Provide helpful error messages
- Use appropriate exit codes
- Clean up resources in `finally` blocks
- Don't expose secrets in error messages

View File

@@ -0,0 +1,32 @@
# Security Patterns for UV Single-File Scripts
> **Status**: 🚧 Placeholder - Content in development
## Overview
Security patterns and best practices for handling secrets, authentication, and input validation in UV single-file
Python scripts.
## Topics to Cover
- [ ] Infisical secrets management integration
- [ ] Keyring for credential storage
- [ ] Environment variable security
- [ ] Input validation and sanitization
- [ ] API token handling
- [ ] SSH key management
- [ ] Avoiding hardcoded credentials
## Quick Reference
For now, see the main security section in [../reference/security-patterns.md](../reference/security-patterns.md).
## TODO
This file will be expanded to include:
- Complete Infisical integration patterns
- Best practices for secret rotation
- Multi-environment secret management
- Security checklist for scripts
- Common security vulnerabilities and fixes

View File

@@ -0,0 +1,61 @@
# System Automation Patterns
> **Status**: 🚧 Placeholder - Content in development
## Overview
Patterns for system administration, monitoring, and automation using psutil, subprocess, and system libraries in UV
single-file scripts.
## Topics to Cover
- [ ] psutil for system monitoring
- [ ] subprocess for command execution
- [ ] File system operations
- [ ] Process management
- [ ] SSH remote execution
- [ ] Cron/scheduled task integration
- [ ] Log file analysis
## Quick Example
```python
#!/usr/bin/env -S uv run
# /// script
# requires-python = ">=3.11"
# dependencies = ["psutil>=5.9.0", "rich>=13.0.0"]
# ///
import psutil
from rich.console import Console
from rich.table import Table
console = Console()
def show_disk_usage():
table = Table(title="Disk Usage")
table.add_column("Device", style="cyan")
table.add_column("Mount", style="magenta")
table.add_column("Used", style="yellow")
table.add_column("Free", style="green")
for partition in psutil.disk_partitions():
usage = psutil.disk_usage(partition.mountpoint)
table.add_row(
partition.device,
partition.mountpoint,
f"{usage.percent}%",
f"{usage.free / (1024**3):.2f} GB"
)
console.print(table)
```
## TODO
This file will be expanded to include:
- Complete psutil monitoring patterns
- Safe subprocess execution
- SSH automation patterns
- System health checks
- Automated maintenance tasks

View File

@@ -0,0 +1,293 @@
# Converting Bash Scripts to Python uv Scripts
This document guides the conversion of bash scripts to Python uv scripts.
## When Conversion Makes Sense
### ✅ Good Candidates for Conversion
**Convert when**:
- Script needs better error handling
- Cross-platform compatibility required
- Complex data processing needed
- API interactions involved
- Script will grow in complexity
**Examples**:
```bash
# Good candidate - API interaction
curl -X POST https://api.example.com/data \
-H "Authorization: Bearer $TOKEN" \
-d '{"status": "active"}'
# Good candidate - Data processing
cat data.json | jq '.users[] | select(.active == true)'
# Good candidate - Complex logic
for file in $(find /data -name "*.log"); do
count=$(grep -c "ERROR" "$file")
if [ $count -gt 100 ]; then
# Complex processing
fi
done
```
### ❌ Keep as Bash When
**Don't convert when**:
- Simple file operations (cp, mv, mkdir)
- Shell-specific features heavily used (job control, pipes)
- System administration tasks
- Script is <10 lines and works fine
- Team primarily knows bash
**Examples**:
```bash
# Keep as bash - Simple operations
#!/bin/bash
mkdir -p /var/log/app
cp config.yaml /etc/app/
# Keep as bash - Shell-specific
#!/bin/bash
find /data -name "*.tmp" -mtime +7 -delete
# Keep as bash - System admin
#!/bin/bash
systemctl restart nginx
journalctl -u nginx -f
```
## Common Conversions
### File Operations
**Bash**:
```bash
#!/bin/bash
if [ -f "/etc/config.yaml" ]; then
cp /etc/config.yaml /backup/
fi
```
**Python**:
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
from pathlib import Path
import shutil
config = Path("/etc/config.yaml")
if config.exists():
shutil.copy(config, "/backup/")
```
### Environment Variables
**Bash**:
```bash
#!/bin/bash
API_URL=${API_URL:-"https://api.example.com"}
if [ -z "$API_TOKEN" ]; then
echo "Error: API_TOKEN not set" >&2
exit 1
fi
```
**Python**:
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
import os
import sys
API_URL = os.getenv("API_URL", "https://api.example.com")
API_TOKEN = os.getenv("API_TOKEN")
if not API_TOKEN:
print("Error: API_TOKEN not set", file=sys.stderr)
sys.exit(1)
```
### Running Commands
**Bash**:
```bash
#!/bin/bash
set -euo pipefail
output=$(systemctl status nginx)
if [ $? -ne 0 ]; then
echo "Error: nginx not running"
exit 1
fi
```
**Python**:
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
import subprocess
import sys
try:
result = subprocess.run(
["systemctl", "status", "nginx"],
capture_output=True,
text=True,
check=True
)
print(result.stdout)
except subprocess.CalledProcessError:
print("Error: nginx not running", file=sys.stderr)
sys.exit(1)
```
### HTTP Requests
**Bash**:
```bash
#!/bin/bash
response=$(curl -s -X GET https://api.github.com/users/octocat)
name=$(echo "$response" | jq -r '.name')
echo "Name: $name"
```
**Python**:
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# ]
# ///
import httpx
response = httpx.get("https://api.github.com/users/octocat")
response.raise_for_status()
data = response.json()
print(f"Name: {data['name']}")
```
### JSON Processing
**Bash**:
```bash
#!/bin/bash
jq '.users[] | select(.active == true) | .name' data.json
```
**Python**:
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
import json
from pathlib import Path
data = json.loads(Path("data.json").read_text())
active_names = [
user["name"]
for user in data["users"]
if user.get("active")
]
print("\n".join(active_names))
```
## Decision Framework
```text
Should I convert this bash script?
├─ Is it >50 lines? → Consider conversion
├─ Does it process JSON/YAML? → Strong candidate
├─ Does it make API calls? → Strong candidate
├─ Does it have complex logic? → Consider conversion
├─ Does it need better error handling? → Consider conversion
├─ Is it mostly shell commands? → Keep as bash
└─ Is it <10 lines and works? → Keep as bash
```
## Hybrid Approach
Sometimes the best solution is calling bash from Python:
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""
Hybrid: Use Python for logic, bash for system commands.
"""
import subprocess
def backup_with_rsync(source: str, dest: str):
"""Use rsync (better than reimplementing in Python)."""
subprocess.run(
["rsync", "-av", "--delete", source, dest],
check=True
)
# Python logic here
# ...
# Leverage bash tools where appropriate
backup_with_rsync("/data/", "/backup/")
```
## Summary
**Convert to Python when**:
- Complex logic or data processing
- API interactions
- Cross-platform needs
- Better error handling required
- Will grow in complexity
**Keep as Bash when**:
- Simple file operations
- System administration
- Heavily uses shell features
- Works well and won't change
- Team expertise is bash
**Consider Hybrid**:
- Complex Python logic + system commands
- Leverage both Python libraries and shell tools

View File

@@ -0,0 +1,63 @@
# Dependency Management Reference
> **Status**: 🚧 Placeholder - Content in development
## Overview
Comprehensive guide to managing dependencies in UV single-file scripts using PEP 723 inline metadata.
## Topics to Cover
- [ ] Version pinning strategies
- [ ] Semantic versioning best practices
- [ ] Dependency conflict resolution
- [ ] Optional dependencies
- [ ] Development vs production dependencies
- [ ] Security updates and vulnerability scanning
- [ ] Lock file equivalents for scripts
## Quick Reference
### Version Pinning Strategies
**Exact pinning** (most restrictive):
```python
# /// script
# dependencies = ["requests==2.31.0"]
# ///
```
**Compatible release** (recommended):
```python
# /// script
# dependencies = ["requests~=2.31.0"] # >=2.31.0, <2.32.0
# ///
```
**Minimum version**:
```python
# /// script
# dependencies = ["requests>=2.31.0"]
# ///
```
### Following Repository Standards
From this repository's `pyproject.toml`, we use:
- `>=` for minimum versions with flexibility
- Specific version ranges for critical dependencies
- Regular dependency audits with Renovate
## TODO
This file will be expanded to include:
- Complete version specifier syntax
- Dependency resolution strategies
- Security scanning integration
- Update strategies and automation
- Conflict resolution techniques

View File

@@ -0,0 +1,391 @@
# PEP 723: Inline Script Metadata Specification
Complete reference for PEP 723 inline script metadata format used by uv.
## Overview
PEP 723 defines a standardized way to embed dependency and configuration metadata directly within Python script files. This eliminates the need for separate `requirements.txt` files and enables self-contained, reproducible scripts.
## Basic Format
```python
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "package-name>=1.0.0",
# ]
# ///
```
**Key Requirements:**
- Must appear as comments (`#`)
- Must use `# /// script` as opening marker
- Must use `# ///` as closing marker
- Must be valid TOML syntax
- Recommended placement: After shebang, before module docstring
## Complete Specification
### Required Fields
#### requires-python
Specifies minimum Python version:
```python
# /// script
# requires-python = ">=3.11"
# ///
```
**Formats:**
```python
requires-python = ">=3.11" # Minimum version
requires-python = ">=3.11,<3.13" # Version range
requires-python = "==3.12" # Exact version
```
#### dependencies
Lists required packages:
```python
# /// script
# dependencies = [
# "httpx>=0.27.0",
# "rich>=13.0.0",
# "typer~=0.9.0",
# ]
# ///
```
**Version Specifiers:**
```python
"package" # Any version
"package>=1.0" # Minimum version
"package>=1.0,<2.0" # Version range
"package~=1.2.3" # Compatible release (1.2.x)
"package==1.2.3" # Exact version
```
### Optional Fields
#### [tool.uv] Section
uv-specific configuration:
```python
# /// script
# requires-python = ">=3.11"
# dependencies = []
#
# [tool.uv]
# exclude-newer = "2024-10-01T00:00:00Z"
# index-url = "https://pypi.org/simple"
# ///
```
**Available options:**
- `exclude-newer`: Only use packages published before this date
- `index-url`: Alternative PyPI index
- `extra-index-url`: Additional package indexes
- `find-links`: Additional package sources
- `no-index`: Ignore PyPI entirely
#### [tool.uv.sources]
Custom package sources:
```python
# /// script
# dependencies = ["my-package"]
#
# [tool.uv.sources]
# my-package = { git = "https://github.com/user/repo", tag = "v1.0" }
# ///
```
#### Valid [tool.uv] Fields
Additional uv-specific configuration (optional):
```python
# /// script
# requires-python = ">=3.11"
# dependencies = []
#
# [tool.uv]
# exclude-newer = "2025-01-01T00:00:00Z" # Reproducibility constraint
# ///
```
**Note**: For custom metadata like purpose, team, author, use Python docstrings instead:
```python
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""
Purpose: cluster-monitoring
Team: infrastructure
Author: devops@example.com
Created: 2024-10-20
"""
```
## Complete Examples
### Minimal Script
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""Simple script with no dependencies"""
print("Hello, world!")
```
### Production Script
```python
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# "rich>=13.0.0",
# "typer>=0.9.0",
# ]
#
# [tool.uv]
# exclude-newer = "2025-01-01T00:00:00Z"
# ///
"""
API client for Proxmox cluster monitoring
Purpose: api-client
Team: infrastructure
Author: devops@spaceships.work
Usage:
check_cluster.py [--node NODE] [--json]
"""
import typer
import httpx
from rich import print
```
### With Git Dependencies
```python
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "my-internal-lib",
# ]
#
# [tool.uv.sources]
# my-internal-lib = { git = "https://github.com/org/lib.git", tag = "v1.2.3" }
# ///
```
### With Local Dependencies
```python
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "my-local-package",
# ]
#
# [tool.uv.sources]
# my-local-package = { path = "../my-package", editable = true }
# ///
```
## Placement Guidelines
### Correct Placement
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""Module docstring comes after metadata"""
import sys
def main():
pass
if __name__ == "__main__":
main()
```
### Multiple Metadata Blocks (Invalid)
```python
# ❌ INVALID - Only one metadata block allowed
# /// script
# requires-python = ">=3.11"
# ///
# /// script
# dependencies = ["httpx"]
# ///
```
## Validation
### Valid Metadata
```python
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0", # Comments allowed in arrays
# "rich",
# ]
# ///
```
### Invalid Metadata
```python
# ❌ Missing closing marker
# /// script
# requires-python = ">=3.11"
# ❌ Invalid TOML syntax
# /// script
# dependencies = httpx # Missing quotes
# ///
# ❌ Not in comments
/// script
requires-python = ">=3.11"
///
```
## Creating Metadata
### Manual Creation
Add metadata block manually:
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# ]
# ///
import httpx
```
### Using uv init
Generate script with metadata:
```bash
uv init --script my_script.py --python 3.11
```
### Using uv add
Add dependencies to existing script:
```bash
uv add --script my_script.py httpx rich typer
```
## Validation Tools
### Check Metadata Validity
```bash
# Parse metadata
uv run --script my_script.py --dry-run
# Validate with custom tool
python tools/validate_script.py my_script.py
```
### Extract Metadata
```python
import re
import tomllib
def extract_metadata(script_path: str) -> dict:
"""Extract PEP 723 metadata from script"""
with open(script_path) as f:
content = f.read()
# Find metadata block
pattern = r'# /// script\n((?:# .*\n)+)# ///'
match = re.search(pattern, content)
if not match:
return {}
# Parse TOML (remove leading # from each line)
toml_lines = match.group(1).split('\n')
toml_content = '\n'.join(line[2:] for line in toml_lines if line.startswith('# '))
return tomllib.loads(toml_content)
```
## Compatibility
### PEP 723 Support
- ✅ uv (native support)
- ✅ pip (via `pip-run`)
- ✅ pipx (v1.4.0+)
- ⚠️ Other tools (check documentation)
### Fallback for Non-Supporting Tools
```python
#!/usr/bin/env python3
# /// script
# requires-python = ">=3.11"
# dependencies = ["httpx"]
# ///
"""
Fallback installation for non-PEP-723 tools:
pip install httpx
python script.py
"""
```
## Best Practices
1. **Always include requires-python** - Prevents compatibility issues
2. **Pin major versions** - `>=X.Y.Z` for stability
3. **Add metadata section** - Document purpose and ownership
4. **Keep dependencies minimal** - Only what's necessary
5. **Document fallbacks** - Help non-uv users
6. **Validate syntax** - Use validation tools
7. **Version consistently** - Match project conventions
## References
- [PEP 723 Specification](https://peps.python.org/pep-0723/)
- [uv Documentation](https://docs.astral.sh/uv/)
- [TOML Specification](https://toml.io/)
- [Python Version Specifiers](https://peps.python.org/pep-0440/)

View File

@@ -0,0 +1,523 @@
# Security Patterns for uv Scripts
Security best practices for Python single-file scripts following Virgo-Core patterns.
## Never Hardcode Secrets
### ❌ Anti-Pattern: Hardcoded Secrets
```python
# NEVER DO THIS
API_KEY = "sk_live_1234567890abcdef"
DATABASE_URL = "postgresql://user:password@localhost/db"
PROXMOX_PASSWORD = "admin123"
```
**Risks:**
- Secrets exposed in version control
- No audit trail
- Difficult to rotate
- Same credentials across environments
### ✅ Pattern 1: Environment Variables
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
import os
import sys
def get_secret(name: str, required: bool = True) -> str:
"""Get secret from environment"""
value = os.getenv(name)
if required and not value:
print(f"Error: {name} environment variable not set", file=sys.stderr)
sys.exit(1)
return value
# Usage
PROXMOX_PASSWORD = get_secret("PROXMOX_PASSWORD")
API_URL = get_secret("PROXMOX_API_URL")
```
**Usage:**
```bash
export PROXMOX_PASSWORD="$(cat ~/.secrets/proxmox_pass)"
./script.py
```
### ✅ Pattern 2: Keyring Integration
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "keyring>=24.0.0",
# ]
# ///
import keyring
import sys
def get_password(service: str, username: str) -> str:
"""Get password from system keyring"""
password = keyring.get_password(service, username)
if not password:
print(f"Error: No password found for {username}@{service}", file=sys.stderr)
print(f"Set with: keyring set {service} {username}", file=sys.stderr)
sys.exit(1)
return password
# Usage
proxmox_password = get_password("proxmox", "terraform@pam")
```
**Setup:**
```bash
# Store password in system keyring
keyring set proxmox terraform@pam
# Prompts for password, stores securely
# Run script (no password in environment)
./script.py
```
### ✅ Pattern 3: Infisical Integration (Repository Standard)
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "infisical-python>=2.3.3",
# ]
# ///
from infisical import InfisicalClient
import os
import sys
def get_infisical_secret(
secret_name: str,
project_id: str,
environment: str = "prod",
path: str = "/"
) -> str:
"""Get secret from Infisical vault"""
try:
client = InfisicalClient()
secret = client.get_secret(
secret_name=secret_name,
project_id=project_id,
environment=environment,
path=path
)
return secret.secret_value
except Exception as e:
print(f"Error retrieving secret {secret_name}: {e}", file=sys.stderr)
sys.exit(1)
# Usage (following Virgo-Core pattern)
PROXMOX_PASSWORD = get_infisical_secret(
secret_name="PROXMOX_PASSWORD",
project_id="7b832220-24c0-45bc-a5f1-ce9794a31259",
environment="prod",
path="/matrix"
)
```
**Setup:**
```bash
# Authenticate with Infisical
infisical login
# Run script (secrets fetched automatically)
./script.py
```
## Input Validation
### ❌ Anti-Pattern: No Validation
```python
import subprocess
# DANGEROUS - Command injection risk
def ping_host(hostname: str):
subprocess.run(f"ping -c 3 {hostname}", shell=True)
# User provides: "localhost; rm -rf /"
ping_host(sys.argv[1]) # 💥 System destroyed
```
### ✅ Pattern: Validate All Inputs
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
import re
import sys
import subprocess
def validate_hostname(hostname: str) -> bool:
"""Validate hostname format"""
# Only allow alphanumeric, dots, hyphens
pattern = r'^[a-zA-Z0-9][a-zA-Z0-9.-]*[a-zA-Z0-9]$'
if not re.match(pattern, hostname):
return False
# Max length check
if len(hostname) > 253:
return False
return True
def ping_host(hostname: str):
"""Safely ping a host"""
if not validate_hostname(hostname):
print(f"Error: Invalid hostname: {hostname}", file=sys.stderr)
sys.exit(1)
# Use list form (no shell injection)
result = subprocess.run(
["ping", "-c", "3", hostname],
capture_output=True,
text=True
)
return result.returncode == 0
if __name__ == "__main__":
if len(sys.argv) < 2:
print("Usage: ping_host.py <hostname>", file=sys.stderr)
sys.exit(1)
ping_host(sys.argv[1])
```
### Input Validation Patterns
```python
import re
from ipaddress import IPv4Address, AddressValueError
def validate_ip(ip: str) -> bool:
"""Validate IPv4 address"""
try:
IPv4Address(ip)
return True
except AddressValueError:
return False
def validate_port(port: int) -> bool:
"""Validate TCP/UDP port number"""
return 1 <= port <= 65535
def validate_vmid(vmid: int) -> bool:
"""Validate Proxmox VMID (100-999999999)"""
return 100 <= vmid <= 999999999
def validate_path(path: str) -> bool:
"""Validate file path (no directory traversal)"""
# Reject paths with ../
if ".." in path:
return False
# Reject absolute paths
if path.startswith("/"):
return False
return True
```
## Dependency Security
### Pin Dependencies
```python
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0", # ✅ Minimum version pinned
# "rich>=13.0.0", # ✅ Known good version
# ]
# ///
```
**Why pin?**
- Prevents automatic upgrades with vulnerabilities
- Ensures reproducible execution
- Allows controlled dependency updates
### Exclude Recent Packages
```python
# /// script
# requires-python = ">=3.11"
# dependencies = ["httpx>=0.27.0"]
#
# [tool.uv]
# exclude-newer = "2024-10-01T00:00:00Z" # Only packages before this date
# ///
```
**Use cases:**
- Prevent supply chain attacks from compromised packages
- Freeze dependencies at known-good state
- Reproducible builds in CI/CD
### Check for Vulnerabilities
```bash
# Use safety or pip-audit
uv pip install safety
safety check --json
# Or use built-in tools
uv pip list --format json | jq '.[] | select(.name == "httpx")'
```
## File Operations Security
### ❌ Anti-Pattern: Unsafe File Access
```python
# DANGEROUS - Path traversal vulnerability
def read_log(filename: str):
with open(f"/var/log/{filename}") as f:
return f.read()
# User provides: "../../../etc/passwd"
read_log(sys.argv[1]) # 💥 Reads /etc/passwd
```
### ✅ Pattern: Safe File Operations
```python
import os
import sys
from pathlib import Path
def safe_read_log(filename: str, log_dir: str = "/var/log") -> str:
"""Safely read log file"""
# Resolve to absolute paths
log_dir_path = Path(log_dir).resolve()
file_path = (log_dir_path / filename).resolve()
# Ensure file is within log directory
try:
file_path.relative_to(log_dir_path)
except ValueError:
print(f"Error: Path traversal detected: {filename}", file=sys.stderr)
sys.exit(1)
# Check file exists and is readable
if not file_path.exists():
print(f"Error: File not found: {filename}", file=sys.stderr)
sys.exit(1)
if not file_path.is_file():
print(f"Error: Not a file: {filename}", file=sys.stderr)
sys.exit(1)
# Read with size limit
MAX_SIZE = 10 * 1024 * 1024 # 10MB
if file_path.stat().st_size > MAX_SIZE:
print(f"Error: File too large: {filename}", file=sys.stderr)
sys.exit(1)
with open(file_path) as f:
return f.read()
```
## Command Execution Security
### ❌ Anti-Pattern: Shell Injection
```python
# DANGEROUS
import subprocess
def list_directory(path: str):
subprocess.run(f"ls -la {path}", shell=True)
# User provides: "; rm -rf /"
list_directory(sys.argv[1]) # 💥 Disaster
```
### ✅ Pattern: Safe Command Execution
```python
import subprocess
import sys
def list_directory(path: str):
"""Safely list directory contents"""
# Validate path first
if not validate_path(path):
print(f"Error: Invalid path: {path}", file=sys.stderr)
sys.exit(1)
# Use list form (no shell)
try:
result = subprocess.run(
["ls", "-la", path],
capture_output=True,
text=True,
check=True,
timeout=5 # Prevent hanging
)
print(result.stdout)
except subprocess.CalledProcessError as e:
print(f"Error: {e.stderr}", file=sys.stderr)
sys.exit(1)
except subprocess.TimeoutExpired:
print("Error: Command timed out", file=sys.stderr)
sys.exit(1)
```
## Logging and Audit
### Secure Logging
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "structlog>=24.0.0",
# ]
# ///
import structlog
import sys
# Configure structured logging
structlog.configure(
processors=[
structlog.processors.add_log_level,
structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.JSONRenderer(),
],
logger_factory=structlog.PrintLoggerFactory(file=sys.stderr),
)
log = structlog.get_logger()
def process_data(user_id: int, action: str):
"""Process user action with audit logging"""
log.info(
"user_action",
user_id=user_id,
action=action,
# Don't log sensitive data!
)
# ... processing logic ...
log.info(
"action_completed",
user_id=user_id,
action=action,
status="success"
)
```
### Never Log Secrets
```python
# ❌ BAD - Logs password
log.info(f"Connecting with password: {password}")
# ✅ GOOD - No sensitive data
log.info("Connecting to API", endpoint=api_url)
# ✅ GOOD - Masked credentials
log.info("Authentication successful", user=username)
```
## Network Security
### HTTPS Only
```python
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = ["httpx>=0.27.0"]
# ///
import httpx
import sys
def fetch_api(url: str) -> dict:
"""Fetch data from API (HTTPS only)"""
# Validate HTTPS
if not url.startswith("https://"):
print("Error: Only HTTPS URLs allowed", file=sys.stderr)
sys.exit(1)
with httpx.Client(verify=True) as client: # Verify SSL certs
response = client.get(url, timeout=10.0)
response.raise_for_status()
return response.json()
```
### Certificate Verification
```python
import httpx
import certifi
# Use trusted CA bundle
client = httpx.Client(verify=certifi.where())
# For internal CAs, specify cert path
client = httpx.Client(verify="/path/to/internal-ca.pem")
# ONLY disable for development (never in production)
if os.getenv("DEVELOPMENT") == "true":
client = httpx.Client(verify=False)
```
## Security Checklist
Before deploying a script:
- [ ] No hardcoded secrets
- [ ] Secrets from environment/keyring/Infisical
- [ ] All inputs validated
- [ ] No shell=True in subprocess
- [ ] File paths checked for traversal
- [ ] Dependencies pinned
- [ ] No sensitive data in logs
- [ ] HTTPS for network requests
- [ ] Certificate verification enabled
- [ ] Timeouts on external calls
- [ ] Error messages don't leak information
- [ ] Principle of least privilege applied
## References
- [OWASP Top 10](https://owasp.org/www-project-top-ten/)
- [Infisical Python SDK](https://infisical.com/docs/sdks/languages/python)
- [Python Security Best Practices](https://python.readthedocs.io/en/latest/library/security_warnings.html)
- Virgo-Core Infisical integration: [../../ansible/tasks/infisical-secret-lookup.yml](../../ansible/tasks/infisical-secret-lookup.yml)

View File

@@ -0,0 +1,495 @@
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "httpx>=0.27.0",
# ]
# ///
"""
Convert Python scripts to uv single-file format (PEP 723)
Purpose: script-conversion-automation
Team: devops
Author: devops@spaceships.work
Converts existing Python scripts to use inline dependency management.
Reads dependencies from requirements.txt or detects from imports.
Usage:
convert_to_uv.py <script.py>
convert_to_uv.py <script.py> --requirements requirements.txt
convert_to_uv.py <script.py> --detect-imports
convert_to_uv.py <script.py> --dry-run
Examples:
# Convert script with requirements.txt in same directory
convert_to_uv.py my_script.py
# Convert script with specific requirements file
convert_to_uv.py my_script.py --requirements ../requirements.txt
# Detect dependencies from imports (basic detection)
convert_to_uv.py my_script.py --detect-imports
# Preview conversion without creating file
convert_to_uv.py my_script.py --dry-run
# Specify output filename
convert_to_uv.py my_script.py --output my_script_new.py
"""
import ast
import re
import sys
import tomllib
from pathlib import Path
# Common import name -> PyPI package name mappings
IMPORT_TO_PACKAGE = {
'cv2': 'opencv-python',
'PIL': 'Pillow',
'yaml': 'PyYAML',
'bs4': 'beautifulsoup4',
'sklearn': 'scikit-learn',
'dotenv': 'python-dotenv',
'claude_agent_sdk': 'claude-agent-sdk',
}
def get_pypi_latest_version(package_name: str) -> str | None:
"""
Query PyPI API for latest version of package.
Returns version string like "1.2.3" or None if not found.
"""
try:
import httpx
url = f"https://pypi.org/pypi/{package_name}/json"
with httpx.Client(timeout=5.0) as client:
response = client.get(url)
if response.status_code == 200:
data = response.json()
return data['info']['version']
except Exception:
# Network error, package not found, etc. - fail silently
pass
return None
def find_version_in_project(package_name: str, script_path: Path) -> str | None:
"""
Look for version constraint in project's pyproject.toml.
Searches up directory tree from script location.
"""
current = script_path.parent
# Search up to 3 levels for pyproject.toml
for _ in range(3):
pyproject_path = current / "pyproject.toml"
if pyproject_path.exists():
try:
content = pyproject_path.read_text(encoding='utf-8')
data = tomllib.loads(content)
# Check [project.dependencies]
deps = data.get('project', {}).get('dependencies', [])
for dep in deps:
if isinstance(dep, str) and dep.startswith(package_name):
# Extract version constraint
# e.g., "package>=1.0.0" -> ">=1.0.0"
version_part = dep[len(package_name):].strip()
if version_part:
return version_part
# Check [tool.uv.sources] or other sections if needed
# (could expand this to check dev-dependencies, etc.)
except Exception:
pass
# Move up one directory
parent = current.parent
if parent == current: # Reached filesystem root
break
current = parent
return None
def normalize_package_name(import_name: str, use_pypi: bool = True) -> str:
"""
Normalize import name to PyPI package name.
Strategy:
1. Check known mappings first (fast)
2. Try hyphen normalization (common pattern: my_package → my-package)
3. Validate with PyPI if enabled
4. Fall back to original
Returns: Normalized package name
"""
# Known mapping takes precedence
if import_name in IMPORT_TO_PACKAGE:
return IMPORT_TO_PACKAGE[import_name]
# Try hyphen normalization (most common pattern)
if '_' in import_name:
hyphenated = import_name.replace('_', '-')
# Validate with PyPI if enabled
if use_pypi and get_pypi_latest_version(hyphenated):
return hyphenated
# Without PyPI, trust the normalization (common convention)
if not use_pypi:
return hyphenated
# Fall back to original
return import_name
def resolve_package_version(
import_name: str,
script_path: Path,
use_pypi: bool = True
) -> str:
"""
Resolve package name and version constraint.
Returns: "package>=X.Y.Z" format
Strategy:
1. Normalize import name to package name (handles underscore → hyphen)
2. Check project's pyproject.toml for version
3. Query PyPI for latest version
4. Fall back to unversioned package name
"""
# Normalize import name to package name
package_name = normalize_package_name(import_name, use_pypi)
# Try to find version in project
version = find_version_in_project(package_name, script_path)
if version:
return f"{package_name}{version}"
# Try PyPI if enabled
if use_pypi:
latest = get_pypi_latest_version(package_name)
if latest:
# Use minimum version constraint with latest
return f"{package_name}>={latest}"
# Fall back to unversioned (let uv resolve)
return package_name
def has_pep723_metadata(content: str) -> bool:
"""Check if script already has PEP 723 metadata"""
pattern = r'# /// script\r?\n((?:#.*(?:\r?\n|(?=\r?\n?#\s*///)))+)(?:\r?\n)?#\s*///'
return bool(re.search(pattern, content, re.MULTILINE))
def read_requirements_file(req_path: Path) -> list[str]:
"""Read dependencies from requirements.txt"""
if not req_path.exists():
return []
try:
content = req_path.read_text(encoding='utf-8')
deps = []
for line in content.splitlines():
line = line.strip()
# Skip empty lines and comments
if not line or line.startswith('#'):
continue
# Skip -e and --editable
if line.startswith('-e') or line.startswith('--editable'):
continue
# Skip -r and --requirement
if line.startswith('-r') or line.startswith('--requirement'):
continue
deps.append(line)
return deps
except (UnicodeDecodeError, OSError) as e:
print(f"Warning: Could not read {req_path}: {e}", file=sys.stderr)
return []
def detect_imports(content: str) -> list[str]:
"""
Detect third-party imports from script (basic detection).
Returns import names, not package names (caller should map these).
"""
imports = set()
try:
tree = ast.parse(content)
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
# Get base module name
base = alias.name.split('.')[0]
imports.add(base)
elif isinstance(node, ast.ImportFrom):
if node.module:
base = node.module.split('.')[0]
imports.add(base)
except SyntaxError as e:
print(f"Warning: Could not parse script for imports: {e}", file=sys.stderr)
return []
# Filter out standard library modules (basic filter)
stdlib_modules = {
'abc', 'argparse', 'ast', 'asyncio', 'base64', 'collections', 'contextlib',
'copy', 'csv', 'dataclasses', 'datetime', 'decimal', 'email', 'enum', 'functools',
'glob', 'hashlib', 'http', 'inspect', 'io', 'itertools', 'json', 'logging',
'math', 'multiprocessing', 'operator', 'os', 'pathlib', 'pickle', 'platform',
'pprint', 'queue', 're', 'secrets', 'shutil', 'socket', 'sqlite3', 'ssl',
'string', 'subprocess', 'sys', 'tempfile', 'threading', 'time', 'tomllib',
'traceback', 'typing', 'unittest', 'urllib', 'uuid', 'warnings', 'weakref',
'xml', 'zipfile', 'zoneinfo'
}
third_party = [imp for imp in imports if imp not in stdlib_modules]
return sorted(third_party)
def generate_header(
dependencies: list[str],
python_version: str = ">=3.11",
quiet: bool = False
) -> str:
"""Generate PEP 723 header with shebang"""
shebang = "#!/usr/bin/env -S uv run --script"
if quiet:
shebang += " --quiet"
# Format dependencies for TOML array
if dependencies:
deps_str = ",\n# ".join(f'"{dep}"' for dep in dependencies)
deps_section = f"# dependencies = [\n# {deps_str},\n# ]"
else:
deps_section = "# dependencies = []"
header = f"""{shebang}
# /// script
# requires-python = "{python_version}"
{deps_section}
# ///
"""
return header
def convert_script(
script_path: Path,
output_path: Path | None = None,
requirements_path: Path | None = None,
detect_imports_flag: bool = False,
dry_run: bool = False,
python_version: str = ">=3.11",
quiet_mode: bool = False,
use_pypi: bool = True
) -> bool:
"""
Convert script to uv format.
Returns True if successful, False otherwise.
"""
# Read original script
try:
content = script_path.read_text(encoding='utf-8')
except (FileNotFoundError, PermissionError, OSError, UnicodeDecodeError) as e:
print(f"Error: Cannot read {script_path}: {e}", file=sys.stderr)
return False
# Check if already has metadata
if has_pep723_metadata(content):
print(f"Error: {script_path} already has PEP 723 metadata", file=sys.stderr)
print(" Use validate_script.py to check the existing metadata", file=sys.stderr)
return False
# Determine dependencies
dependencies = []
if requirements_path:
# Use specified requirements file
dependencies = read_requirements_file(requirements_path)
if dependencies:
print(f"Found {len(dependencies)} dependencies in {requirements_path}")
else:
# Look for requirements.txt in same directory
default_req = script_path.parent / "requirements.txt"
if default_req.exists():
dependencies = read_requirements_file(default_req)
if dependencies:
print(f"Found {len(dependencies)} dependencies in {default_req}")
# Optionally detect imports
if detect_imports_flag:
detected = detect_imports(content)
if detected:
print(f"Detected imports: {', '.join(detected)}")
print("Resolving versions...")
# If no dependencies from requirements, use detected
if not dependencies:
# Use smart version resolution
resolved = []
for imp in detected:
# Normalize package name first
normalized_pkg = normalize_package_name(imp, use_pypi)
# Then resolve version
dep = resolve_package_version(imp, script_path, use_pypi=use_pypi)
resolved.append(dep)
# Show what was resolved
if imp in IMPORT_TO_PACKAGE:
print(f" - Mapped '{imp}''{IMPORT_TO_PACKAGE[imp]}' (known mapping)")
elif imp != normalized_pkg:
print(f" - Normalized '{imp}''{normalized_pkg}' (auto-detected)")
if '>=' in dep:
version = dep.split('>=')[1]
source = "from project" if find_version_in_project(dep.split('>=')[0], script_path) else "from PyPI"
print(f" - Resolved version: {version} {source}")
else:
print(f" - Using package: {dep} (uv will resolve version)")
dependencies = resolved
# Generate header
header = generate_header(dependencies, python_version, quiet_mode)
# Remove old shebang if present
lines = content.split('\n')
if lines and lines[0].startswith('#!'):
# Skip old shebang
content_without_shebang = '\n'.join(lines[1:])
else:
content_without_shebang = content
# Combine header and content
new_content = header + content_without_shebang
# Determine output path
if output_path is None:
# Default: add _uv before extension
stem = script_path.stem
suffix = script_path.suffix
output_path = script_path.parent / f"{stem}_uv{suffix}"
# Dry run - just print
if dry_run:
print("\n" + "=" * 60)
print("DRY RUN - Preview of converted script:")
print("=" * 60)
print(new_content[:500]) # Show first 500 chars
if len(new_content) > 500:
print(f"\n... ({len(new_content) - 500} more characters)")
print("=" * 60)
print(f"Would create: {output_path}")
return True
# Write output
try:
output_path.write_text(new_content, encoding='utf-8')
print(f"✓ Created: {output_path}")
# Make executable
import stat
current_permissions = output_path.stat().st_mode
output_path.chmod(current_permissions | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
print("✓ Made executable")
# Print next steps
print("\nNext steps:")
print(f" 1. Review dependencies in {output_path}")
print(" 2. Add version constraints if needed")
print(f" 3. Test: {output_path}")
print(f" 4. Validate: validate_script.py {output_path}")
return True
except (PermissionError, OSError) as e:
print(f"Error: Cannot write {output_path}: {e}", file=sys.stderr)
return False
def main():
"""Main entry point"""
import argparse
parser = argparse.ArgumentParser(
description="Convert Python scripts to uv single-file format",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__
)
parser.add_argument('script', help='Python script to convert')
parser.add_argument(
'--requirements', '-r',
help='Path to requirements.txt (default: look in same directory)'
)
parser.add_argument(
'--detect-imports', '-d',
action='store_true',
help='Detect dependencies from imports (basic detection)'
)
parser.add_argument(
'--output', '-o',
help='Output filename (default: <script>_uv.py)'
)
parser.add_argument(
'--python-version', '-p',
default='>=3.11',
help='Python version constraint (default: >=3.11)'
)
parser.add_argument(
'--quiet', '-q',
action='store_true',
help='Add --quiet flag to shebang'
)
parser.add_argument(
'--dry-run', '-n',
action='store_true',
help='Preview conversion without creating file'
)
parser.add_argument(
'--no-pypi',
action='store_true',
help='Skip PyPI queries for version resolution (faster, offline)'
)
args = parser.parse_args()
script_path = Path(args.script)
if not script_path.exists():
print(f"Error: File not found: {script_path}", file=sys.stderr)
sys.exit(1)
# Parse requirements path if provided
req_path = Path(args.requirements) if args.requirements else None
# Parse output path if provided
out_path = Path(args.output) if args.output else None
# Convert
success = convert_script(
script_path=script_path,
output_path=out_path,
requirements_path=req_path,
detect_imports_flag=args.detect_imports,
dry_run=args.dry_run,
python_version=args.python_version,
quiet_mode=args.quiet,
use_pypi=not args.no_pypi
)
sys.exit(0 if success else 1)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,356 @@
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""
Validate PEP 723 inline script metadata
Checks Python scripts for:
- Valid PEP 723 metadata block
- Required fields (requires-python, dependencies)
- TOML syntax validity
- Shebang presence and format
- Security issues
Usage:
validate_script.py <script.py>
validate_script.py --strict <script.py>
validate_script.py --force <script>
Examples:
# Basic validation
validate_script.py my_script.py
# Strict mode (all best practices)
validate_script.py --strict my_script.py
# Validate executable Python script without .py extension
validate_script.py my_script
# Force validation, skip extension check
validate_script.py --force my_script
# Validate all scripts in directory
find . -name '*.py' -exec python validate_script.py {} \\;
"""
import ast
import os
import re
import sys
import tomllib
from dataclasses import dataclass
from pathlib import Path
@dataclass
class ValidationResult:
"""Validation result"""
valid: bool
has_metadata: bool
has_shebang: bool
has_docstring: bool
warnings: list[str]
errors: list[str]
def extract_metadata_block(content: str) -> str | None:
"""Extract PEP 723 metadata block"""
# Match metadata block with CRLF tolerance and flexible whitespace
# Uses lookahead to allow last metadata line without trailing newline
pattern = r'# /// script\r?\n((?:#.*(?:\r?\n|(?=\r?\n?#\s*///)))+)(?:\r?\n)?#\s*///'
match = re.search(pattern, content, re.MULTILINE)
if not match:
return None
# Extract TOML content (remove leading # and optional whitespace from each line)
lines = match.group(1).splitlines()
toml_lines = []
for line in lines:
if line.startswith('#'):
# Strip '#' followed by optional space or tab
stripped = re.sub(r'^#[ \t]?', '', line)
toml_lines.append(stripped)
else:
# Preserve non-comment lines (shouldn't occur with our regex but be safe)
toml_lines.append(line)
return '\n'.join(toml_lines)
def validate_toml_syntax(toml_content: str) -> list[str]:
"""Validate TOML syntax using structured parsing"""
errors = []
# Parse TOML content
try:
data = tomllib.loads(toml_content)
except tomllib.TOMLDecodeError as e:
errors.append(f"Invalid TOML syntax: {e}")
return errors
# Validate required fields
if 'requires-python' not in data:
errors.append("Missing 'requires-python' field")
elif not isinstance(data['requires-python'], str):
errors.append("'requires-python' must be a string")
if 'dependencies' not in data:
errors.append("Missing 'dependencies' field")
else:
dependencies = data['dependencies']
# Dependencies should be a list/array
if not isinstance(dependencies, list):
errors.append("'dependencies' must be an array/list")
else:
# Validate each dependency item
for idx, dep in enumerate(dependencies):
if not isinstance(dep, str):
errors.append(
f"Dependency at index {idx} must be a string, got {type(dep).__name__}")
return errors
def check_shebang(content: str) -> tuple[bool, list[str]]:
"""Check shebang line"""
warnings = []
lines = content.split('\n')
if not lines:
return False, ["Empty file"]
first_line = lines[0]
if not first_line.startswith('#!'):
return False, []
# Check for recommended shebangs
recommended = [
'#!/usr/bin/env -S uv run --script',
'#!/usr/bin/env -S uv run --script --quiet',
]
if first_line not in recommended:
warnings.append(f"Shebang not recommended. Use: {recommended[0]}")
return True, warnings
def check_security_issues(content: str) -> list[str]:
"""Check for common security issues"""
warnings = []
# Check for hardcoded secrets
secret_patterns = [
(r'password\s*=\s*["\']', "Possible hardcoded password"),
(r'api[_-]?key\s*=\s*["\']', "Possible hardcoded API key"),
(r'secret\s*=\s*["\']', "Possible hardcoded secret"),
(r'token\s*=\s*["\']', "Possible hardcoded token"),
]
for pattern, message in secret_patterns:
if re.search(pattern, content, re.IGNORECASE):
warnings.append(f"Security: {message}")
# Check for shell=True
if re.search(r'shell\s*=\s*True', content):
warnings.append(
"Security: subprocess.run with shell=True (command injection risk)")
# Check for eval/exec
if re.search(r'\b(eval|exec)\s*\(', content):
warnings.append(
"Security: Use of eval() or exec() (code injection risk)")
return warnings
def is_valid_python_file(script_path: Path) -> tuple[bool, str]:
"""
Check if a non-.py file is a valid Python script.
Returns:
Tuple of (is_valid, reason) where reason describes why it's valid or invalid
"""
try:
content = script_path.read_text(encoding='utf-8')
except (FileNotFoundError, PermissionError, OSError, UnicodeDecodeError) as e:
return False, f"Cannot read file: {e}"
# Check if file is executable with Python shebang
is_executable = os.access(script_path, os.X_OK)
lines = content.split('\n')
has_python_shebang = False
if lines and lines[0].startswith('#!'):
shebang = lines[0].lower()
has_python_shebang = 'python' in shebang
if is_executable and has_python_shebang:
return True, "executable with Python shebang"
# Try to parse as Python to confirm it's valid Python code
try:
ast.parse(content)
return True, "valid Python syntax"
except SyntaxError as e:
return False, f"not valid Python: {e}"
def validate_script(script_path: Path, strict: bool = False) -> ValidationResult:
"""Validate Python script"""
result = ValidationResult(
valid=True,
has_metadata=False,
has_shebang=False,
has_docstring=False,
warnings=[],
errors=[]
)
# Read file
try:
content = script_path.read_text(encoding='utf-8')
except (FileNotFoundError, PermissionError, OSError, UnicodeDecodeError) as e:
result.valid = False
result.errors.append(f"Failed to read file: {e}")
return result
# Check shebang
has_shebang, shebang_warnings = check_shebang(content)
result.has_shebang = has_shebang
result.warnings.extend(shebang_warnings)
if strict and not has_shebang:
result.errors.append("Missing shebang (required in strict mode)")
result.valid = False
# Check for metadata block
metadata = extract_metadata_block(content)
result.has_metadata = metadata is not None
if not metadata:
result.errors.append("No PEP 723 metadata block found")
result.valid = False
return result
# Validate TOML syntax
toml_errors = validate_toml_syntax(metadata)
result.errors.extend(toml_errors)
if toml_errors:
result.valid = False
# Check for module docstring using AST parsing
try:
module_node = ast.parse(content)
module_docstring = ast.get_docstring(module_node)
result.has_docstring = module_docstring is not None
except SyntaxError as e:
result.has_docstring = False
result.warnings.append(
f"Could not parse file for docstring check: {e}")
if strict and not result.has_docstring:
result.warnings.append(
"Missing module docstring (recommended in strict mode)")
# Security checks (always warnings, never errors - these are heuristic checks)
security_warnings = check_security_issues(content)
result.warnings.extend(security_warnings)
return result
def main():
"""Main entry point"""
import argparse
parser = argparse.ArgumentParser(
description="Validate PEP 723 script metadata",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__
)
parser.add_argument('script', help='Python script to validate')
parser.add_argument('--strict', action='store_true',
help='Enable strict validation')
parser.add_argument('--force', action='store_true',
help='Skip Python file extension check')
args = parser.parse_args()
script_path = Path(args.script)
if not script_path.exists():
print(f"Error: File not found: {script_path}", file=sys.stderr)
sys.exit(1)
# Check if file is a Python file
if script_path.suffix != '.py':
if args.force:
print(
f"Warning: File lacks .py extension, but --force was specified", file=sys.stderr)
else:
# Check if it's a valid Python file by other means
is_valid, reason = is_valid_python_file(script_path)
if not is_valid:
print(
f"Error: Not a Python file: {script_path}", file=sys.stderr)
print(f" Reason: {reason}", file=sys.stderr)
print(f" Hint: File must either:", file=sys.stderr)
print(f" - Have a .py extension, OR", file=sys.stderr)
print(f" - Be executable with a Python shebang, OR",
file=sys.stderr)
print(f" - Contain valid Python syntax", file=sys.stderr)
print(f" Use --force to skip this check", file=sys.stderr)
sys.exit(1)
else:
print(
f"Info: File accepted as Python ({reason})", file=sys.stderr)
# Validate
result = validate_script(script_path, strict=args.strict)
# Print results
print(f"Validating: {script_path}")
print("=" * 60)
if result.has_shebang:
print("✓ Has shebang")
else:
print("✗ Missing shebang")
if result.has_metadata:
print("✓ Has PEP 723 metadata")
else:
print("✗ Missing PEP 723 metadata")
if result.has_docstring:
print("✓ Has docstring")
else:
print("○ No docstring")
if result.warnings:
print("\nWarnings:")
for warning in result.warnings:
print(f"{warning}")
if result.errors:
print("\nErrors:")
for error in result.errors:
print(f"{error}")
print("\n" + "=" * 60)
if result.valid:
print("Status: ✓ VALID")
sys.exit(0)
else:
print("Status: ✗ INVALID")
sys.exit(1)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,67 @@
# CI/CD Integration for UV Scripts
> **Status**: 🚧 Placeholder - Content in development
## Overview
Integrating UV single-file scripts into CI/CD pipelines with GitHub Actions, GitLab CI, and other platforms.
## Topics to Cover
- [ ] GitHub Actions workflows
- [ ] GitLab CI configuration
- [ ] Pre-commit hooks integration
- [ ] Automated testing
- [ ] Security scanning
- [ ] Deployment strategies
- [ ] Version management
## Quick Example
### GitHub Actions
```yaml
name: Test UV Scripts
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v3
with:
version: "latest"
- name: Run script tests
run: |
uv run scripts/check_health.py --validate
uv run scripts/analyze_data.py --dry-run
```
### Pre-commit Hook
```yaml
# .pre-commit-config.yaml
repos:
- repo: local
hooks:
- id: validate-uv-scripts
name: Validate UV Scripts
entry: uv run scripts/validate_all.py
language: system
pass_filenames: false
```
## TODO
This file will be expanded to include:
- Complete GitHub Actions examples
- GitLab CI patterns
- Pre-commit hook configurations
- Automated deployment workflows
- Security scanning integration

View File

@@ -0,0 +1,58 @@
# Team Adoption Guide
> **Status**: 🚧 Placeholder - Content in development
## Overview
Guide for rolling out UV single-file scripts across development teams, establishing standards, and ensuring adoption.
## Topics to Cover
- [ ] Introduction and training materials
- [ ] Team standards and conventions
- [ ] Code review guidelines
- [ ] Migration from existing scripts
- [ ] Tooling setup across platforms
- [ ] Documentation requirements
- [ ] Success metrics and tracking
## Adoption Roadmap
### Phase 1: Pilot (Week 1-2)
- Select 2-3 team members for initial trial
- Convert 3-5 existing scripts
- Gather feedback and iterate
### Phase 2: Expansion (Week 3-4)
- Team-wide training session
- Establish coding standards
- Set up CI/CD integration
- Create internal documentation
### Phase 3: Full Adoption (Week 5+)
- All new scripts use UV format
- Migration plan for legacy scripts
- Regular reviews and improvements
## Team Standards Checklist
- [ ] Required shebang format: `#!/usr/bin/env -S uv run`
- [ ] Minimum Python version: `>=3.11`
- [ ] Required metadata fields
- [ ] Naming conventions
- [ ] Documentation requirements
- [ ] Testing requirements
- [ ] Security review process
## TODO
This file will be expanded to include:
- Detailed training materials
- Complete adoption roadmap
- Team communication templates
- Migration guides
- Success stories and case studies

View File

@@ -0,0 +1,72 @@
# Testing Strategies for UV Scripts
> **Status**: 🚧 Placeholder - Content in development
## Overview
Strategies for testing UV single-file scripts, from inline tests to pytest integration.
## Topics to Cover
- [ ] Inline test functions
- [ ] pytest integration
- [ ] Mock and fixtures for testing
- [ ] Testing CLI applications
- [ ] API client testing with mocks
- [ ] Coverage reporting
- [ ] Test automation in CI/CD
## Quick Example
### Inline Testing
```python
#!/usr/bin/env -S uv run
# /// script
# requires-python = ">=3.11"
# ///
def add(a: int, b: int) -> int:
return a + b
def test_add():
assert add(2, 3) == 5
assert add(-1, 1) == 0
print("✓ All tests passed")
if __name__ == "__main__":
# Run tests
test_add()
# Or run main application
result = add(10, 20)
print(f"Result: {result}")
```
### Pytest Integration
```python
#!/usr/bin/env -S uv run
# /// script
# requires-python = ">=3.11"
# dependencies = ["pytest>=7.0.0"]
# ///
"""
Run with: pytest this_script.py
"""
def multiply(a: int, b: int) -> int:
return a * b
def test_multiply():
assert multiply(2, 3) == 6
assert multiply(-1, 5) == -5
```
## TODO
This file will be expanded to include:
- Complete testing patterns
- Mocking external dependencies
- Test organization strategies
- CI/CD integration examples
- Coverage tools and reporting