Initial commit
This commit is contained in:
545
skills/python-uv-scripts/SKILL.md
Normal file
545
skills/python-uv-scripts/SKILL.md
Normal file
@@ -0,0 +1,545 @@
|
||||
---
|
||||
name: python-uv-scripts
|
||||
description: >
|
||||
Python single-file script development using uv and PEP 723 inline metadata.
|
||||
Prevents invalid patterns like [tool.uv.metadata].
|
||||
Use when creating standalone Python utilities, converting scripts to uv format,
|
||||
managing script dependencies, implementing script testing,
|
||||
or establishing team standards for script development.
|
||||
---
|
||||
|
||||
# Python Single-File Scripts with uv
|
||||
|
||||
Expert guidance for creating production-ready, self-contained Python scripts using uv's inline dependency management
|
||||
(PEP 723).
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Create Your First uv Script
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# "rich>=13.0.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import httpx
|
||||
from rich import print
|
||||
|
||||
response = httpx.get("https://api.github.com")
|
||||
print(f"[green]Status: {response.status_code}[/green]")
|
||||
```
|
||||
|
||||
Make it executable:
|
||||
|
||||
```bash
|
||||
chmod +x script.py
|
||||
./script.py # uv automatically installs dependencies
|
||||
```
|
||||
|
||||
### Convert Existing Script
|
||||
|
||||
```bash
|
||||
# Add inline metadata to existing script
|
||||
./tools/convert_to_uv.py existing_script.py
|
||||
|
||||
# Validate PEP 723 metadata
|
||||
./tools/validate_script.py script.py
|
||||
```
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### What is PEP 723?
|
||||
|
||||
**PEP 723** defines inline script metadata for Python files:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "package>=1.0.0",
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
|
||||
- ✅ Dependencies live with the code
|
||||
- ✅ No separate `requirements.txt`
|
||||
- ✅ Reproducible execution
|
||||
- ✅ Version constraints included
|
||||
- ✅ Self-documenting
|
||||
|
||||
### uv Script Execution Modes
|
||||
|
||||
**Mode 1: Inline Dependencies** (Recommended for utilities)
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# dependencies = ["httpx"]
|
||||
# ///
|
||||
```
|
||||
|
||||
**Mode 2: Project Mode** (For larger scripts)
|
||||
|
||||
```bash
|
||||
uv run script.py # Uses pyproject.toml
|
||||
```
|
||||
|
||||
### Mode 3: No Dependencies
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run
|
||||
# Standard library only
|
||||
```
|
||||
|
||||
## Critical Anti-Patterns: What NOT to Do
|
||||
|
||||
### ❌ NEVER Use [tool.uv.metadata]
|
||||
|
||||
**WRONG** - This will cause errors:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# [tool.uv.metadata] # ❌ THIS DOES NOT WORK
|
||||
# purpose = "testing"
|
||||
# ///
|
||||
```
|
||||
|
||||
**Error**:
|
||||
|
||||
```text
|
||||
error: TOML parse error at line 3, column 7
|
||||
unknown field `metadata`
|
||||
```
|
||||
|
||||
**Why**: `[tool.uv.metadata]` is not part of PEP 723 and is not supported by uv.
|
||||
|
||||
**CORRECT** - Use Python docstrings for metadata:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""
|
||||
Purpose: Testing automation
|
||||
Team: DevOps
|
||||
Author: team@example.com
|
||||
"""
|
||||
```
|
||||
|
||||
**Valid `tool.uv` fields** (if needed):
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# [tool.uv]
|
||||
# exclude-newer = "2025-01-01T00:00:00Z" # For reproducibility
|
||||
# ///
|
||||
```
|
||||
|
||||
## Real-World Examples from This Repository
|
||||
|
||||
### Example 1: Cluster Health Checker
|
||||
|
||||
See [examples/03-production-ready/check_cluster_health_enhanced.py](examples/03-production-ready/check_cluster_health_enhanced.py)
|
||||
|
||||
**Current version** (basic):
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
import subprocess
|
||||
# Manual dependency installation required
|
||||
```
|
||||
|
||||
**Enhanced with uv** (production-ready):
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script --quiet
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "rich>=13.0.0",
|
||||
# "typer>=0.9.0",
|
||||
# ]
|
||||
# ///
|
||||
"""
|
||||
Purpose: Cluster health monitoring
|
||||
Team: Infrastructure
|
||||
"""
|
||||
|
||||
import typer
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
```
|
||||
|
||||
### Example 2: CEPH Health Monitor
|
||||
|
||||
See [examples/03-production-ready/ceph_health.py](examples/03-production-ready/ceph_health.py)
|
||||
|
||||
Pattern: JSON API interaction with structured output
|
||||
|
||||
## Best Practices from This Repository
|
||||
|
||||
### 1. Security Patterns
|
||||
|
||||
See [reference/security-patterns.md](reference/security-patterns.md) for complete security guide including:
|
||||
|
||||
- Secrets management (environment variables, keyring, Infisical)
|
||||
- Input validation
|
||||
- Dependency security
|
||||
- File operations security
|
||||
- Command execution security
|
||||
|
||||
### 2. Version Pinning Strategy
|
||||
|
||||
Following this repository's approach (from `pyproject.toml`):
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0", # Minimum version for compatibility
|
||||
# "rich>=13.0.0", # Known good version
|
||||
# "ansible>=11.1.0", # Match project requirements
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
**Pinning levels:**
|
||||
|
||||
- `>=X.Y.Z` - Minimum version (most flexible)
|
||||
- `~=X.Y.Z` - Compatible release (patch updates only)
|
||||
- `==X.Y.Z` - Exact version (most strict)
|
||||
|
||||
See [reference/dependency-management.md](reference/dependency-management.md).
|
||||
|
||||
### 3. Team Standards
|
||||
|
||||
**File naming:**
|
||||
|
||||
```bash
|
||||
check_cluster_health.py # ✅ Descriptive, snake_case
|
||||
validate_template.py # ✅ Action-oriented
|
||||
cluster.py # ❌ Too generic
|
||||
```
|
||||
|
||||
**Shebang pattern:**
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script --quiet
|
||||
# --quiet suppresses uv's own output
|
||||
```
|
||||
|
||||
**Documentation template:**
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""
|
||||
Check Proxmox cluster health
|
||||
|
||||
Purpose: cluster-monitoring
|
||||
Team: infrastructure
|
||||
Author: devops@spaceships.work
|
||||
|
||||
Usage:
|
||||
python check_cluster_health.py [--node NODE] [--json]
|
||||
|
||||
Examples:
|
||||
python check_cluster_health.py --node foxtrot
|
||||
python check_cluster_health.py --json
|
||||
"""
|
||||
```
|
||||
|
||||
### 4. Error Handling Patterns
|
||||
|
||||
Following Ansible best practices from this repository:
|
||||
|
||||
```python
|
||||
import sys
|
||||
import subprocess
|
||||
|
||||
def run_command(cmd: str) -> str:
|
||||
"""Execute command with proper error handling"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
cmd.split(),
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True
|
||||
)
|
||||
return result.stdout
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"Error: Command failed: {cmd}", file=sys.stderr)
|
||||
print(f" {e.stderr}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
except FileNotFoundError:
|
||||
print(f"Error: Command not found: {cmd.split()[0]}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
See [patterns/error-handling.md](patterns/error-handling.md).
|
||||
|
||||
### 5. Testing Patterns
|
||||
|
||||
**Inline testing** (for simple scripts):
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
|
||||
def validate_ip(ip: str) -> bool:
|
||||
"""Validate IP address format"""
|
||||
import re
|
||||
pattern = r'^(\d{1,3}\.){3}\d{1,3}$'
|
||||
return bool(re.match(pattern, ip))
|
||||
|
||||
# Inline tests
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
|
||||
# Run tests if --test flag provided
|
||||
if len(sys.argv) > 1 and sys.argv[1] == "--test":
|
||||
assert validate_ip("192.168.1.1") == True
|
||||
assert validate_ip("256.1.1.1") == False
|
||||
print("All tests passed!")
|
||||
sys.exit(0)
|
||||
|
||||
# Normal execution
|
||||
print(validate_ip("192.168.3.5"))
|
||||
```
|
||||
|
||||
See [workflows/testing-strategies.md](workflows/testing-strategies.md).
|
||||
|
||||
## When NOT to Use Single-File Scripts
|
||||
|
||||
See [anti-patterns/when-not-to-use.md](anti-patterns/when-not-to-use.md) for details.
|
||||
|
||||
**Use a proper project instead when:**
|
||||
|
||||
- ❌ Script exceeds 500 lines
|
||||
- ❌ Multiple modules/files needed
|
||||
- ❌ Complex configuration management
|
||||
- ❌ Requires packaging/distribution
|
||||
- ❌ Shared library code across multiple scripts
|
||||
- ❌ Web applications or long-running services
|
||||
|
||||
**Example - Too Complex for Single File:**
|
||||
|
||||
```python
|
||||
# This should be a uv project, not a script:
|
||||
# - 15+ dependencies
|
||||
# - Database models
|
||||
# - API routes
|
||||
# - Background workers
|
||||
# - Configuration management
|
||||
# - Multiple environments
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
See pattern guides for complete examples:
|
||||
|
||||
- [CLI Applications](patterns/cli-applications.md) - Typer, Click, argparse patterns
|
||||
- [API Clients](patterns/api-clients.md) - httpx, requests, authentication
|
||||
- [Data Processing](patterns/data-processing.md) - Polars, pandas, analysis
|
||||
- [System Automation](patterns/system-automation.md) - psutil, subprocess, system admin
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
```yaml
|
||||
name: Run Health Checks
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 */6 * * *' # Every 6 hours
|
||||
|
||||
jobs:
|
||||
health-check:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v3
|
||||
|
||||
- name: Check cluster health
|
||||
run: |
|
||||
uv run --script tools/check_cluster_health.py --json
|
||||
env:
|
||||
PROXMOX_TOKEN: ${{ secrets.PROXMOX_TOKEN }}
|
||||
```
|
||||
|
||||
### GitLab CI
|
||||
|
||||
```yaml
|
||||
cluster-health:
|
||||
image: ghcr.io/astral-sh/uv:python3.11-bookworm-slim
|
||||
script:
|
||||
- uv run --script tools/check_cluster_health.py
|
||||
only:
|
||||
- schedules
|
||||
```
|
||||
|
||||
See [workflows/ci-cd-integration.md](workflows/ci-cd-integration.md).
|
||||
|
||||
## Tools Available
|
||||
|
||||
### Script Validation
|
||||
|
||||
```bash
|
||||
# Validate PEP 723 metadata
|
||||
./tools/validate_script.py script.py
|
||||
|
||||
# Output:
|
||||
# ✓ Valid PEP 723 metadata
|
||||
# ✓ Python version specified
|
||||
# ✓ Dependencies properly formatted
|
||||
```
|
||||
|
||||
### Script Conversion
|
||||
|
||||
```bash
|
||||
# Convert requirements.txt-based script to uv
|
||||
./tools/convert_to_uv.py old_script.py
|
||||
|
||||
# Creates:
|
||||
# - old_script_uv.py with inline dependencies
|
||||
# - Preserves original script
|
||||
```
|
||||
|
||||
## Progressive Disclosure
|
||||
|
||||
For deeper knowledge:
|
||||
|
||||
### Reference Documentation
|
||||
|
||||
- [PEP 723 Specification](reference/pep-723-spec.md) - Complete inline metadata spec
|
||||
- [Dependency Management](reference/dependency-management.md) - Version pinning strategies
|
||||
- [Security Patterns](reference/security-patterns.md) - Secrets, validation, input sanitization
|
||||
|
||||
### Pattern Guides
|
||||
|
||||
- [CLI Applications](patterns/cli-applications.md) - Typer, Click, argparse patterns
|
||||
- [API Clients](patterns/api-clients.md) - httpx, requests, authentication
|
||||
- [Data Processing](patterns/data-processing.md) - Polars, pandas, analysis
|
||||
- [System Automation](patterns/system-automation.md) - psutil, subprocess, system admin
|
||||
- [Error Handling](patterns/error-handling.md) - Exception handling, logging
|
||||
|
||||
> **Note:** See [Common Patterns](#common-patterns) section above for quick access to these guides.
|
||||
|
||||
### Working Examples
|
||||
|
||||
- [NetBox API Client](examples/04-api-clients/netbox_client.py) - Production-ready API client with Infisical, validation, error handling, and Rich output
|
||||
- [Cluster Health Checker](examples/03-production-ready/check_cluster_health_enhanced.py) - Production-ready monitoring script with Typer, Rich, and JSON output
|
||||
|
||||
### Anti-Patterns
|
||||
|
||||
- [When NOT to Use](anti-patterns/when-not-to-use.md) - Signs you need a proper project
|
||||
- [Common Mistakes](anti-patterns/common-mistakes.md) - Pitfalls and how to avoid them
|
||||
|
||||
### Workflows
|
||||
|
||||
- [Team Adoption](workflows/team-adoption.md) - Rolling out uv scripts across teams
|
||||
- [CI/CD Integration](workflows/ci-cd-integration.md) - GitHub Actions, GitLab CI
|
||||
- [Testing Strategies](workflows/testing-strategies.md) - Inline tests, pytest integration
|
||||
|
||||
## Related Skills
|
||||
|
||||
- **Ansible Best Practices** - Many Ansible modules could be standalone uv scripts
|
||||
- **Proxmox Infrastructure** - Validation tools use this pattern
|
||||
- **NetBox + PowerDNS Integration** - API interaction scripts
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Shebang Options
|
||||
|
||||
```python
|
||||
# Standard script execution
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# Quiet mode (suppress uv output)
|
||||
#!/usr/bin/env -S uv run --script --quiet
|
||||
|
||||
# With Python version
|
||||
#!/usr/bin/env -S uv run --script --python 3.11
|
||||
```
|
||||
|
||||
### Common Dependencies
|
||||
|
||||
```python
|
||||
# CLI applications
|
||||
"typer>=0.9.0" # Modern CLI framework
|
||||
"click>=8.0.0" # Alternative CLI framework
|
||||
"rich>=13.0.0" # Rich text and formatting
|
||||
|
||||
# API clients
|
||||
"httpx>=0.27.0" # Modern async HTTP client
|
||||
"requests>=2.31.0" # Traditional HTTP client
|
||||
|
||||
# Data processing
|
||||
"polars>=0.20.0" # Fast dataframe library
|
||||
"pandas>=2.0.0" # Traditional dataframe library
|
||||
|
||||
# Infrastructure
|
||||
"ansible>=11.1.0" # Automation (from this repo)
|
||||
"infisical-python>=2.3.3" # Secrets (from this repo)
|
||||
|
||||
# System automation
|
||||
"psutil>=5.9.0" # System monitoring
|
||||
```
|
||||
|
||||
### Metadata Template
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script --quiet
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# # Add dependencies here
|
||||
# ]
|
||||
# ///
|
||||
"""
|
||||
One-line description
|
||||
|
||||
Purpose: describe-purpose
|
||||
Team: team-name
|
||||
Author: email@example.com
|
||||
|
||||
Usage:
|
||||
python script.py [OPTIONS]
|
||||
|
||||
Examples:
|
||||
python script.py --help
|
||||
"""
|
||||
```
|
||||
|
||||
## Best Practices Summary
|
||||
|
||||
1. **Always specify Python version** - `requires-python = ">=3.11"`
|
||||
2. **Pin dependencies appropriately** - Use `>=X.Y.Z` for utilities
|
||||
3. **Add metadata in docstrings** - Put team info, purpose, and author in module docstring
|
||||
4. **Include comprehensive docstrings** - Document purpose, usage, and examples
|
||||
5. **Handle errors gracefully** - Use try/except with clear messages
|
||||
6. **Validate inputs** - Check arguments before processing
|
||||
7. **Use quiet mode** - `--quiet` flag for production scripts
|
||||
8. **Keep it focused** - Single file, single purpose
|
||||
9. **Test inline** - Add `--test` flag for simple validation
|
||||
10. **Secure secrets** - Never hardcode, use env vars or keyring
|
||||
373
skills/python-uv-scripts/anti-patterns/common-mistakes.md
Normal file
373
skills/python-uv-scripts/anti-patterns/common-mistakes.md
Normal file
@@ -0,0 +1,373 @@
|
||||
# Anti-Patterns: Common Mistakes with uv Scripts
|
||||
|
||||
This document lists common mistakes when creating uv scripts and how to avoid them.
|
||||
|
||||
## Critical Mistakes
|
||||
|
||||
### ❌ Using [tool.uv.metadata]
|
||||
|
||||
**WRONG**:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# [tool.uv.metadata] # ❌ THIS DOES NOT WORK
|
||||
# purpose = "testing"
|
||||
# team = "devops"
|
||||
# ///
|
||||
```
|
||||
|
||||
**Error**:
|
||||
|
||||
```text
|
||||
error: TOML parse error at line 4, column 7
|
||||
|
|
||||
4 | [tool.uv.metadata]
|
||||
| ^^
|
||||
unknown field `metadata`
|
||||
```
|
||||
|
||||
**Why**: `[tool.uv.metadata]` is not part of PEP 723 and is not supported by uv.
|
||||
|
||||
**CORRECT**: Use docstrings for metadata:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""
|
||||
Purpose: Testing automation
|
||||
Team: DevOps
|
||||
Author: team@example.com
|
||||
"""
|
||||
```
|
||||
|
||||
**NOTE**: `[tool.uv]` (without `.metadata`) IS valid for fields like `exclude-newer`. See reference/pep-723-spec.md.
|
||||
|
||||
### ❌ Adding Custom TOML Fields
|
||||
|
||||
**WRONG**:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# author = "me" # ❌ INVALID
|
||||
# version = "1.0.0" # ❌ INVALID
|
||||
# description = "test" # ❌ INVALID
|
||||
# dependencies = []
|
||||
# ///
|
||||
```
|
||||
|
||||
**CORRECT**: Only use `requires-python` and `dependencies` at the top level:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""
|
||||
Author: me
|
||||
Version: 1.0.0
|
||||
Description: test
|
||||
"""
|
||||
```
|
||||
|
||||
### ❌ Hardcoded Secrets
|
||||
|
||||
**WRONG**:
|
||||
|
||||
```python
|
||||
API_KEY = "sk-1234567890abcdef" # ❌ NEVER DO THIS
|
||||
PASSWORD = "super_secret_password" # ❌ NEVER DO THIS
|
||||
```
|
||||
|
||||
**CORRECT**: Use environment variables:
|
||||
|
||||
```python
|
||||
import os
|
||||
import sys
|
||||
|
||||
API_KEY = os.getenv("API_KEY")
|
||||
if not API_KEY:
|
||||
print("Error: API_KEY environment variable not set", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
**BETTER**: Use Infisical (following repo pattern):
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "infisical-python>=2.3.3",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
from infisical import InfisicalClient
|
||||
|
||||
client = InfisicalClient()
|
||||
api_key = client.get_secret("API_KEY", path="/production")
|
||||
```
|
||||
|
||||
### ❌ Missing Error Handling
|
||||
|
||||
**WRONG**:
|
||||
|
||||
```python
|
||||
import httpx
|
||||
|
||||
response = httpx.get("https://api.example.com") # ❌ No error handling
|
||||
data = response.json()
|
||||
```
|
||||
|
||||
**CORRECT**:
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import sys
|
||||
|
||||
try:
|
||||
response = httpx.get("https://api.example.com", timeout=10.0)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
except httpx.HTTPStatusError as e:
|
||||
print(f"HTTP error: {e.response.status_code}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
except httpx.RequestError as e:
|
||||
print(f"Request failed: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
## When NOT to Use Single-File Scripts
|
||||
|
||||
### ❌ Complex Applications
|
||||
|
||||
Don't use single-file scripts when:
|
||||
|
||||
- **Script exceeds 500 lines** → Use a proper uv project
|
||||
- **Multiple modules needed** → Use a proper uv project
|
||||
- **Shared code across scripts** → Use a proper uv project with shared library
|
||||
- **Web applications** → Use a proper uv project (Flask/FastAPI/Django)
|
||||
- **Long-running services** → Use a proper uv project
|
||||
- **Complex configuration** → Use a proper uv project with config files
|
||||
|
||||
**Example - Too Complex**:
|
||||
|
||||
```python
|
||||
# This should be a uv project, not a script:
|
||||
# - 15+ dependencies
|
||||
# - Database models
|
||||
# - API routes
|
||||
# - Background workers
|
||||
# - Multiple configuration files
|
||||
# - 1000+ lines of code
|
||||
```
|
||||
|
||||
### ❌ Heavy Dependencies
|
||||
|
||||
**WRONG**: Using heavy ML/data libraries in scripts:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "tensorflow>=2.15.0", # ❌ Too heavy for script
|
||||
# "torch>=2.1.0", # ❌ Too heavy for script
|
||||
# "transformers>=4.35.0", # ❌ Too heavy for script
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
**Why**: These create very large environments. Use a proper project instead.
|
||||
|
||||
**OK**: Lightweight data processing:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "polars>=0.20.0", # ✓ Reasonable for scripts
|
||||
# "httpx>=0.27.0", # ✓ Reasonable for scripts
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
## Common Syntax Mistakes
|
||||
|
||||
### ❌ Missing # on TOML Lines
|
||||
|
||||
**WRONG**:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
requires-python = ">=3.11" # ❌ Missing # at start
|
||||
dependencies = [] # ❌ Missing # at start
|
||||
# ///
|
||||
```
|
||||
|
||||
**CORRECT**:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11" # ✓ Each line starts with #
|
||||
# dependencies = [] # ✓ Each line starts with #
|
||||
# ///
|
||||
```
|
||||
|
||||
### ❌ Wrong Marker Format
|
||||
|
||||
**WRONG**:
|
||||
|
||||
```python
|
||||
# /// scripts # ❌ Wrong: "scripts" not "script"
|
||||
# requires-python = ">=3.11"
|
||||
# ///
|
||||
|
||||
# // script # ❌ Wrong: "//" not "///"
|
||||
# requires-python = ">=3.11"
|
||||
# //
|
||||
|
||||
# /// script # ❌ Wrong: No closing marker
|
||||
# requires-python = ">=3.11"
|
||||
```
|
||||
|
||||
**CORRECT**:
|
||||
|
||||
```python
|
||||
# /// script # ✓ Exactly "/// script"
|
||||
# requires-python = ">=3.11"
|
||||
# /// # ✓ Exactly "///"
|
||||
```
|
||||
|
||||
## Dependency Management Mistakes
|
||||
|
||||
### ❌ No Version Constraints
|
||||
|
||||
**WRONG**:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "httpx", # ❌ No version specified
|
||||
# "requests", # ❌ Could break unexpectedly
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
**CORRECT**:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0", # ✓ Minimum version specified
|
||||
# "requests>=2.31.0", # ✓ Prevents breaking changes
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
### ❌ Overly Strict Pinning
|
||||
|
||||
**WRONG** (for utility scripts):
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "httpx==0.27.0", # ❌ Too strict, prevents updates
|
||||
# "rich==13.7.0", # ❌ Blocks security fixes
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
**CORRECT**:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0", # ✓ Allows updates
|
||||
# "rich>=13.0.0", # ✓ Allows security fixes
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
**NOTE**: Exact pinning (`==`) is appropriate for deployment scripts where reproducibility is critical.
|
||||
|
||||
## Additional Pitfalls
|
||||
|
||||
### ❌ Missing Shebang
|
||||
|
||||
**WRONG**:
|
||||
|
||||
```python
|
||||
# No shebang - script won't be directly executable
|
||||
# /// script
|
||||
# dependencies = ["requests"]
|
||||
# ///
|
||||
```
|
||||
|
||||
**CORRECT**:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# dependencies = ["requests"]
|
||||
# ///
|
||||
```
|
||||
|
||||
### ❌ Broad Exception Handling
|
||||
|
||||
**WRONG**:
|
||||
|
||||
```python
|
||||
try:
|
||||
do_something()
|
||||
except Exception: # Too broad!
|
||||
pass
|
||||
```
|
||||
|
||||
**CORRECT**:
|
||||
|
||||
```python
|
||||
try:
|
||||
do_something()
|
||||
except (FileNotFoundError, PermissionError) as e:
|
||||
print(f"Error: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
### ❌ Platform-Specific Code Without Guards
|
||||
|
||||
**WRONG**:
|
||||
|
||||
```python
|
||||
import pwd # Unix-only, crashes on Windows
|
||||
user = pwd.getpwuid(os.getuid())
|
||||
```
|
||||
|
||||
**CORRECT**:
|
||||
|
||||
```python
|
||||
import sys
|
||||
if sys.platform != "win32":
|
||||
import pwd
|
||||
user = pwd.getpwuid(os.getuid())
|
||||
else:
|
||||
user = os.environ.get("USERNAME")
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
**Never use**:
|
||||
|
||||
- `[tool.uv.metadata]` (invalid field)
|
||||
- Custom fields in PEP 723 metadata (only `requires-python` and `dependencies`)
|
||||
- Hardcoded secrets
|
||||
- Scripts for complex applications (>500 lines, web apps, services)
|
||||
|
||||
**Always use**:
|
||||
|
||||
- Valid PEP 723 format only
|
||||
- `[tool.uv]` (without `.metadata`) for valid fields like `exclude-newer`
|
||||
- Environment variables or Infisical for secrets
|
||||
- Error handling for external calls
|
||||
- Version constraints on dependencies
|
||||
- Proper projects when scripts become too complex
|
||||
363
skills/python-uv-scripts/anti-patterns/when-not-to-use.md
Normal file
363
skills/python-uv-scripts/anti-patterns/when-not-to-use.md
Normal file
@@ -0,0 +1,363 @@
|
||||
# When NOT to Use Single-File Scripts
|
||||
|
||||
This document helps determine when to use a single-file uv script vs. when to use a proper uv project.
|
||||
|
||||
## Decision Tree
|
||||
|
||||
```text
|
||||
Is this a Python program?
|
||||
├─ No → Use appropriate language/tool
|
||||
└─ Yes
|
||||
├─ Is it a one-time task or simple automation?
|
||||
│ ├─ Yes → Consider single-file script
|
||||
│ └─ No → Use proper uv project
|
||||
└─ Does it meet ANY of these criteria?
|
||||
├─ >500 lines of code → Use proper uv project
|
||||
├─ Multiple Python files needed → Use proper uv project
|
||||
├─ Web application or API → Use proper uv project
|
||||
├─ Long-running service → Use proper uv project
|
||||
├─ Complex configuration → Use proper uv project
|
||||
├─ Shared library code → Use proper uv project
|
||||
├─ Heavy ML/data dependencies → Use proper uv project
|
||||
└─ None of the above → Single-file script is appropriate
|
||||
```
|
||||
|
||||
## Use Proper uv Project When
|
||||
|
||||
### 1. Code Complexity
|
||||
|
||||
**Use project if**:
|
||||
|
||||
- Script exceeds 500 lines
|
||||
- Logic is spread across multiple functions/classes
|
||||
- Code would benefit from splitting into modules
|
||||
- Complex data models or class hierarchies
|
||||
|
||||
**Example - Too Complex for Script**:
|
||||
|
||||
```python
|
||||
# This needs a proper project structure:
|
||||
# - 800 lines of code
|
||||
# - Multiple classes (User, Database, API, Config)
|
||||
# - Would be clearer as separate modules
|
||||
# - Needs tests
|
||||
```
|
||||
|
||||
### 2. Multiple Files Needed
|
||||
|
||||
**Use project if**:
|
||||
|
||||
- Shared utilities across multiple scripts
|
||||
- Common data models used by different tools
|
||||
- Reusable library code
|
||||
- Multiple entry points
|
||||
|
||||
**Example**:
|
||||
|
||||
```text
|
||||
# This needs a project:
|
||||
my-tool/
|
||||
├── src/
|
||||
│ ├── __init__.py
|
||||
│ ├── database.py # Shared by multiple tools
|
||||
│ ├── models.py # Common data structures
|
||||
│ └── utils.py # Utility functions
|
||||
├── scripts/
|
||||
│ ├── import_data.py # Uses shared code
|
||||
│ └── export_data.py # Uses shared code
|
||||
└── pyproject.toml
|
||||
```
|
||||
|
||||
### 3. Web Applications
|
||||
|
||||
**Use project for**:
|
||||
|
||||
- Flask/FastAPI/Django applications
|
||||
- REST APIs
|
||||
- Web services
|
||||
- Applications with routes/controllers
|
||||
|
||||
**Example - Needs Project**:
|
||||
|
||||
```python
|
||||
# Don't use single-file script for web apps:
|
||||
from flask import Flask, request, jsonify
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
# 50+ routes
|
||||
# Database models
|
||||
# Authentication
|
||||
# Background tasks
|
||||
# Configuration management
|
||||
|
||||
# This should be a proper project structure
|
||||
```
|
||||
|
||||
### 4. Long-Running Services
|
||||
|
||||
**Use project for**:
|
||||
|
||||
- Daemons
|
||||
- Background workers
|
||||
- Queue consumers
|
||||
- Services that run continuously
|
||||
|
||||
**Example**:
|
||||
|
||||
```python
|
||||
# Don't use script for services:
|
||||
# - Runs 24/7
|
||||
# - Monitors message queue
|
||||
# - Complex retry logic
|
||||
# - Logging configuration
|
||||
# - Health checks
|
||||
# - Graceful shutdown
|
||||
|
||||
# Needs proper project structure with:
|
||||
# - Proper logging setup
|
||||
# - Configuration management
|
||||
# - Process management (systemd/supervisor)
|
||||
```
|
||||
|
||||
### 5. Complex Configuration
|
||||
|
||||
**Use project if**:
|
||||
|
||||
- Multiple environment configs (dev/staging/prod)
|
||||
- YAML/JSON configuration files
|
||||
- Feature flags
|
||||
- Database connection pools
|
||||
|
||||
**Example**:
|
||||
|
||||
```text
|
||||
# This needs a project:
|
||||
config/
|
||||
├── dev.yaml
|
||||
├── staging.yaml
|
||||
└── production.yaml
|
||||
|
||||
# Single-file scripts should use simple env vars instead
|
||||
```
|
||||
|
||||
### 6. Heavy Dependencies
|
||||
|
||||
**Use project for**:
|
||||
|
||||
- Machine learning frameworks (TensorFlow, PyTorch)
|
||||
- Large data processing (PySpark)
|
||||
- Complex scientific computing
|
||||
- GUI frameworks
|
||||
|
||||
**Example - Too Heavy**:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "tensorflow>=2.15.0", # ❌ ~500MB download
|
||||
# "torch>=2.1.0", # ❌ ~800MB download
|
||||
# "transformers>=4.35.0", # ❌ Complex dependency tree
|
||||
# ]
|
||||
# ///
|
||||
|
||||
# Use proper project with managed virtual environment
|
||||
```
|
||||
|
||||
### 7. Testing Requirements
|
||||
|
||||
**Use project if**:
|
||||
|
||||
- Comprehensive test suite needed
|
||||
- Multiple test files
|
||||
- Fixtures and mocking
|
||||
- CI/CD integration
|
||||
|
||||
**Example**:
|
||||
|
||||
```text
|
||||
# This needs a project:
|
||||
tests/
|
||||
├── unit/
|
||||
│ ├── test_models.py
|
||||
│ ├── test_utils.py
|
||||
│ └── test_api.py
|
||||
├── integration/
|
||||
│ └── test_database.py
|
||||
└── conftest.py
|
||||
|
||||
# Can't reasonably organize this with single-file script
|
||||
```
|
||||
|
||||
### 8. Team Collaboration
|
||||
|
||||
**Use project if**:
|
||||
|
||||
- Multiple developers working on code
|
||||
- Code review processes
|
||||
- Versioning and releases
|
||||
- Documentation requirements
|
||||
|
||||
## Single-File Scripts ARE Appropriate For
|
||||
|
||||
### ✅ Good Use Cases
|
||||
|
||||
**One-time tasks**:
|
||||
|
||||
```python
|
||||
# Convert CSV format
|
||||
# Migrate data between systems
|
||||
# Clean up old files
|
||||
# Generate reports
|
||||
```
|
||||
|
||||
**Simple automation**:
|
||||
|
||||
```python
|
||||
# Check server health
|
||||
# Send notifications
|
||||
# Backup files
|
||||
# Parse logs
|
||||
```
|
||||
|
||||
**CLI utilities**:
|
||||
|
||||
```python
|
||||
# Format files
|
||||
# Validate data
|
||||
# Query APIs
|
||||
# Process input
|
||||
```
|
||||
|
||||
**Prototyping**:
|
||||
|
||||
```python
|
||||
# Test API endpoints
|
||||
# Experiment with libraries
|
||||
# Quick data analysis
|
||||
# Proof of concept
|
||||
```
|
||||
|
||||
### ✅ Characteristics of Good Single-File Scripts
|
||||
|
||||
- **<500 lines of code**
|
||||
- **Self-contained logic**
|
||||
- **Simple, clear purpose**
|
||||
- **Minimal dependencies (1-5 packages)**
|
||||
- **Standalone execution**
|
||||
- **Quick to understand**
|
||||
|
||||
## Examples of Good Single-File Scripts
|
||||
|
||||
### Example 1: Health Check
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "psutil>=5.9.0",
|
||||
# "rich>=13.0.0",
|
||||
# ]
|
||||
# ///
|
||||
"""Check system health and display metrics."""
|
||||
|
||||
import psutil
|
||||
from rich import print
|
||||
|
||||
def main():
|
||||
cpu = psutil.cpu_percent()
|
||||
mem = psutil.virtual_memory().percent
|
||||
disk = psutil.disk_usage('/').percent
|
||||
|
||||
print(f"CPU: {cpu}% | Memory: {mem}% | Disk: {disk}%")
|
||||
|
||||
if cpu > 80 or mem > 80 or disk > 80:
|
||||
print("[red]⚠ High resource usage![/red]")
|
||||
exit(1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
```
|
||||
|
||||
### Example 2: API Query
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# ]
|
||||
# ///
|
||||
"""Query API and display results."""
|
||||
|
||||
import httpx
|
||||
import sys
|
||||
|
||||
def main():
|
||||
response = httpx.get("https://api.github.com/users/octocat")
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
print(f"Name: {data['name']}")
|
||||
print(f"Public repos: {data['public_repos']}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
```
|
||||
|
||||
## Migration Path
|
||||
|
||||
**When a script outgrows single-file format**:
|
||||
|
||||
1. Create proper uv project:
|
||||
|
||||
```bash
|
||||
uv init my-tool
|
||||
cd my-tool
|
||||
```
|
||||
|
||||
2. Move script logic to `src/`:
|
||||
|
||||
```bash
|
||||
mv script.py src/my_tool/main.py
|
||||
```
|
||||
|
||||
3. Add dependencies to `pyproject.toml`:
|
||||
|
||||
```toml
|
||||
[project]
|
||||
dependencies = [
|
||||
"httpx>=0.27.0",
|
||||
"rich>=13.0.0",
|
||||
]
|
||||
```
|
||||
|
||||
4. Create entry point in `pyproject.toml`:
|
||||
|
||||
```toml
|
||||
[project.scripts]
|
||||
my-tool = "my_tool.main:main"
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
**Use single-file scripts for**:
|
||||
|
||||
- Simple automation (<500 lines)
|
||||
- One-off tasks
|
||||
- CLI utilities
|
||||
- Prototypes
|
||||
- Standalone tools
|
||||
|
||||
**Use proper uv projects for**:
|
||||
|
||||
- Complex applications (>500 lines)
|
||||
- Multiple files/modules
|
||||
- Web applications
|
||||
- Long-running services
|
||||
- Heavy dependencies
|
||||
- Team collaboration
|
||||
- Comprehensive testing
|
||||
|
||||
**When in doubt**: Start with a script. If it grows too complex, migrate to a project.
|
||||
72
skills/python-uv-scripts/assets/templates/api-client.py
Executable file
72
skills/python-uv-scripts/assets/templates/api-client.py
Executable file
@@ -0,0 +1,72 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# "rich>=13.0.0",
|
||||
# ]
|
||||
# ///
|
||||
"""
|
||||
API client template using httpx.
|
||||
|
||||
Demonstrates HTTP requests, error handling, and structured output.
|
||||
Includes environment variable usage for API credentials.
|
||||
|
||||
Usage:
|
||||
export API_URL="https://api.example.com"
|
||||
export API_TOKEN="your-token"
|
||||
python api-client.py
|
||||
"""
|
||||
|
||||
import httpx
|
||||
import os
|
||||
import sys
|
||||
from rich import print
|
||||
from rich.console import Console
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
def get_env_var(name: str) -> str:
|
||||
"""Get required environment variable or exit with error."""
|
||||
value = os.getenv(name)
|
||||
if not value:
|
||||
console.print(f"[red]Error: {name} environment variable not set[/red]")
|
||||
sys.exit(1)
|
||||
return value
|
||||
|
||||
|
||||
def fetch_data(api_url: str, token: str):
|
||||
"""Fetch data from API with error handling."""
|
||||
headers = {"Authorization": f"Bearer {token}"}
|
||||
|
||||
try:
|
||||
with httpx.Client() as client:
|
||||
response = client.get(api_url, headers=headers, timeout=10.0)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
except httpx.HTTPStatusError as e:
|
||||
console.print(f"[red]HTTP error {e.response.status_code}[/red]")
|
||||
console.print(f"Response: {e.response.text}")
|
||||
sys.exit(1)
|
||||
except httpx.RequestError as e:
|
||||
console.print(f"[red]Request failed: {e}[/red]")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point."""
|
||||
api_url = get_env_var("API_URL")
|
||||
api_token = get_env_var("API_TOKEN")
|
||||
|
||||
console.print(f"[cyan]Fetching data from {api_url}...[/cyan]")
|
||||
|
||||
data = fetch_data(api_url, api_token)
|
||||
|
||||
# Process and display data
|
||||
print(data)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
24
skills/python-uv-scripts/assets/templates/basic-script.py
Executable file
24
skills/python-uv-scripts/assets/templates/basic-script.py
Executable file
@@ -0,0 +1,24 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""
|
||||
Basic Python script template using uv.
|
||||
|
||||
This is a minimal working example showing the correct PEP 723 format.
|
||||
No external dependencies - uses only Python standard library.
|
||||
|
||||
Usage:
|
||||
python basic-script.py
|
||||
"""
|
||||
|
||||
def main():
|
||||
"""Main entry point for the script."""
|
||||
print("Hello from uv!")
|
||||
|
||||
# Your code here
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
62
skills/python-uv-scripts/assets/templates/cli-app.py
Executable file
62
skills/python-uv-scripts/assets/templates/cli-app.py
Executable file
@@ -0,0 +1,62 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "typer>=0.9.0",
|
||||
# "rich>=13.0.0",
|
||||
# ]
|
||||
# ///
|
||||
"""
|
||||
CLI application template using Typer and Rich.
|
||||
|
||||
Demonstrates command-line argument parsing, subcommands, and formatted output.
|
||||
|
||||
Usage:
|
||||
python cli-app.py --help
|
||||
python cli-app.py greet "World"
|
||||
python cli-app.py process input.txt --output output.txt
|
||||
"""
|
||||
|
||||
import typer
|
||||
from rich import print
|
||||
from rich.console import Console
|
||||
from pathlib import Path
|
||||
|
||||
app = typer.Typer()
|
||||
console = Console()
|
||||
|
||||
|
||||
@app.command()
|
||||
def greet(name: str):
|
||||
"""Greet someone by name."""
|
||||
print(f"[green]Hello, {name}![/green]")
|
||||
|
||||
|
||||
@app.command()
|
||||
def process(
|
||||
input_file: Path = typer.Argument(..., help="Input file to process"),
|
||||
output: Path = typer.Option(None, "--output", "-o", help="Output file path"),
|
||||
):
|
||||
"""Process a file and optionally write results."""
|
||||
if not input_file.exists():
|
||||
console.print(f"[red]Error: {input_file} not found[/red]")
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
# Process file
|
||||
with open(input_file) as f:
|
||||
content = f.read()
|
||||
|
||||
console.print(f"[cyan]Processing {input_file}...[/cyan]")
|
||||
|
||||
# Your processing logic here
|
||||
result = content.upper() # Example transformation
|
||||
|
||||
if output:
|
||||
output.write_text(result)
|
||||
console.print(f"[green]✓ Written to {output}[/green]")
|
||||
else:
|
||||
print(result)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
app()
|
||||
84
skills/python-uv-scripts/assets/templates/data-processor.py
Executable file
84
skills/python-uv-scripts/assets/templates/data-processor.py
Executable file
@@ -0,0 +1,84 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "polars>=0.20.0",
|
||||
# "rich>=13.0.0",
|
||||
# ]
|
||||
# ///
|
||||
"""
|
||||
Data processing template using Polars.
|
||||
|
||||
Demonstrates reading, transforming, and analyzing data with
|
||||
modern data processing libraries.
|
||||
|
||||
Usage:
|
||||
python data-processor.py input.csv
|
||||
python data-processor.py data/*.csv --output results/
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
import polars as pl
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
def process_csv(file_path: Path) -> pl.DataFrame:
|
||||
"""Read and process a CSV file."""
|
||||
try:
|
||||
df = pl.read_csv(file_path)
|
||||
console.print(f"[cyan]Loaded {len(df)} rows from {file_path.name}[/cyan]")
|
||||
|
||||
# Example transformations
|
||||
# df = df.filter(pl.col("status") == "active")
|
||||
# df = df.with_columns(pl.col("amount").cast(pl.Float64))
|
||||
|
||||
return df
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error processing {file_path}: {e}[/red]")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def display_summary(df: pl.DataFrame):
|
||||
"""Display data summary using Rich tables."""
|
||||
table = Table(title="Data Summary")
|
||||
table.add_column("Metric", style="cyan")
|
||||
table.add_column("Value", style="green")
|
||||
|
||||
table.add_row("Total Rows", str(len(df)))
|
||||
table.add_row("Columns", str(len(df.columns)))
|
||||
|
||||
# Example statistics
|
||||
# if "amount" in df.columns:
|
||||
# table.add_row("Total Amount", f"${df['amount'].sum():,.2f}")
|
||||
|
||||
console.print(table)
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point."""
|
||||
if len(sys.argv) < 2:
|
||||
console.print("[red]Usage: python data-processor.py <input.csv>[/red]")
|
||||
sys.exit(1)
|
||||
|
||||
input_path = Path(sys.argv[1])
|
||||
|
||||
if not input_path.exists():
|
||||
console.print(f"[red]Error: {input_path} not found[/red]")
|
||||
sys.exit(1)
|
||||
|
||||
df = process_csv(input_path)
|
||||
display_summary(df)
|
||||
|
||||
# Optional: Save results
|
||||
# output_path = Path("output.csv")
|
||||
# df.write_csv(output_path)
|
||||
# console.print(f"[green]✓ Saved to {output_path}[/green]")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,257 @@
|
||||
#!/usr/bin/env -S uv run --script --quiet
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "rich>=13.0.0",
|
||||
# "typer>=0.9.0",
|
||||
# ]
|
||||
# ///
|
||||
"""
|
||||
Proxmox Cluster Health Checker - Production Ready Example
|
||||
|
||||
Purpose: cluster-monitoring
|
||||
Team: infrastructure
|
||||
Author: devops@spaceships.work
|
||||
|
||||
This is an enhanced version of the basic cluster health checker,
|
||||
demonstrating all best practices for production uv scripts.
|
||||
|
||||
Features:
|
||||
- Rich CLI with Typer
|
||||
- Structured output with Rich
|
||||
- Proper error handling
|
||||
- Input validation
|
||||
- Security best practices
|
||||
- Comprehensive logging
|
||||
- Exit codes for automation
|
||||
|
||||
Usage:
|
||||
# Interactive mode with rich output
|
||||
./check_cluster_health_enhanced.py --node foxtrot
|
||||
|
||||
# JSON output for automation
|
||||
./check_cluster_health_enhanced.py --node foxtrot --json
|
||||
|
||||
# Quiet mode (only errors)
|
||||
./check_cluster_health_enhanced.py --node foxtrot --quiet
|
||||
|
||||
Examples:
|
||||
# Check specific cluster node
|
||||
./check_cluster_health_enhanced.py --node golf
|
||||
|
||||
# CI/CD integration
|
||||
./check_cluster_health_enhanced.py --json | jq '.is_healthy'
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass, asdict
|
||||
from typing import Optional
|
||||
|
||||
import typer
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
from rich import print as rprint
|
||||
|
||||
app = typer.Typer(help="Check Proxmox cluster health")
|
||||
console = Console()
|
||||
|
||||
|
||||
@dataclass
|
||||
class NodeStatus:
|
||||
"""Cluster node status"""
|
||||
name: str
|
||||
online: bool
|
||||
node_id: int
|
||||
ip: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class ClusterHealth:
|
||||
"""Overall cluster health status"""
|
||||
cluster_name: str
|
||||
quorate: bool
|
||||
node_count: int
|
||||
expected_votes: int
|
||||
total_votes: int
|
||||
nodes: list[NodeStatus]
|
||||
warnings: list[str]
|
||||
errors: list[str]
|
||||
|
||||
@property
|
||||
def is_healthy(self) -> bool:
|
||||
"""Check if cluster is healthy"""
|
||||
return self.quorate and len(self.errors) == 0
|
||||
|
||||
|
||||
def validate_hostname(hostname: str) -> bool:
|
||||
"""Validate hostname format"""
|
||||
pattern = r'^[a-zA-Z0-9][a-zA-Z0-9.-]*[a-zA-Z0-9]$'
|
||||
return bool(re.match(pattern, hostname)) and len(hostname) <= 253
|
||||
|
||||
|
||||
def run_ssh_command(node: str, command: str) -> str:
|
||||
"""Execute command on remote node via SSH"""
|
||||
if not validate_hostname(node):
|
||||
console.print(f"[red]Error: Invalid hostname: {node}[/red]", file=sys.stderr)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["ssh", f"root@{node}", command],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
timeout=30
|
||||
)
|
||||
return result.stdout
|
||||
except subprocess.CalledProcessError as e:
|
||||
console.print(f"[red]Command failed: {command}[/red]", file=sys.stderr)
|
||||
console.print(f" {e.stderr}", file=sys.stderr)
|
||||
raise typer.Exit(code=1)
|
||||
except subprocess.TimeoutExpired:
|
||||
console.print(f"[red]Command timed out: {command}[/red]", file=sys.stderr)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
|
||||
def check_cluster_status(node: str) -> ClusterHealth:
|
||||
"""Check cluster status and return health object"""
|
||||
health = ClusterHealth(
|
||||
cluster_name="",
|
||||
quorate=False,
|
||||
node_count=0,
|
||||
expected_votes=0,
|
||||
total_votes=0,
|
||||
nodes=[],
|
||||
warnings=[],
|
||||
errors=[]
|
||||
)
|
||||
|
||||
# Get cluster status
|
||||
output = run_ssh_command(node, "pvecm status")
|
||||
|
||||
# Parse cluster name
|
||||
if match := re.search(r'Cluster name:\s+(\S+)', output):
|
||||
health.cluster_name = match.group(1)
|
||||
|
||||
# Parse quorum
|
||||
if match := re.search(r'Quorate:\s+(\w+)', output):
|
||||
health.quorate = match.group(1).lower() == 'yes'
|
||||
|
||||
if not health.quorate:
|
||||
health.errors.append("Cluster does not have quorum!")
|
||||
|
||||
# Parse votes
|
||||
if match := re.search(r'Nodes:\s+(\d+)', output):
|
||||
health.node_count = int(match.group(1))
|
||||
|
||||
if match := re.search(r'Expected votes:\s+(\d+)', output):
|
||||
health.expected_votes = int(match.group(1))
|
||||
|
||||
if match := re.search(r'Total votes:\s+(\d+)', output):
|
||||
health.total_votes = int(match.group(1))
|
||||
|
||||
# Get node list
|
||||
output = run_ssh_command(node, "pvecm nodes")
|
||||
for line in output.strip().split('\n')[1:]: # Skip header
|
||||
if not line.strip():
|
||||
continue
|
||||
|
||||
parts = line.split()
|
||||
if len(parts) >= 3:
|
||||
try:
|
||||
health.nodes.append(NodeStatus(
|
||||
name=parts[2] if len(parts) >= 3 else "unknown",
|
||||
online=True,
|
||||
node_id=int(parts[0]),
|
||||
ip=parts[3] if len(parts) >= 4 else "unknown"
|
||||
))
|
||||
except (ValueError, IndexError):
|
||||
health.warnings.append(f"Failed to parse node line: {line}")
|
||||
|
||||
return health
|
||||
|
||||
|
||||
@app.command()
|
||||
def main(
|
||||
node: str = typer.Option("foxtrot", help="Cluster node to check"),
|
||||
json_output: bool = typer.Option(False, "--json", help="Output as JSON"),
|
||||
quiet: bool = typer.Option(False, help="Only show errors"),
|
||||
):
|
||||
"""
|
||||
Check Proxmox cluster health
|
||||
|
||||
Connects to a cluster node via SSH and checks:
|
||||
- Quorum status
|
||||
- Node membership
|
||||
- Vote distribution
|
||||
"""
|
||||
try:
|
||||
health = check_cluster_status(node)
|
||||
|
||||
if json_output:
|
||||
# JSON output for automation
|
||||
print(json.dumps(asdict(health), indent=2))
|
||||
sys.exit(0 if health.is_healthy else 1)
|
||||
|
||||
if not quiet:
|
||||
# Rich table output
|
||||
console.print(f"\n[bold]Cluster Health: {health.cluster_name}[/bold]")
|
||||
console.print("=" * 60)
|
||||
|
||||
# Status
|
||||
quorum_icon = "✓" if health.quorate else "✗"
|
||||
quorum_color = "green" if health.quorate else "red"
|
||||
console.print(f"Quorum: [{quorum_color}]{quorum_icon} {'YES' if health.quorate else 'NO'}[/{quorum_color}]")
|
||||
console.print(f"Nodes: {health.node_count} ({health.total_votes}/{health.expected_votes} votes)")
|
||||
|
||||
# Nodes table
|
||||
table = Table(title="\nCluster Nodes")
|
||||
table.add_column("Node", style="cyan")
|
||||
table.add_column("Node ID", style="magenta")
|
||||
table.add_column("IP Address", style="yellow")
|
||||
table.add_column("Status", style="green")
|
||||
|
||||
for node_status in health.nodes:
|
||||
status_icon = "✓" if node_status.online else "✗"
|
||||
table.add_row(
|
||||
node_status.name,
|
||||
str(node_status.node_id),
|
||||
node_status.ip,
|
||||
status_icon
|
||||
)
|
||||
|
||||
console.print(table)
|
||||
|
||||
# Warnings
|
||||
if health.warnings:
|
||||
console.print("\n[yellow]Warnings:[/yellow]")
|
||||
for warning in health.warnings:
|
||||
console.print(f" ⚠ {warning}")
|
||||
|
||||
# Errors
|
||||
if health.errors:
|
||||
console.print("\n[red]Errors:[/red]")
|
||||
for error in health.errors:
|
||||
console.print(f" ✗ {error}")
|
||||
|
||||
console.print("\n" + "=" * 60)
|
||||
|
||||
# Final status
|
||||
if health.is_healthy:
|
||||
if not quiet:
|
||||
console.print("[green]Status: ✓ HEALTHY[/green]\n")
|
||||
sys.exit(0)
|
||||
else:
|
||||
console.print("[red]Status: ✗ UNHEALTHY[/red]\n", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"[red]Unexpected error: {e}[/red]", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
app()
|
||||
358
skills/python-uv-scripts/examples/04-api-clients/netbox_client.py
Executable file
358
skills/python-uv-scripts/examples/04-api-clients/netbox_client.py
Executable file
@@ -0,0 +1,358 @@
|
||||
#!/usr/bin/env -S uv run --script --quiet
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "pynetbox>=7.0.0",
|
||||
# "infisical-python>=2.3.3",
|
||||
# "rich>=13.0.0",
|
||||
# ]
|
||||
# ///
|
||||
"""
|
||||
NetBox API Client Example
|
||||
|
||||
Purpose: api-client-example
|
||||
Team: infrastructure
|
||||
Author: devops@spaceships.work
|
||||
|
||||
Demonstrates best practices for API client scripts using uv:
|
||||
- PEP 723 inline dependencies
|
||||
- Infisical for secrets management
|
||||
- Error handling and validation
|
||||
- Rich output formatting
|
||||
- Type hints and documentation
|
||||
|
||||
This is a production-ready example showing all patterns together.
|
||||
|
||||
Usage:
|
||||
# List Matrix cluster VMs
|
||||
./netbox_client.py
|
||||
|
||||
# Query specific VM
|
||||
./netbox_client.py --vm docker-01
|
||||
|
||||
Example output:
|
||||
Matrix Cluster Virtual Machines
|
||||
┌────┬───────────┬──────┬─────────┬──────────────┬──────────────────────────────────┐
|
||||
│ ID │ Name │vCPUs │Memory MB│ Status │ Primary IP │
|
||||
├────┼───────────┼──────┼─────────┼──────────────┼──────────────────────────────────┤
|
||||
│ 1 │ docker-01 │ 4 │ 8192 │ active │ 192.168.3.10/24 │
|
||||
│ 2 │ k8s-01 │ 8 │ 16384 │ active │ 192.168.3.20/24 │
|
||||
└────┴───────────┴──────┴─────────┴──────────────┴──────────────────────────────────┘
|
||||
"""
|
||||
|
||||
import re
|
||||
import sys
|
||||
from typing import Optional
|
||||
from dataclasses import dataclass
|
||||
|
||||
import pynetbox
|
||||
from infisical import InfisicalClient
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
from rich.panel import Panel
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Configuration
|
||||
# ============================================================================
|
||||
|
||||
@dataclass
|
||||
class NetBoxConfig:
|
||||
"""NetBox connection configuration."""
|
||||
url: str = "https://netbox.spaceships.work"
|
||||
project_id: str = "7b832220-24c0-45bc-a5f1-ce9794a31259"
|
||||
environment: str = "prod"
|
||||
path: str = "/matrix"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Authentication
|
||||
# ============================================================================
|
||||
|
||||
def get_netbox_client(config: NetBoxConfig) -> Optional[pynetbox.api]:
|
||||
"""
|
||||
Get authenticated NetBox API client.
|
||||
|
||||
Uses Infisical to securely retrieve API token (Virgo-Core security pattern).
|
||||
|
||||
Args:
|
||||
config: NetBox configuration
|
||||
|
||||
Returns:
|
||||
Authenticated pynetbox client or None on error
|
||||
|
||||
Raises:
|
||||
Exception: If token cannot be retrieved or connection fails
|
||||
"""
|
||||
try:
|
||||
# Get token from Infisical (never hardcoded)
|
||||
client = InfisicalClient()
|
||||
token = client.get_secret(
|
||||
secret_name="NETBOX_API_TOKEN",
|
||||
project_id=config.project_id,
|
||||
environment=config.environment,
|
||||
path=config.path
|
||||
).secret_value
|
||||
|
||||
if not token:
|
||||
raise ValueError("NETBOX_API_TOKEN is empty")
|
||||
|
||||
# Connect to NetBox
|
||||
nb = pynetbox.api(config.url, token=token)
|
||||
|
||||
# Test connection
|
||||
_ = nb.status()
|
||||
|
||||
return nb
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"[red]Failed to connect to NetBox: {e}[/red]")
|
||||
return None
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Data Validation
|
||||
# ============================================================================
|
||||
|
||||
def validate_vm_name(name: str) -> bool:
|
||||
"""
|
||||
Validate VM name format.
|
||||
|
||||
Pattern: lowercase letters, numbers, hyphens only
|
||||
Example: docker-01, k8s-01-master
|
||||
|
||||
Args:
|
||||
name: VM name to validate
|
||||
|
||||
Returns:
|
||||
True if valid, False otherwise
|
||||
"""
|
||||
pattern = r'^[a-z0-9-]+$'
|
||||
return bool(re.match(pattern, name))
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# NetBox Queries
|
||||
# ============================================================================
|
||||
|
||||
def get_cluster_vms(nb: pynetbox.api, cluster_name: str = "Matrix") -> list:
|
||||
"""
|
||||
Get all VMs in a cluster.
|
||||
|
||||
Args:
|
||||
nb: NetBox API client
|
||||
cluster_name: Cluster name to query
|
||||
|
||||
Returns:
|
||||
List of VM objects
|
||||
"""
|
||||
try:
|
||||
vms = nb.virtualization.virtual_machines.filter(cluster=cluster_name.lower())
|
||||
return list(vms)
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error querying VMs: {e}[/red]")
|
||||
return []
|
||||
|
||||
|
||||
def get_vm_details(nb: pynetbox.api, vm_name: str) -> Optional[dict]:
|
||||
"""
|
||||
Get detailed information about a VM.
|
||||
|
||||
Args:
|
||||
nb: NetBox API client
|
||||
vm_name: VM name
|
||||
|
||||
Returns:
|
||||
VM details dict or None if not found
|
||||
"""
|
||||
try:
|
||||
# Validate name
|
||||
if not validate_vm_name(vm_name):
|
||||
console.print(f"[red]Invalid VM name format: {vm_name}[/red]")
|
||||
return None
|
||||
|
||||
vm = nb.virtualization.virtual_machines.get(name=vm_name)
|
||||
|
||||
if not vm:
|
||||
console.print(f"[yellow]VM '{vm_name}' not found[/yellow]")
|
||||
return None
|
||||
|
||||
# Get interfaces
|
||||
interfaces = nb.virtualization.interfaces.filter(virtual_machine_id=vm.id)
|
||||
|
||||
# Get IPs
|
||||
ips = []
|
||||
for iface in interfaces:
|
||||
iface_ips = nb.ipam.ip_addresses.filter(vminterface_id=iface.id)
|
||||
ips.extend(iface_ips)
|
||||
|
||||
return {
|
||||
"vm": vm,
|
||||
"interfaces": list(interfaces),
|
||||
"ips": ips
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error getting VM details: {e}[/red]")
|
||||
return None
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Output Formatting
|
||||
# ============================================================================
|
||||
|
||||
def display_vms_table(vms: list) -> None:
|
||||
"""
|
||||
Display VMs in a formatted table.
|
||||
|
||||
Args:
|
||||
vms: List of VM objects
|
||||
"""
|
||||
if not vms:
|
||||
console.print("[yellow]No VMs found[/yellow]")
|
||||
return
|
||||
|
||||
table = Table(title="Matrix Cluster Virtual Machines")
|
||||
table.add_column("ID", style="cyan", justify="right")
|
||||
table.add_column("Name", style="green")
|
||||
table.add_column("vCPUs", justify="right")
|
||||
table.add_column("Memory (MB)", justify="right")
|
||||
table.add_column("Status")
|
||||
table.add_column("Primary IP", style="yellow")
|
||||
|
||||
for vm in vms:
|
||||
table.add_row(
|
||||
str(vm.id),
|
||||
vm.name,
|
||||
str(vm.vcpus) if vm.vcpus else "N/A",
|
||||
str(vm.memory) if vm.memory else "N/A",
|
||||
vm.status.value if hasattr(vm.status, 'value') else str(vm.status),
|
||||
str(vm.primary_ip4.address) if vm.primary_ip4 else "N/A"
|
||||
)
|
||||
|
||||
console.print(table)
|
||||
console.print(f"\n[green]Total VMs: {len(vms)}[/green]")
|
||||
|
||||
|
||||
def display_vm_details(details: dict) -> None:
|
||||
"""
|
||||
Display detailed VM information.
|
||||
|
||||
Args:
|
||||
details: VM details dict from get_vm_details()
|
||||
"""
|
||||
vm = details["vm"]
|
||||
interfaces = details["interfaces"]
|
||||
ips = details["ips"]
|
||||
|
||||
# VM info panel
|
||||
info = (
|
||||
f"[green]ID:[/green] {vm.id}\n"
|
||||
f"[green]Name:[/green] {vm.name}\n"
|
||||
f"[green]Cluster:[/green] {vm.cluster.name if vm.cluster else 'N/A'}\n"
|
||||
f"[green]Status:[/green] {vm.status}\n"
|
||||
f"[green]vCPUs:[/green] {vm.vcpus or 'N/A'}\n"
|
||||
f"[green]Memory:[/green] {vm.memory or 'N/A'} MB\n"
|
||||
f"[green]Disk:[/green] {vm.disk or 'N/A'} GB\n"
|
||||
f"[green]Primary IP:[/green] {vm.primary_ip4.address if vm.primary_ip4 else 'N/A'}"
|
||||
)
|
||||
|
||||
console.print(Panel(info, title=f"VM: {vm.name}", border_style="cyan"))
|
||||
|
||||
# Interfaces table
|
||||
if interfaces:
|
||||
iface_table = Table(title="Network Interfaces")
|
||||
iface_table.add_column("Name", style="cyan")
|
||||
iface_table.add_column("Type")
|
||||
iface_table.add_column("Enabled")
|
||||
iface_table.add_column("MTU")
|
||||
|
||||
for iface in interfaces:
|
||||
iface_table.add_row(
|
||||
iface.name,
|
||||
iface.type.value if hasattr(iface.type, 'value') else str(iface.type),
|
||||
"✓" if iface.enabled else "✗",
|
||||
str(iface.mtu) if iface.mtu else "default"
|
||||
)
|
||||
|
||||
console.print("\n", iface_table)
|
||||
|
||||
# IPs table
|
||||
if ips:
|
||||
ip_table = Table(title="IP Addresses")
|
||||
ip_table.add_column("Address", style="yellow")
|
||||
ip_table.add_column("DNS Name", style="green")
|
||||
ip_table.add_column("Status")
|
||||
|
||||
for ip in ips:
|
||||
ip_table.add_row(
|
||||
str(ip.address),
|
||||
ip.dns_name or "",
|
||||
ip.status.value if hasattr(ip.status, 'value') else str(ip.status)
|
||||
)
|
||||
|
||||
console.print("\n", ip_table)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Main
|
||||
# ============================================================================
|
||||
|
||||
def main(vm_name: Optional[str] = None) -> int:
|
||||
"""
|
||||
Main entry point.
|
||||
|
||||
Args:
|
||||
vm_name: Optional specific VM to query
|
||||
|
||||
Returns:
|
||||
Exit code (0 = success, 1 = error)
|
||||
"""
|
||||
config = NetBoxConfig()
|
||||
|
||||
# Get NetBox client
|
||||
nb = get_netbox_client(config)
|
||||
if not nb:
|
||||
return 1
|
||||
|
||||
try:
|
||||
if vm_name:
|
||||
# Query specific VM
|
||||
details = get_vm_details(nb, vm_name)
|
||||
if details:
|
||||
display_vm_details(details)
|
||||
return 0
|
||||
return 1
|
||||
else:
|
||||
# List all VMs in Matrix cluster
|
||||
vms = get_cluster_vms(nb, cluster_name="Matrix")
|
||||
display_vms_table(vms)
|
||||
return 0
|
||||
|
||||
except KeyboardInterrupt:
|
||||
console.print("\n[yellow]Interrupted by user[/yellow]")
|
||||
return 1
|
||||
except Exception as e:
|
||||
console.print(f"[red]Unexpected error: {e}[/red]")
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="NetBox API client example",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__
|
||||
)
|
||||
parser.add_argument(
|
||||
"--vm",
|
||||
help="Specific VM to query (default: list all)"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
sys.exit(main(vm_name=args.vm))
|
||||
209
skills/python-uv-scripts/patterns/api-clients.md
Normal file
209
skills/python-uv-scripts/patterns/api-clients.md
Normal file
@@ -0,0 +1,209 @@
|
||||
# API Client Patterns
|
||||
|
||||
Patterns for building API clients with uv scripts.
|
||||
|
||||
## Basic GET Request
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import httpx
|
||||
|
||||
response = httpx.get("https://api.github.com/users/octocat")
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
print(f"Name: {data['name']}")
|
||||
```
|
||||
|
||||
## Authenticated Requests
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import httpx
|
||||
import os
|
||||
import sys
|
||||
|
||||
api_token = os.getenv("API_TOKEN")
|
||||
if not api_token:
|
||||
print("Error: API_TOKEN not set", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
headers = {"Authorization": f"Bearer {api_token}"}
|
||||
|
||||
response = httpx.get(
|
||||
"https://api.example.com/data",
|
||||
headers=headers,
|
||||
timeout=10.0
|
||||
)
|
||||
response.raise_for_status()
|
||||
print(response.json())
|
||||
```
|
||||
|
||||
## POST with JSON
|
||||
|
||||
```python
|
||||
import httpx
|
||||
|
||||
data = {
|
||||
"name": "example",
|
||||
"status": "active"
|
||||
}
|
||||
|
||||
response = httpx.post(
|
||||
"https://api.example.com/resources",
|
||||
json=data,
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
timeout=10.0
|
||||
)
|
||||
response.raise_for_status()
|
||||
result = response.json()
|
||||
print(f"Created: {result['id']}")
|
||||
```
|
||||
|
||||
## Query Parameters
|
||||
|
||||
```python
|
||||
import httpx
|
||||
|
||||
params = {
|
||||
"q": "python",
|
||||
"sort": "stars",
|
||||
"order": "desc"
|
||||
}
|
||||
|
||||
response = httpx.get(
|
||||
"https://api.github.com/search/repositories",
|
||||
params=params
|
||||
)
|
||||
response.raise_for_status()
|
||||
repos = response.json()
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import sys
|
||||
|
||||
try:
|
||||
with httpx.Client(timeout=10.0) as client:
|
||||
response = client.get("https://api.example.com/data")
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
except httpx.HTTPStatusError as e:
|
||||
status = e.response.status_code
|
||||
if status == 401:
|
||||
print("Error: Unauthorized - check API key", file=sys.stderr)
|
||||
elif status == 404:
|
||||
print("Error: Resource not found", file=sys.stderr)
|
||||
elif status >= 500:
|
||||
print(f"Error: Server error ({status})", file=sys.stderr)
|
||||
else:
|
||||
print(f"Error: HTTP {status}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
except httpx.RequestError as e:
|
||||
print(f"Error: Request failed - {type(e).__name__}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
## Retry Logic
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# "tenacity>=8.2.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import httpx
|
||||
from tenacity import retry, stop_after_attempt, wait_exponential
|
||||
|
||||
@retry(
|
||||
stop=stop_after_attempt(3),
|
||||
wait=wait_exponential(multiplier=1, min=2, max=10)
|
||||
)
|
||||
def fetch_data(url: str):
|
||||
"""Fetch data with automatic retry."""
|
||||
response = httpx.get(url, timeout=10.0)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
data = fetch_data("https://api.example.com/data")
|
||||
```
|
||||
|
||||
## Pagination
|
||||
|
||||
```python
|
||||
import httpx
|
||||
|
||||
def fetch_all_pages(base_url: str, headers: dict):
|
||||
"""Fetch all pages from paginated API."""
|
||||
all_results = []
|
||||
next_url = base_url
|
||||
|
||||
with httpx.Client(headers=headers, timeout=10.0) as client:
|
||||
while next_url:
|
||||
response = client.get(next_url)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
all_results.extend(data["results"])
|
||||
|
||||
# Get next page URL
|
||||
next_url = data.get("next")
|
||||
|
||||
return all_results
|
||||
```
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import time
|
||||
|
||||
def fetch_with_rate_limit(urls: list[str], requests_per_second: int = 2):
|
||||
"""Fetch URLs respecting rate limit."""
|
||||
delay = 1.0 / requests_per_second
|
||||
results = []
|
||||
|
||||
for url in urls:
|
||||
response = httpx.get(url)
|
||||
response.raise_for_status()
|
||||
results.append(response.json())
|
||||
|
||||
time.sleep(delay)
|
||||
|
||||
return results
|
||||
```
|
||||
|
||||
## Complete Example
|
||||
|
||||
For a complete API client template, see: `assets/templates/api-client.py`
|
||||
|
||||
## Best Practices
|
||||
|
||||
- Always set timeouts (default: 10 seconds)
|
||||
- Use `with httpx.Client()` for multiple requests
|
||||
- Handle specific HTTP status codes (401, 404, 500)
|
||||
- Don't log sensitive data (tokens, responses)
|
||||
- Use environment variables for credentials
|
||||
- Implement retry logic for transient failures
|
||||
- Respect rate limits
|
||||
129
skills/python-uv-scripts/patterns/cli-applications.md
Normal file
129
skills/python-uv-scripts/patterns/cli-applications.md
Normal file
@@ -0,0 +1,129 @@
|
||||
# CLI Application Patterns
|
||||
|
||||
Patterns for building command-line applications with uv scripts.
|
||||
|
||||
## Basic CLI with Typer
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "typer>=0.9.0",
|
||||
# "rich>=13.0.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import typer
|
||||
from rich import print
|
||||
|
||||
app = typer.Typer()
|
||||
|
||||
@app.command()
|
||||
def main(
|
||||
name: str = typer.Argument(..., help="Your name"),
|
||||
greeting: str = typer.Option("Hello", "--greeting", "-g"),
|
||||
):
|
||||
"""Greet someone."""
|
||||
print(f"[green]{greeting}, {name}![/green]")
|
||||
|
||||
if __name__ == "__main__":
|
||||
app()
|
||||
```
|
||||
|
||||
## Multiple Subcommands
|
||||
|
||||
```python
|
||||
import typer
|
||||
|
||||
app = typer.Typer()
|
||||
|
||||
@app.command()
|
||||
def create(name: str):
|
||||
"""Create a new resource."""
|
||||
print(f"Creating: {name}")
|
||||
|
||||
@app.command()
|
||||
def delete(name: str):
|
||||
"""Delete a resource."""
|
||||
print(f"Deleting: {name}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
app()
|
||||
```
|
||||
|
||||
Usage:
|
||||
|
||||
```bash
|
||||
./script.py create foo
|
||||
./script.py delete bar
|
||||
```
|
||||
|
||||
## File Input/Output
|
||||
|
||||
```python
|
||||
import typer
|
||||
from pathlib import Path
|
||||
|
||||
def process_file(
|
||||
input_file: Path = typer.Argument(..., exists=True),
|
||||
output: Path = typer.Option(None, "--output", "-o"),
|
||||
):
|
||||
"""Process a file."""
|
||||
content = input_file.read_text()
|
||||
|
||||
# Process
|
||||
result = content.upper()
|
||||
|
||||
if output:
|
||||
output.write_text(result)
|
||||
print(f"Written to: {output}")
|
||||
else:
|
||||
print(result)
|
||||
```
|
||||
|
||||
## Progress Bars
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "rich>=13.0.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
from rich.progress import track
|
||||
import time
|
||||
|
||||
for item in track(range(100), description="Processing..."):
|
||||
time.sleep(0.01) # Simulate work
|
||||
```
|
||||
|
||||
## Formatted Tables
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "rich>=13.0.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
|
||||
console = Console()
|
||||
|
||||
table = Table(title="Results")
|
||||
table.add_column("Name", style="cyan")
|
||||
table.add_column("Status", style="green")
|
||||
|
||||
table.add_row("Task 1", "✓ Complete")
|
||||
table.add_row("Task 2", "⏳ Pending")
|
||||
|
||||
console.print(table)
|
||||
```
|
||||
|
||||
For complete template, see: `assets/templates/cli-app.py`
|
||||
54
skills/python-uv-scripts/patterns/data-processing.md
Normal file
54
skills/python-uv-scripts/patterns/data-processing.md
Normal file
@@ -0,0 +1,54 @@
|
||||
# Data Processing Patterns
|
||||
|
||||
> **Status**: 🚧 Placeholder - Content in development
|
||||
|
||||
## Overview
|
||||
|
||||
Patterns for data analysis, ETL, and processing using Polars, pandas, and other data libraries in UV single-file
|
||||
scripts.
|
||||
|
||||
## Topics to Cover
|
||||
|
||||
- [ ] Polars patterns (recommended for performance)
|
||||
- [ ] Pandas alternatives
|
||||
- [ ] CSV/Excel processing
|
||||
- [ ] JSON data manipulation
|
||||
- [ ] Data validation and cleaning
|
||||
- [ ] Aggregation and transformation
|
||||
- [ ] Memory-efficient processing
|
||||
|
||||
## Quick Example
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["polars>=0.20.0"]
|
||||
# ///
|
||||
import polars as pl
|
||||
|
||||
def analyze_csv(file_path: str):
|
||||
df = pl.read_csv(file_path)
|
||||
|
||||
# Basic analysis
|
||||
summary = df.describe()
|
||||
print(summary)
|
||||
|
||||
# Filter and aggregate
|
||||
result = (
|
||||
df.filter(pl.col("value") > 100)
|
||||
.groupby("category")
|
||||
.agg(pl.col("value").mean())
|
||||
)
|
||||
print(result)
|
||||
```
|
||||
|
||||
## TODO
|
||||
|
||||
This file will be expanded to include:
|
||||
|
||||
- Complete Polars patterns
|
||||
- Performance optimization techniques
|
||||
- Large file processing strategies
|
||||
- Data validation patterns
|
||||
- Export formats and options
|
||||
198
skills/python-uv-scripts/patterns/error-handling.md
Normal file
198
skills/python-uv-scripts/patterns/error-handling.md
Normal file
@@ -0,0 +1,198 @@
|
||||
# Error Handling Patterns
|
||||
|
||||
Best practices for error handling in uv scripts.
|
||||
|
||||
## Exit Codes
|
||||
|
||||
Always use appropriate exit codes:
|
||||
|
||||
- `0` - Success
|
||||
- `1` - General error
|
||||
- `2` - Invalid usage
|
||||
|
||||
```python
|
||||
import sys
|
||||
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: script.py <input>", file=sys.stderr)
|
||||
sys.exit(2) # Invalid usage
|
||||
|
||||
try:
|
||||
result = process()
|
||||
except Exception as e:
|
||||
print(f"Error: {e}", file=sys.stderr)
|
||||
sys.exit(1) # General error
|
||||
|
||||
# Success
|
||||
sys.exit(0)
|
||||
```
|
||||
|
||||
## Try-Except Pattern
|
||||
|
||||
```python
|
||||
import sys
|
||||
|
||||
def main():
|
||||
try:
|
||||
# Operations that might fail
|
||||
data = fetch_data()
|
||||
result = process(data)
|
||||
save(result)
|
||||
|
||||
except FileNotFoundError as e:
|
||||
print(f"File not found: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
except ValueError as e:
|
||||
print(f"Invalid data: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
except Exception as e:
|
||||
print(f"Unexpected error: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
```
|
||||
|
||||
## HTTP Error Handling
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import httpx
|
||||
import sys
|
||||
|
||||
try:
|
||||
response = httpx.get("https://api.example.com", timeout=10.0)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
except httpx.HTTPStatusError as e:
|
||||
print(f"HTTP {e.response.status_code} error", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
except httpx.RequestError as e:
|
||||
print(f"Request failed: {type(e).__name__}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
except httpx.TimeoutException:
|
||||
print("Request timed out", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
## File Operation Errors
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
def read_config(path: Path):
|
||||
"""Read configuration file with error handling."""
|
||||
try:
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"Config not found: {path}")
|
||||
|
||||
if not path.is_file():
|
||||
raise ValueError(f"Not a file: {path}")
|
||||
|
||||
return path.read_text()
|
||||
|
||||
except PermissionError:
|
||||
print(f"Permission denied: {path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error reading config: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
## Subprocess Errors
|
||||
|
||||
```python
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["command", "arg"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True
|
||||
)
|
||||
print(result.stdout)
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"Command failed with exit code {e.returncode}", file=sys.stderr)
|
||||
print(f"Error output: {e.stderr}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
except FileNotFoundError:
|
||||
print("Command not found", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
## Validation Errors
|
||||
|
||||
```python
|
||||
import sys
|
||||
|
||||
def validate_input(value: str) -> int:
|
||||
"""Validate and convert input."""
|
||||
if not value:
|
||||
print("Error: Empty value", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
try:
|
||||
number = int(value)
|
||||
except ValueError:
|
||||
print(f"Error: Not a number: {value}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
if number < 0:
|
||||
print("Error: Must be non-negative", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
return number
|
||||
```
|
||||
|
||||
## Graceful Cleanup
|
||||
|
||||
```python
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
def main():
|
||||
temp_file = Path("/tmp/data.tmp")
|
||||
|
||||
try:
|
||||
# Create temporary file
|
||||
temp_file.write_text("data")
|
||||
|
||||
# Process
|
||||
result = process(temp_file)
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
finally:
|
||||
# Always cleanup
|
||||
if temp_file.exists():
|
||||
temp_file.unlink()
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
- Always write errors to `stderr`
|
||||
- Use specific exception types
|
||||
- Provide helpful error messages
|
||||
- Use appropriate exit codes
|
||||
- Clean up resources in `finally` blocks
|
||||
- Don't expose secrets in error messages
|
||||
32
skills/python-uv-scripts/patterns/security-patterns.md
Normal file
32
skills/python-uv-scripts/patterns/security-patterns.md
Normal file
@@ -0,0 +1,32 @@
|
||||
# Security Patterns for UV Single-File Scripts
|
||||
|
||||
> **Status**: 🚧 Placeholder - Content in development
|
||||
|
||||
## Overview
|
||||
|
||||
Security patterns and best practices for handling secrets, authentication, and input validation in UV single-file
|
||||
Python scripts.
|
||||
|
||||
## Topics to Cover
|
||||
|
||||
- [ ] Infisical secrets management integration
|
||||
- [ ] Keyring for credential storage
|
||||
- [ ] Environment variable security
|
||||
- [ ] Input validation and sanitization
|
||||
- [ ] API token handling
|
||||
- [ ] SSH key management
|
||||
- [ ] Avoiding hardcoded credentials
|
||||
|
||||
## Quick Reference
|
||||
|
||||
For now, see the main security section in [../reference/security-patterns.md](../reference/security-patterns.md).
|
||||
|
||||
## TODO
|
||||
|
||||
This file will be expanded to include:
|
||||
|
||||
- Complete Infisical integration patterns
|
||||
- Best practices for secret rotation
|
||||
- Multi-environment secret management
|
||||
- Security checklist for scripts
|
||||
- Common security vulnerabilities and fixes
|
||||
61
skills/python-uv-scripts/patterns/system-automation.md
Normal file
61
skills/python-uv-scripts/patterns/system-automation.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# System Automation Patterns
|
||||
|
||||
> **Status**: 🚧 Placeholder - Content in development
|
||||
|
||||
## Overview
|
||||
|
||||
Patterns for system administration, monitoring, and automation using psutil, subprocess, and system libraries in UV
|
||||
single-file scripts.
|
||||
|
||||
## Topics to Cover
|
||||
|
||||
- [ ] psutil for system monitoring
|
||||
- [ ] subprocess for command execution
|
||||
- [ ] File system operations
|
||||
- [ ] Process management
|
||||
- [ ] SSH remote execution
|
||||
- [ ] Cron/scheduled task integration
|
||||
- [ ] Log file analysis
|
||||
|
||||
## Quick Example
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["psutil>=5.9.0", "rich>=13.0.0"]
|
||||
# ///
|
||||
import psutil
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
|
||||
console = Console()
|
||||
|
||||
def show_disk_usage():
|
||||
table = Table(title="Disk Usage")
|
||||
table.add_column("Device", style="cyan")
|
||||
table.add_column("Mount", style="magenta")
|
||||
table.add_column("Used", style="yellow")
|
||||
table.add_column("Free", style="green")
|
||||
|
||||
for partition in psutil.disk_partitions():
|
||||
usage = psutil.disk_usage(partition.mountpoint)
|
||||
table.add_row(
|
||||
partition.device,
|
||||
partition.mountpoint,
|
||||
f"{usage.percent}%",
|
||||
f"{usage.free / (1024**3):.2f} GB"
|
||||
)
|
||||
|
||||
console.print(table)
|
||||
```
|
||||
|
||||
## TODO
|
||||
|
||||
This file will be expanded to include:
|
||||
|
||||
- Complete psutil monitoring patterns
|
||||
- Safe subprocess execution
|
||||
- SSH automation patterns
|
||||
- System health checks
|
||||
- Automated maintenance tasks
|
||||
293
skills/python-uv-scripts/reference/bash-to-python.md
Normal file
293
skills/python-uv-scripts/reference/bash-to-python.md
Normal file
@@ -0,0 +1,293 @@
|
||||
# Converting Bash Scripts to Python uv Scripts
|
||||
|
||||
This document guides the conversion of bash scripts to Python uv scripts.
|
||||
|
||||
## When Conversion Makes Sense
|
||||
|
||||
### ✅ Good Candidates for Conversion
|
||||
|
||||
**Convert when**:
|
||||
|
||||
- Script needs better error handling
|
||||
- Cross-platform compatibility required
|
||||
- Complex data processing needed
|
||||
- API interactions involved
|
||||
- Script will grow in complexity
|
||||
|
||||
**Examples**:
|
||||
|
||||
```bash
|
||||
# Good candidate - API interaction
|
||||
curl -X POST https://api.example.com/data \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-d '{"status": "active"}'
|
||||
|
||||
# Good candidate - Data processing
|
||||
cat data.json | jq '.users[] | select(.active == true)'
|
||||
|
||||
# Good candidate - Complex logic
|
||||
for file in $(find /data -name "*.log"); do
|
||||
count=$(grep -c "ERROR" "$file")
|
||||
if [ $count -gt 100 ]; then
|
||||
# Complex processing
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
### ❌ Keep as Bash When
|
||||
|
||||
**Don't convert when**:
|
||||
|
||||
- Simple file operations (cp, mv, mkdir)
|
||||
- Shell-specific features heavily used (job control, pipes)
|
||||
- System administration tasks
|
||||
- Script is <10 lines and works fine
|
||||
- Team primarily knows bash
|
||||
|
||||
**Examples**:
|
||||
|
||||
```bash
|
||||
# Keep as bash - Simple operations
|
||||
#!/bin/bash
|
||||
mkdir -p /var/log/app
|
||||
cp config.yaml /etc/app/
|
||||
|
||||
# Keep as bash - Shell-specific
|
||||
#!/bin/bash
|
||||
find /data -name "*.tmp" -mtime +7 -delete
|
||||
|
||||
# Keep as bash - System admin
|
||||
#!/bin/bash
|
||||
systemctl restart nginx
|
||||
journalctl -u nginx -f
|
||||
```
|
||||
|
||||
## Common Conversions
|
||||
|
||||
### File Operations
|
||||
|
||||
**Bash**:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
if [ -f "/etc/config.yaml" ]; then
|
||||
cp /etc/config.yaml /backup/
|
||||
fi
|
||||
```
|
||||
|
||||
**Python**:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
|
||||
config = Path("/etc/config.yaml")
|
||||
if config.exists():
|
||||
shutil.copy(config, "/backup/")
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
**Bash**:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
API_URL=${API_URL:-"https://api.example.com"}
|
||||
if [ -z "$API_TOKEN" ]; then
|
||||
echo "Error: API_TOKEN not set" >&2
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
**Python**:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
API_URL = os.getenv("API_URL", "https://api.example.com")
|
||||
API_TOKEN = os.getenv("API_TOKEN")
|
||||
|
||||
if not API_TOKEN:
|
||||
print("Error: API_TOKEN not set", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
### Running Commands
|
||||
|
||||
**Bash**:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
output=$(systemctl status nginx)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Error: nginx not running"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
**Python**:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["systemctl", "status", "nginx"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True
|
||||
)
|
||||
print(result.stdout)
|
||||
except subprocess.CalledProcessError:
|
||||
print("Error: nginx not running", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
### HTTP Requests
|
||||
|
||||
**Bash**:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
response=$(curl -s -X GET https://api.github.com/users/octocat)
|
||||
name=$(echo "$response" | jq -r '.name')
|
||||
echo "Name: $name"
|
||||
```
|
||||
|
||||
**Python**:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import httpx
|
||||
|
||||
response = httpx.get("https://api.github.com/users/octocat")
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
print(f"Name: {data['name']}")
|
||||
```
|
||||
|
||||
### JSON Processing
|
||||
|
||||
**Bash**:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
jq '.users[] | select(.active == true) | .name' data.json
|
||||
```
|
||||
|
||||
**Python**:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
data = json.loads(Path("data.json").read_text())
|
||||
active_names = [
|
||||
user["name"]
|
||||
for user in data["users"]
|
||||
if user.get("active")
|
||||
]
|
||||
print("\n".join(active_names))
|
||||
```
|
||||
|
||||
## Decision Framework
|
||||
|
||||
```text
|
||||
Should I convert this bash script?
|
||||
|
||||
├─ Is it >50 lines? → Consider conversion
|
||||
├─ Does it process JSON/YAML? → Strong candidate
|
||||
├─ Does it make API calls? → Strong candidate
|
||||
├─ Does it have complex logic? → Consider conversion
|
||||
├─ Does it need better error handling? → Consider conversion
|
||||
├─ Is it mostly shell commands? → Keep as bash
|
||||
└─ Is it <10 lines and works? → Keep as bash
|
||||
```
|
||||
|
||||
## Hybrid Approach
|
||||
|
||||
Sometimes the best solution is calling bash from Python:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""
|
||||
Hybrid: Use Python for logic, bash for system commands.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
|
||||
def backup_with_rsync(source: str, dest: str):
|
||||
"""Use rsync (better than reimplementing in Python)."""
|
||||
subprocess.run(
|
||||
["rsync", "-av", "--delete", source, dest],
|
||||
check=True
|
||||
)
|
||||
|
||||
# Python logic here
|
||||
# ...
|
||||
|
||||
# Leverage bash tools where appropriate
|
||||
backup_with_rsync("/data/", "/backup/")
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
**Convert to Python when**:
|
||||
|
||||
- Complex logic or data processing
|
||||
- API interactions
|
||||
- Cross-platform needs
|
||||
- Better error handling required
|
||||
- Will grow in complexity
|
||||
|
||||
**Keep as Bash when**:
|
||||
|
||||
- Simple file operations
|
||||
- System administration
|
||||
- Heavily uses shell features
|
||||
- Works well and won't change
|
||||
- Team expertise is bash
|
||||
|
||||
**Consider Hybrid**:
|
||||
|
||||
- Complex Python logic + system commands
|
||||
- Leverage both Python libraries and shell tools
|
||||
63
skills/python-uv-scripts/reference/dependency-management.md
Normal file
63
skills/python-uv-scripts/reference/dependency-management.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# Dependency Management Reference
|
||||
|
||||
> **Status**: 🚧 Placeholder - Content in development
|
||||
|
||||
## Overview
|
||||
|
||||
Comprehensive guide to managing dependencies in UV single-file scripts using PEP 723 inline metadata.
|
||||
|
||||
## Topics to Cover
|
||||
|
||||
- [ ] Version pinning strategies
|
||||
- [ ] Semantic versioning best practices
|
||||
- [ ] Dependency conflict resolution
|
||||
- [ ] Optional dependencies
|
||||
- [ ] Development vs production dependencies
|
||||
- [ ] Security updates and vulnerability scanning
|
||||
- [ ] Lock file equivalents for scripts
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Version Pinning Strategies
|
||||
|
||||
**Exact pinning** (most restrictive):
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = ["requests==2.31.0"]
|
||||
# ///
|
||||
```
|
||||
|
||||
**Compatible release** (recommended):
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = ["requests~=2.31.0"] # >=2.31.0, <2.32.0
|
||||
# ///
|
||||
```
|
||||
|
||||
**Minimum version**:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = ["requests>=2.31.0"]
|
||||
# ///
|
||||
```
|
||||
|
||||
### Following Repository Standards
|
||||
|
||||
From this repository's `pyproject.toml`, we use:
|
||||
|
||||
- `>=` for minimum versions with flexibility
|
||||
- Specific version ranges for critical dependencies
|
||||
- Regular dependency audits with Renovate
|
||||
|
||||
## TODO
|
||||
|
||||
This file will be expanded to include:
|
||||
|
||||
- Complete version specifier syntax
|
||||
- Dependency resolution strategies
|
||||
- Security scanning integration
|
||||
- Update strategies and automation
|
||||
- Conflict resolution techniques
|
||||
391
skills/python-uv-scripts/reference/pep-723-spec.md
Normal file
391
skills/python-uv-scripts/reference/pep-723-spec.md
Normal file
@@ -0,0 +1,391 @@
|
||||
# PEP 723: Inline Script Metadata Specification
|
||||
|
||||
Complete reference for PEP 723 inline script metadata format used by uv.
|
||||
|
||||
## Overview
|
||||
|
||||
PEP 723 defines a standardized way to embed dependency and configuration metadata directly within Python script files. This eliminates the need for separate `requirements.txt` files and enables self-contained, reproducible scripts.
|
||||
|
||||
## Basic Format
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "package-name>=1.0.0",
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
**Key Requirements:**
|
||||
|
||||
- Must appear as comments (`#`)
|
||||
- Must use `# /// script` as opening marker
|
||||
- Must use `# ///` as closing marker
|
||||
- Must be valid TOML syntax
|
||||
- Recommended placement: After shebang, before module docstring
|
||||
|
||||
## Complete Specification
|
||||
|
||||
### Required Fields
|
||||
|
||||
#### requires-python
|
||||
|
||||
Specifies minimum Python version:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# ///
|
||||
```
|
||||
|
||||
**Formats:**
|
||||
|
||||
```python
|
||||
requires-python = ">=3.11" # Minimum version
|
||||
requires-python = ">=3.11,<3.13" # Version range
|
||||
requires-python = "==3.12" # Exact version
|
||||
```
|
||||
|
||||
#### dependencies
|
||||
|
||||
Lists required packages:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# "rich>=13.0.0",
|
||||
# "typer~=0.9.0",
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
**Version Specifiers:**
|
||||
|
||||
```python
|
||||
"package" # Any version
|
||||
"package>=1.0" # Minimum version
|
||||
"package>=1.0,<2.0" # Version range
|
||||
"package~=1.2.3" # Compatible release (1.2.x)
|
||||
"package==1.2.3" # Exact version
|
||||
```
|
||||
|
||||
### Optional Fields
|
||||
|
||||
#### [tool.uv] Section
|
||||
|
||||
uv-specific configuration:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
#
|
||||
# [tool.uv]
|
||||
# exclude-newer = "2024-10-01T00:00:00Z"
|
||||
# index-url = "https://pypi.org/simple"
|
||||
# ///
|
||||
```
|
||||
|
||||
**Available options:**
|
||||
|
||||
- `exclude-newer`: Only use packages published before this date
|
||||
- `index-url`: Alternative PyPI index
|
||||
- `extra-index-url`: Additional package indexes
|
||||
- `find-links`: Additional package sources
|
||||
- `no-index`: Ignore PyPI entirely
|
||||
|
||||
#### [tool.uv.sources]
|
||||
|
||||
Custom package sources:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = ["my-package"]
|
||||
#
|
||||
# [tool.uv.sources]
|
||||
# my-package = { git = "https://github.com/user/repo", tag = "v1.0" }
|
||||
# ///
|
||||
```
|
||||
|
||||
#### Valid [tool.uv] Fields
|
||||
|
||||
Additional uv-specific configuration (optional):
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
#
|
||||
# [tool.uv]
|
||||
# exclude-newer = "2025-01-01T00:00:00Z" # Reproducibility constraint
|
||||
# ///
|
||||
```
|
||||
|
||||
**Note**: For custom metadata like purpose, team, author, use Python docstrings instead:
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""
|
||||
Purpose: cluster-monitoring
|
||||
Team: infrastructure
|
||||
Author: devops@example.com
|
||||
Created: 2024-10-20
|
||||
"""
|
||||
```
|
||||
|
||||
## Complete Examples
|
||||
|
||||
### Minimal Script
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""Simple script with no dependencies"""
|
||||
|
||||
print("Hello, world!")
|
||||
```
|
||||
|
||||
### Production Script
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script --quiet
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# "rich>=13.0.0",
|
||||
# "typer>=0.9.0",
|
||||
# ]
|
||||
#
|
||||
# [tool.uv]
|
||||
# exclude-newer = "2025-01-01T00:00:00Z"
|
||||
# ///
|
||||
"""
|
||||
API client for Proxmox cluster monitoring
|
||||
|
||||
Purpose: api-client
|
||||
Team: infrastructure
|
||||
Author: devops@spaceships.work
|
||||
|
||||
Usage:
|
||||
check_cluster.py [--node NODE] [--json]
|
||||
"""
|
||||
|
||||
import typer
|
||||
import httpx
|
||||
from rich import print
|
||||
```
|
||||
|
||||
### With Git Dependencies
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "my-internal-lib",
|
||||
# ]
|
||||
#
|
||||
# [tool.uv.sources]
|
||||
# my-internal-lib = { git = "https://github.com/org/lib.git", tag = "v1.2.3" }
|
||||
# ///
|
||||
```
|
||||
|
||||
### With Local Dependencies
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "my-local-package",
|
||||
# ]
|
||||
#
|
||||
# [tool.uv.sources]
|
||||
# my-local-package = { path = "../my-package", editable = true }
|
||||
# ///
|
||||
```
|
||||
|
||||
## Placement Guidelines
|
||||
|
||||
### Correct Placement
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""Module docstring comes after metadata"""
|
||||
|
||||
import sys
|
||||
|
||||
def main():
|
||||
pass
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
```
|
||||
|
||||
### Multiple Metadata Blocks (Invalid)
|
||||
|
||||
```python
|
||||
# ❌ INVALID - Only one metadata block allowed
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# ///
|
||||
|
||||
# /// script
|
||||
# dependencies = ["httpx"]
|
||||
# ///
|
||||
```
|
||||
|
||||
## Validation
|
||||
|
||||
### Valid Metadata
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0", # Comments allowed in arrays
|
||||
# "rich",
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
### Invalid Metadata
|
||||
|
||||
```python
|
||||
# ❌ Missing closing marker
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
|
||||
# ❌ Invalid TOML syntax
|
||||
# /// script
|
||||
# dependencies = httpx # Missing quotes
|
||||
# ///
|
||||
|
||||
# ❌ Not in comments
|
||||
/// script
|
||||
requires-python = ">=3.11"
|
||||
///
|
||||
```
|
||||
|
||||
## Creating Metadata
|
||||
|
||||
### Manual Creation
|
||||
|
||||
Add metadata block manually:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import httpx
|
||||
```
|
||||
|
||||
### Using uv init
|
||||
|
||||
Generate script with metadata:
|
||||
|
||||
```bash
|
||||
uv init --script my_script.py --python 3.11
|
||||
```
|
||||
|
||||
### Using uv add
|
||||
|
||||
Add dependencies to existing script:
|
||||
|
||||
```bash
|
||||
uv add --script my_script.py httpx rich typer
|
||||
```
|
||||
|
||||
## Validation Tools
|
||||
|
||||
### Check Metadata Validity
|
||||
|
||||
```bash
|
||||
# Parse metadata
|
||||
uv run --script my_script.py --dry-run
|
||||
|
||||
# Validate with custom tool
|
||||
python tools/validate_script.py my_script.py
|
||||
```
|
||||
|
||||
### Extract Metadata
|
||||
|
||||
```python
|
||||
import re
|
||||
import tomllib
|
||||
|
||||
def extract_metadata(script_path: str) -> dict:
|
||||
"""Extract PEP 723 metadata from script"""
|
||||
with open(script_path) as f:
|
||||
content = f.read()
|
||||
|
||||
# Find metadata block
|
||||
pattern = r'# /// script\n((?:# .*\n)+)# ///'
|
||||
match = re.search(pattern, content)
|
||||
|
||||
if not match:
|
||||
return {}
|
||||
|
||||
# Parse TOML (remove leading # from each line)
|
||||
toml_lines = match.group(1).split('\n')
|
||||
toml_content = '\n'.join(line[2:] for line in toml_lines if line.startswith('# '))
|
||||
|
||||
return tomllib.loads(toml_content)
|
||||
```
|
||||
|
||||
## Compatibility
|
||||
|
||||
### PEP 723 Support
|
||||
|
||||
- ✅ uv (native support)
|
||||
- ✅ pip (via `pip-run`)
|
||||
- ✅ pipx (v1.4.0+)
|
||||
- ⚠️ Other tools (check documentation)
|
||||
|
||||
### Fallback for Non-Supporting Tools
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["httpx"]
|
||||
# ///
|
||||
"""
|
||||
Fallback installation for non-PEP-723 tools:
|
||||
pip install httpx
|
||||
python script.py
|
||||
"""
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always include requires-python** - Prevents compatibility issues
|
||||
2. **Pin major versions** - `>=X.Y.Z` for stability
|
||||
3. **Add metadata section** - Document purpose and ownership
|
||||
4. **Keep dependencies minimal** - Only what's necessary
|
||||
5. **Document fallbacks** - Help non-uv users
|
||||
6. **Validate syntax** - Use validation tools
|
||||
7. **Version consistently** - Match project conventions
|
||||
|
||||
## References
|
||||
|
||||
- [PEP 723 Specification](https://peps.python.org/pep-0723/)
|
||||
- [uv Documentation](https://docs.astral.sh/uv/)
|
||||
- [TOML Specification](https://toml.io/)
|
||||
- [Python Version Specifiers](https://peps.python.org/pep-0440/)
|
||||
523
skills/python-uv-scripts/reference/security-patterns.md
Normal file
523
skills/python-uv-scripts/reference/security-patterns.md
Normal file
@@ -0,0 +1,523 @@
|
||||
# Security Patterns for uv Scripts
|
||||
|
||||
Security best practices for Python single-file scripts following Virgo-Core patterns.
|
||||
|
||||
## Never Hardcode Secrets
|
||||
|
||||
### ❌ Anti-Pattern: Hardcoded Secrets
|
||||
|
||||
```python
|
||||
# NEVER DO THIS
|
||||
API_KEY = "sk_live_1234567890abcdef"
|
||||
DATABASE_URL = "postgresql://user:password@localhost/db"
|
||||
PROXMOX_PASSWORD = "admin123"
|
||||
```
|
||||
|
||||
**Risks:**
|
||||
|
||||
- Secrets exposed in version control
|
||||
- No audit trail
|
||||
- Difficult to rotate
|
||||
- Same credentials across environments
|
||||
|
||||
### ✅ Pattern 1: Environment Variables
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
def get_secret(name: str, required: bool = True) -> str:
|
||||
"""Get secret from environment"""
|
||||
value = os.getenv(name)
|
||||
|
||||
if required and not value:
|
||||
print(f"Error: {name} environment variable not set", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
return value
|
||||
|
||||
# Usage
|
||||
PROXMOX_PASSWORD = get_secret("PROXMOX_PASSWORD")
|
||||
API_URL = get_secret("PROXMOX_API_URL")
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
|
||||
```bash
|
||||
export PROXMOX_PASSWORD="$(cat ~/.secrets/proxmox_pass)"
|
||||
./script.py
|
||||
```
|
||||
|
||||
### ✅ Pattern 2: Keyring Integration
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "keyring>=24.0.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import keyring
|
||||
import sys
|
||||
|
||||
def get_password(service: str, username: str) -> str:
|
||||
"""Get password from system keyring"""
|
||||
password = keyring.get_password(service, username)
|
||||
|
||||
if not password:
|
||||
print(f"Error: No password found for {username}@{service}", file=sys.stderr)
|
||||
print(f"Set with: keyring set {service} {username}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
return password
|
||||
|
||||
# Usage
|
||||
proxmox_password = get_password("proxmox", "terraform@pam")
|
||||
```
|
||||
|
||||
**Setup:**
|
||||
|
||||
```bash
|
||||
# Store password in system keyring
|
||||
keyring set proxmox terraform@pam
|
||||
# Prompts for password, stores securely
|
||||
|
||||
# Run script (no password in environment)
|
||||
./script.py
|
||||
```
|
||||
|
||||
### ✅ Pattern 3: Infisical Integration (Repository Standard)
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "infisical-python>=2.3.3",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
from infisical import InfisicalClient
|
||||
import os
|
||||
import sys
|
||||
|
||||
def get_infisical_secret(
|
||||
secret_name: str,
|
||||
project_id: str,
|
||||
environment: str = "prod",
|
||||
path: str = "/"
|
||||
) -> str:
|
||||
"""Get secret from Infisical vault"""
|
||||
try:
|
||||
client = InfisicalClient()
|
||||
secret = client.get_secret(
|
||||
secret_name=secret_name,
|
||||
project_id=project_id,
|
||||
environment=environment,
|
||||
path=path
|
||||
)
|
||||
return secret.secret_value
|
||||
except Exception as e:
|
||||
print(f"Error retrieving secret {secret_name}: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Usage (following Virgo-Core pattern)
|
||||
PROXMOX_PASSWORD = get_infisical_secret(
|
||||
secret_name="PROXMOX_PASSWORD",
|
||||
project_id="7b832220-24c0-45bc-a5f1-ce9794a31259",
|
||||
environment="prod",
|
||||
path="/matrix"
|
||||
)
|
||||
```
|
||||
|
||||
**Setup:**
|
||||
|
||||
```bash
|
||||
# Authenticate with Infisical
|
||||
infisical login
|
||||
|
||||
# Run script (secrets fetched automatically)
|
||||
./script.py
|
||||
```
|
||||
|
||||
## Input Validation
|
||||
|
||||
### ❌ Anti-Pattern: No Validation
|
||||
|
||||
```python
|
||||
import subprocess
|
||||
|
||||
# DANGEROUS - Command injection risk
|
||||
def ping_host(hostname: str):
|
||||
subprocess.run(f"ping -c 3 {hostname}", shell=True)
|
||||
|
||||
# User provides: "localhost; rm -rf /"
|
||||
ping_host(sys.argv[1]) # 💥 System destroyed
|
||||
```
|
||||
|
||||
### ✅ Pattern: Validate All Inputs
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
|
||||
import re
|
||||
import sys
|
||||
import subprocess
|
||||
|
||||
def validate_hostname(hostname: str) -> bool:
|
||||
"""Validate hostname format"""
|
||||
# Only allow alphanumeric, dots, hyphens
|
||||
pattern = r'^[a-zA-Z0-9][a-zA-Z0-9.-]*[a-zA-Z0-9]$'
|
||||
if not re.match(pattern, hostname):
|
||||
return False
|
||||
|
||||
# Max length check
|
||||
if len(hostname) > 253:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def ping_host(hostname: str):
|
||||
"""Safely ping a host"""
|
||||
if not validate_hostname(hostname):
|
||||
print(f"Error: Invalid hostname: {hostname}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Use list form (no shell injection)
|
||||
result = subprocess.run(
|
||||
["ping", "-c", "3", hostname],
|
||||
capture_output=True,
|
||||
text=True
|
||||
)
|
||||
|
||||
return result.returncode == 0
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: ping_host.py <hostname>", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
ping_host(sys.argv[1])
|
||||
```
|
||||
|
||||
### Input Validation Patterns
|
||||
|
||||
```python
|
||||
import re
|
||||
from ipaddress import IPv4Address, AddressValueError
|
||||
|
||||
def validate_ip(ip: str) -> bool:
|
||||
"""Validate IPv4 address"""
|
||||
try:
|
||||
IPv4Address(ip)
|
||||
return True
|
||||
except AddressValueError:
|
||||
return False
|
||||
|
||||
def validate_port(port: int) -> bool:
|
||||
"""Validate TCP/UDP port number"""
|
||||
return 1 <= port <= 65535
|
||||
|
||||
def validate_vmid(vmid: int) -> bool:
|
||||
"""Validate Proxmox VMID (100-999999999)"""
|
||||
return 100 <= vmid <= 999999999
|
||||
|
||||
def validate_path(path: str) -> bool:
|
||||
"""Validate file path (no directory traversal)"""
|
||||
# Reject paths with ../
|
||||
if ".." in path:
|
||||
return False
|
||||
|
||||
# Reject absolute paths
|
||||
if path.startswith("/"):
|
||||
return False
|
||||
|
||||
return True
|
||||
```
|
||||
|
||||
## Dependency Security
|
||||
|
||||
### Pin Dependencies
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0", # ✅ Minimum version pinned
|
||||
# "rich>=13.0.0", # ✅ Known good version
|
||||
# ]
|
||||
# ///
|
||||
```
|
||||
|
||||
**Why pin?**
|
||||
|
||||
- Prevents automatic upgrades with vulnerabilities
|
||||
- Ensures reproducible execution
|
||||
- Allows controlled dependency updates
|
||||
|
||||
### Exclude Recent Packages
|
||||
|
||||
```python
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["httpx>=0.27.0"]
|
||||
#
|
||||
# [tool.uv]
|
||||
# exclude-newer = "2024-10-01T00:00:00Z" # Only packages before this date
|
||||
# ///
|
||||
```
|
||||
|
||||
**Use cases:**
|
||||
|
||||
- Prevent supply chain attacks from compromised packages
|
||||
- Freeze dependencies at known-good state
|
||||
- Reproducible builds in CI/CD
|
||||
|
||||
### Check for Vulnerabilities
|
||||
|
||||
```bash
|
||||
# Use safety or pip-audit
|
||||
uv pip install safety
|
||||
safety check --json
|
||||
|
||||
# Or use built-in tools
|
||||
uv pip list --format json | jq '.[] | select(.name == "httpx")'
|
||||
```
|
||||
|
||||
## File Operations Security
|
||||
|
||||
### ❌ Anti-Pattern: Unsafe File Access
|
||||
|
||||
```python
|
||||
# DANGEROUS - Path traversal vulnerability
|
||||
def read_log(filename: str):
|
||||
with open(f"/var/log/{filename}") as f:
|
||||
return f.read()
|
||||
|
||||
# User provides: "../../../etc/passwd"
|
||||
read_log(sys.argv[1]) # 💥 Reads /etc/passwd
|
||||
```
|
||||
|
||||
### ✅ Pattern: Safe File Operations
|
||||
|
||||
```python
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
def safe_read_log(filename: str, log_dir: str = "/var/log") -> str:
|
||||
"""Safely read log file"""
|
||||
# Resolve to absolute paths
|
||||
log_dir_path = Path(log_dir).resolve()
|
||||
file_path = (log_dir_path / filename).resolve()
|
||||
|
||||
# Ensure file is within log directory
|
||||
try:
|
||||
file_path.relative_to(log_dir_path)
|
||||
except ValueError:
|
||||
print(f"Error: Path traversal detected: {filename}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Check file exists and is readable
|
||||
if not file_path.exists():
|
||||
print(f"Error: File not found: {filename}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if not file_path.is_file():
|
||||
print(f"Error: Not a file: {filename}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Read with size limit
|
||||
MAX_SIZE = 10 * 1024 * 1024 # 10MB
|
||||
if file_path.stat().st_size > MAX_SIZE:
|
||||
print(f"Error: File too large: {filename}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
with open(file_path) as f:
|
||||
return f.read()
|
||||
```
|
||||
|
||||
## Command Execution Security
|
||||
|
||||
### ❌ Anti-Pattern: Shell Injection
|
||||
|
||||
```python
|
||||
# DANGEROUS
|
||||
import subprocess
|
||||
|
||||
def list_directory(path: str):
|
||||
subprocess.run(f"ls -la {path}", shell=True)
|
||||
|
||||
# User provides: "; rm -rf /"
|
||||
list_directory(sys.argv[1]) # 💥 Disaster
|
||||
```
|
||||
|
||||
### ✅ Pattern: Safe Command Execution
|
||||
|
||||
```python
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
def list_directory(path: str):
|
||||
"""Safely list directory contents"""
|
||||
# Validate path first
|
||||
if not validate_path(path):
|
||||
print(f"Error: Invalid path: {path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Use list form (no shell)
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["ls", "-la", path],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
timeout=5 # Prevent hanging
|
||||
)
|
||||
print(result.stdout)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"Error: {e.stderr}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
except subprocess.TimeoutExpired:
|
||||
print("Error: Command timed out", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
## Logging and Audit
|
||||
|
||||
### Secure Logging
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "structlog>=24.0.0",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import structlog
|
||||
import sys
|
||||
|
||||
# Configure structured logging
|
||||
structlog.configure(
|
||||
processors=[
|
||||
structlog.processors.add_log_level,
|
||||
structlog.processors.TimeStamper(fmt="iso"),
|
||||
structlog.processors.JSONRenderer(),
|
||||
],
|
||||
logger_factory=structlog.PrintLoggerFactory(file=sys.stderr),
|
||||
)
|
||||
|
||||
log = structlog.get_logger()
|
||||
|
||||
def process_data(user_id: int, action: str):
|
||||
"""Process user action with audit logging"""
|
||||
log.info(
|
||||
"user_action",
|
||||
user_id=user_id,
|
||||
action=action,
|
||||
# Don't log sensitive data!
|
||||
)
|
||||
|
||||
# ... processing logic ...
|
||||
|
||||
log.info(
|
||||
"action_completed",
|
||||
user_id=user_id,
|
||||
action=action,
|
||||
status="success"
|
||||
)
|
||||
```
|
||||
|
||||
### Never Log Secrets
|
||||
|
||||
```python
|
||||
# ❌ BAD - Logs password
|
||||
log.info(f"Connecting with password: {password}")
|
||||
|
||||
# ✅ GOOD - No sensitive data
|
||||
log.info("Connecting to API", endpoint=api_url)
|
||||
|
||||
# ✅ GOOD - Masked credentials
|
||||
log.info("Authentication successful", user=username)
|
||||
```
|
||||
|
||||
## Network Security
|
||||
|
||||
### HTTPS Only
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# dependencies = ["httpx>=0.27.0"]
|
||||
# ///
|
||||
|
||||
import httpx
|
||||
import sys
|
||||
|
||||
def fetch_api(url: str) -> dict:
|
||||
"""Fetch data from API (HTTPS only)"""
|
||||
# Validate HTTPS
|
||||
if not url.startswith("https://"):
|
||||
print("Error: Only HTTPS URLs allowed", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
with httpx.Client(verify=True) as client: # Verify SSL certs
|
||||
response = client.get(url, timeout=10.0)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
```
|
||||
|
||||
### Certificate Verification
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import certifi
|
||||
|
||||
# Use trusted CA bundle
|
||||
client = httpx.Client(verify=certifi.where())
|
||||
|
||||
# For internal CAs, specify cert path
|
||||
client = httpx.Client(verify="/path/to/internal-ca.pem")
|
||||
|
||||
# ONLY disable for development (never in production)
|
||||
if os.getenv("DEVELOPMENT") == "true":
|
||||
client = httpx.Client(verify=False)
|
||||
```
|
||||
|
||||
## Security Checklist
|
||||
|
||||
Before deploying a script:
|
||||
|
||||
- [ ] No hardcoded secrets
|
||||
- [ ] Secrets from environment/keyring/Infisical
|
||||
- [ ] All inputs validated
|
||||
- [ ] No shell=True in subprocess
|
||||
- [ ] File paths checked for traversal
|
||||
- [ ] Dependencies pinned
|
||||
- [ ] No sensitive data in logs
|
||||
- [ ] HTTPS for network requests
|
||||
- [ ] Certificate verification enabled
|
||||
- [ ] Timeouts on external calls
|
||||
- [ ] Error messages don't leak information
|
||||
- [ ] Principle of least privilege applied
|
||||
|
||||
## References
|
||||
|
||||
- [OWASP Top 10](https://owasp.org/www-project-top-ten/)
|
||||
- [Infisical Python SDK](https://infisical.com/docs/sdks/languages/python)
|
||||
- [Python Security Best Practices](https://python.readthedocs.io/en/latest/library/security_warnings.html)
|
||||
- Virgo-Core Infisical integration: [../../ansible/tasks/infisical-secret-lookup.yml](../../ansible/tasks/infisical-secret-lookup.yml)
|
||||
495
skills/python-uv-scripts/tools/convert_to_uv.py
Executable file
495
skills/python-uv-scripts/tools/convert_to_uv.py
Executable file
@@ -0,0 +1,495 @@
|
||||
#!/usr/bin/env -S uv run --script --quiet
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = [
|
||||
# "httpx>=0.27.0",
|
||||
# ]
|
||||
# ///
|
||||
"""
|
||||
Convert Python scripts to uv single-file format (PEP 723)
|
||||
|
||||
Purpose: script-conversion-automation
|
||||
Team: devops
|
||||
Author: devops@spaceships.work
|
||||
|
||||
Converts existing Python scripts to use inline dependency management.
|
||||
Reads dependencies from requirements.txt or detects from imports.
|
||||
|
||||
Usage:
|
||||
convert_to_uv.py <script.py>
|
||||
convert_to_uv.py <script.py> --requirements requirements.txt
|
||||
convert_to_uv.py <script.py> --detect-imports
|
||||
convert_to_uv.py <script.py> --dry-run
|
||||
|
||||
Examples:
|
||||
# Convert script with requirements.txt in same directory
|
||||
convert_to_uv.py my_script.py
|
||||
|
||||
# Convert script with specific requirements file
|
||||
convert_to_uv.py my_script.py --requirements ../requirements.txt
|
||||
|
||||
# Detect dependencies from imports (basic detection)
|
||||
convert_to_uv.py my_script.py --detect-imports
|
||||
|
||||
# Preview conversion without creating file
|
||||
convert_to_uv.py my_script.py --dry-run
|
||||
|
||||
# Specify output filename
|
||||
convert_to_uv.py my_script.py --output my_script_new.py
|
||||
"""
|
||||
|
||||
import ast
|
||||
import re
|
||||
import sys
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
|
||||
# Common import name -> PyPI package name mappings
|
||||
IMPORT_TO_PACKAGE = {
|
||||
'cv2': 'opencv-python',
|
||||
'PIL': 'Pillow',
|
||||
'yaml': 'PyYAML',
|
||||
'bs4': 'beautifulsoup4',
|
||||
'sklearn': 'scikit-learn',
|
||||
'dotenv': 'python-dotenv',
|
||||
'claude_agent_sdk': 'claude-agent-sdk',
|
||||
}
|
||||
|
||||
|
||||
def get_pypi_latest_version(package_name: str) -> str | None:
|
||||
"""
|
||||
Query PyPI API for latest version of package.
|
||||
|
||||
Returns version string like "1.2.3" or None if not found.
|
||||
"""
|
||||
try:
|
||||
import httpx
|
||||
|
||||
url = f"https://pypi.org/pypi/{package_name}/json"
|
||||
with httpx.Client(timeout=5.0) as client:
|
||||
response = client.get(url)
|
||||
if response.status_code == 200:
|
||||
data = response.json()
|
||||
return data['info']['version']
|
||||
except Exception:
|
||||
# Network error, package not found, etc. - fail silently
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def find_version_in_project(package_name: str, script_path: Path) -> str | None:
|
||||
"""
|
||||
Look for version constraint in project's pyproject.toml.
|
||||
|
||||
Searches up directory tree from script location.
|
||||
"""
|
||||
current = script_path.parent
|
||||
|
||||
# Search up to 3 levels for pyproject.toml
|
||||
for _ in range(3):
|
||||
pyproject_path = current / "pyproject.toml"
|
||||
if pyproject_path.exists():
|
||||
try:
|
||||
content = pyproject_path.read_text(encoding='utf-8')
|
||||
data = tomllib.loads(content)
|
||||
|
||||
# Check [project.dependencies]
|
||||
deps = data.get('project', {}).get('dependencies', [])
|
||||
for dep in deps:
|
||||
if isinstance(dep, str) and dep.startswith(package_name):
|
||||
# Extract version constraint
|
||||
# e.g., "package>=1.0.0" -> ">=1.0.0"
|
||||
version_part = dep[len(package_name):].strip()
|
||||
if version_part:
|
||||
return version_part
|
||||
|
||||
# Check [tool.uv.sources] or other sections if needed
|
||||
# (could expand this to check dev-dependencies, etc.)
|
||||
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Move up one directory
|
||||
parent = current.parent
|
||||
if parent == current: # Reached filesystem root
|
||||
break
|
||||
current = parent
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def normalize_package_name(import_name: str, use_pypi: bool = True) -> str:
|
||||
"""
|
||||
Normalize import name to PyPI package name.
|
||||
|
||||
Strategy:
|
||||
1. Check known mappings first (fast)
|
||||
2. Try hyphen normalization (common pattern: my_package → my-package)
|
||||
3. Validate with PyPI if enabled
|
||||
4. Fall back to original
|
||||
|
||||
Returns: Normalized package name
|
||||
"""
|
||||
# Known mapping takes precedence
|
||||
if import_name in IMPORT_TO_PACKAGE:
|
||||
return IMPORT_TO_PACKAGE[import_name]
|
||||
|
||||
# Try hyphen normalization (most common pattern)
|
||||
if '_' in import_name:
|
||||
hyphenated = import_name.replace('_', '-')
|
||||
|
||||
# Validate with PyPI if enabled
|
||||
if use_pypi and get_pypi_latest_version(hyphenated):
|
||||
return hyphenated
|
||||
|
||||
# Without PyPI, trust the normalization (common convention)
|
||||
if not use_pypi:
|
||||
return hyphenated
|
||||
|
||||
# Fall back to original
|
||||
return import_name
|
||||
|
||||
|
||||
def resolve_package_version(
|
||||
import_name: str,
|
||||
script_path: Path,
|
||||
use_pypi: bool = True
|
||||
) -> str:
|
||||
"""
|
||||
Resolve package name and version constraint.
|
||||
|
||||
Returns: "package>=X.Y.Z" format
|
||||
|
||||
Strategy:
|
||||
1. Normalize import name to package name (handles underscore → hyphen)
|
||||
2. Check project's pyproject.toml for version
|
||||
3. Query PyPI for latest version
|
||||
4. Fall back to unversioned package name
|
||||
"""
|
||||
# Normalize import name to package name
|
||||
package_name = normalize_package_name(import_name, use_pypi)
|
||||
|
||||
# Try to find version in project
|
||||
version = find_version_in_project(package_name, script_path)
|
||||
if version:
|
||||
return f"{package_name}{version}"
|
||||
|
||||
# Try PyPI if enabled
|
||||
if use_pypi:
|
||||
latest = get_pypi_latest_version(package_name)
|
||||
if latest:
|
||||
# Use minimum version constraint with latest
|
||||
return f"{package_name}>={latest}"
|
||||
|
||||
# Fall back to unversioned (let uv resolve)
|
||||
return package_name
|
||||
|
||||
|
||||
def has_pep723_metadata(content: str) -> bool:
|
||||
"""Check if script already has PEP 723 metadata"""
|
||||
pattern = r'# /// script\r?\n((?:#.*(?:\r?\n|(?=\r?\n?#\s*///)))+)(?:\r?\n)?#\s*///'
|
||||
return bool(re.search(pattern, content, re.MULTILINE))
|
||||
|
||||
|
||||
def read_requirements_file(req_path: Path) -> list[str]:
|
||||
"""Read dependencies from requirements.txt"""
|
||||
if not req_path.exists():
|
||||
return []
|
||||
|
||||
try:
|
||||
content = req_path.read_text(encoding='utf-8')
|
||||
deps = []
|
||||
for line in content.splitlines():
|
||||
line = line.strip()
|
||||
# Skip empty lines and comments
|
||||
if not line or line.startswith('#'):
|
||||
continue
|
||||
# Skip -e and --editable
|
||||
if line.startswith('-e') or line.startswith('--editable'):
|
||||
continue
|
||||
# Skip -r and --requirement
|
||||
if line.startswith('-r') or line.startswith('--requirement'):
|
||||
continue
|
||||
deps.append(line)
|
||||
return deps
|
||||
except (UnicodeDecodeError, OSError) as e:
|
||||
print(f"Warning: Could not read {req_path}: {e}", file=sys.stderr)
|
||||
return []
|
||||
|
||||
|
||||
def detect_imports(content: str) -> list[str]:
|
||||
"""
|
||||
Detect third-party imports from script (basic detection).
|
||||
|
||||
Returns import names, not package names (caller should map these).
|
||||
"""
|
||||
imports = set()
|
||||
|
||||
try:
|
||||
tree = ast.parse(content)
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.Import):
|
||||
for alias in node.names:
|
||||
# Get base module name
|
||||
base = alias.name.split('.')[0]
|
||||
imports.add(base)
|
||||
elif isinstance(node, ast.ImportFrom):
|
||||
if node.module:
|
||||
base = node.module.split('.')[0]
|
||||
imports.add(base)
|
||||
except SyntaxError as e:
|
||||
print(f"Warning: Could not parse script for imports: {e}", file=sys.stderr)
|
||||
return []
|
||||
|
||||
# Filter out standard library modules (basic filter)
|
||||
stdlib_modules = {
|
||||
'abc', 'argparse', 'ast', 'asyncio', 'base64', 'collections', 'contextlib',
|
||||
'copy', 'csv', 'dataclasses', 'datetime', 'decimal', 'email', 'enum', 'functools',
|
||||
'glob', 'hashlib', 'http', 'inspect', 'io', 'itertools', 'json', 'logging',
|
||||
'math', 'multiprocessing', 'operator', 'os', 'pathlib', 'pickle', 'platform',
|
||||
'pprint', 'queue', 're', 'secrets', 'shutil', 'socket', 'sqlite3', 'ssl',
|
||||
'string', 'subprocess', 'sys', 'tempfile', 'threading', 'time', 'tomllib',
|
||||
'traceback', 'typing', 'unittest', 'urllib', 'uuid', 'warnings', 'weakref',
|
||||
'xml', 'zipfile', 'zoneinfo'
|
||||
}
|
||||
|
||||
third_party = [imp for imp in imports if imp not in stdlib_modules]
|
||||
return sorted(third_party)
|
||||
|
||||
|
||||
def generate_header(
|
||||
dependencies: list[str],
|
||||
python_version: str = ">=3.11",
|
||||
quiet: bool = False
|
||||
) -> str:
|
||||
"""Generate PEP 723 header with shebang"""
|
||||
shebang = "#!/usr/bin/env -S uv run --script"
|
||||
if quiet:
|
||||
shebang += " --quiet"
|
||||
|
||||
# Format dependencies for TOML array
|
||||
if dependencies:
|
||||
deps_str = ",\n# ".join(f'"{dep}"' for dep in dependencies)
|
||||
deps_section = f"# dependencies = [\n# {deps_str},\n# ]"
|
||||
else:
|
||||
deps_section = "# dependencies = []"
|
||||
|
||||
header = f"""{shebang}
|
||||
# /// script
|
||||
# requires-python = "{python_version}"
|
||||
{deps_section}
|
||||
# ///
|
||||
"""
|
||||
return header
|
||||
|
||||
|
||||
def convert_script(
|
||||
script_path: Path,
|
||||
output_path: Path | None = None,
|
||||
requirements_path: Path | None = None,
|
||||
detect_imports_flag: bool = False,
|
||||
dry_run: bool = False,
|
||||
python_version: str = ">=3.11",
|
||||
quiet_mode: bool = False,
|
||||
use_pypi: bool = True
|
||||
) -> bool:
|
||||
"""
|
||||
Convert script to uv format.
|
||||
|
||||
Returns True if successful, False otherwise.
|
||||
"""
|
||||
# Read original script
|
||||
try:
|
||||
content = script_path.read_text(encoding='utf-8')
|
||||
except (FileNotFoundError, PermissionError, OSError, UnicodeDecodeError) as e:
|
||||
print(f"Error: Cannot read {script_path}: {e}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
# Check if already has metadata
|
||||
if has_pep723_metadata(content):
|
||||
print(f"Error: {script_path} already has PEP 723 metadata", file=sys.stderr)
|
||||
print(" Use validate_script.py to check the existing metadata", file=sys.stderr)
|
||||
return False
|
||||
|
||||
# Determine dependencies
|
||||
dependencies = []
|
||||
|
||||
if requirements_path:
|
||||
# Use specified requirements file
|
||||
dependencies = read_requirements_file(requirements_path)
|
||||
if dependencies:
|
||||
print(f"Found {len(dependencies)} dependencies in {requirements_path}")
|
||||
else:
|
||||
# Look for requirements.txt in same directory
|
||||
default_req = script_path.parent / "requirements.txt"
|
||||
if default_req.exists():
|
||||
dependencies = read_requirements_file(default_req)
|
||||
if dependencies:
|
||||
print(f"Found {len(dependencies)} dependencies in {default_req}")
|
||||
|
||||
# Optionally detect imports
|
||||
if detect_imports_flag:
|
||||
detected = detect_imports(content)
|
||||
if detected:
|
||||
print(f"Detected imports: {', '.join(detected)}")
|
||||
print("Resolving versions...")
|
||||
|
||||
# If no dependencies from requirements, use detected
|
||||
if not dependencies:
|
||||
# Use smart version resolution
|
||||
resolved = []
|
||||
for imp in detected:
|
||||
# Normalize package name first
|
||||
normalized_pkg = normalize_package_name(imp, use_pypi)
|
||||
|
||||
# Then resolve version
|
||||
dep = resolve_package_version(imp, script_path, use_pypi=use_pypi)
|
||||
resolved.append(dep)
|
||||
|
||||
# Show what was resolved
|
||||
if imp in IMPORT_TO_PACKAGE:
|
||||
print(f" - Mapped '{imp}' → '{IMPORT_TO_PACKAGE[imp]}' (known mapping)")
|
||||
elif imp != normalized_pkg:
|
||||
print(f" - Normalized '{imp}' → '{normalized_pkg}' (auto-detected)")
|
||||
|
||||
if '>=' in dep:
|
||||
version = dep.split('>=')[1]
|
||||
source = "from project" if find_version_in_project(dep.split('>=')[0], script_path) else "from PyPI"
|
||||
print(f" - Resolved version: {version} {source}")
|
||||
else:
|
||||
print(f" - Using package: {dep} (uv will resolve version)")
|
||||
|
||||
dependencies = resolved
|
||||
|
||||
# Generate header
|
||||
header = generate_header(dependencies, python_version, quiet_mode)
|
||||
|
||||
# Remove old shebang if present
|
||||
lines = content.split('\n')
|
||||
if lines and lines[0].startswith('#!'):
|
||||
# Skip old shebang
|
||||
content_without_shebang = '\n'.join(lines[1:])
|
||||
else:
|
||||
content_without_shebang = content
|
||||
|
||||
# Combine header and content
|
||||
new_content = header + content_without_shebang
|
||||
|
||||
# Determine output path
|
||||
if output_path is None:
|
||||
# Default: add _uv before extension
|
||||
stem = script_path.stem
|
||||
suffix = script_path.suffix
|
||||
output_path = script_path.parent / f"{stem}_uv{suffix}"
|
||||
|
||||
# Dry run - just print
|
||||
if dry_run:
|
||||
print("\n" + "=" * 60)
|
||||
print("DRY RUN - Preview of converted script:")
|
||||
print("=" * 60)
|
||||
print(new_content[:500]) # Show first 500 chars
|
||||
if len(new_content) > 500:
|
||||
print(f"\n... ({len(new_content) - 500} more characters)")
|
||||
print("=" * 60)
|
||||
print(f"Would create: {output_path}")
|
||||
return True
|
||||
|
||||
# Write output
|
||||
try:
|
||||
output_path.write_text(new_content, encoding='utf-8')
|
||||
print(f"✓ Created: {output_path}")
|
||||
|
||||
# Make executable
|
||||
import stat
|
||||
current_permissions = output_path.stat().st_mode
|
||||
output_path.chmod(current_permissions | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
|
||||
print("✓ Made executable")
|
||||
|
||||
# Print next steps
|
||||
print("\nNext steps:")
|
||||
print(f" 1. Review dependencies in {output_path}")
|
||||
print(" 2. Add version constraints if needed")
|
||||
print(f" 3. Test: {output_path}")
|
||||
print(f" 4. Validate: validate_script.py {output_path}")
|
||||
|
||||
return True
|
||||
except (PermissionError, OSError) as e:
|
||||
print(f"Error: Cannot write {output_path}: {e}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point"""
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Convert Python scripts to uv single-file format",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__
|
||||
)
|
||||
parser.add_argument('script', help='Python script to convert')
|
||||
parser.add_argument(
|
||||
'--requirements', '-r',
|
||||
help='Path to requirements.txt (default: look in same directory)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--detect-imports', '-d',
|
||||
action='store_true',
|
||||
help='Detect dependencies from imports (basic detection)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--output', '-o',
|
||||
help='Output filename (default: <script>_uv.py)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--python-version', '-p',
|
||||
default='>=3.11',
|
||||
help='Python version constraint (default: >=3.11)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--quiet', '-q',
|
||||
action='store_true',
|
||||
help='Add --quiet flag to shebang'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--dry-run', '-n',
|
||||
action='store_true',
|
||||
help='Preview conversion without creating file'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--no-pypi',
|
||||
action='store_true',
|
||||
help='Skip PyPI queries for version resolution (faster, offline)'
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
script_path = Path(args.script)
|
||||
|
||||
if not script_path.exists():
|
||||
print(f"Error: File not found: {script_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Parse requirements path if provided
|
||||
req_path = Path(args.requirements) if args.requirements else None
|
||||
|
||||
# Parse output path if provided
|
||||
out_path = Path(args.output) if args.output else None
|
||||
|
||||
# Convert
|
||||
success = convert_script(
|
||||
script_path=script_path,
|
||||
output_path=out_path,
|
||||
requirements_path=req_path,
|
||||
detect_imports_flag=args.detect_imports,
|
||||
dry_run=args.dry_run,
|
||||
python_version=args.python_version,
|
||||
quiet_mode=args.quiet,
|
||||
use_pypi=not args.no_pypi
|
||||
)
|
||||
|
||||
sys.exit(0 if success else 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
356
skills/python-uv-scripts/tools/validate_script.py
Executable file
356
skills/python-uv-scripts/tools/validate_script.py
Executable file
@@ -0,0 +1,356 @@
|
||||
#!/usr/bin/env -S uv run --script --quiet
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""
|
||||
Validate PEP 723 inline script metadata
|
||||
|
||||
Checks Python scripts for:
|
||||
- Valid PEP 723 metadata block
|
||||
- Required fields (requires-python, dependencies)
|
||||
- TOML syntax validity
|
||||
- Shebang presence and format
|
||||
- Security issues
|
||||
|
||||
Usage:
|
||||
validate_script.py <script.py>
|
||||
validate_script.py --strict <script.py>
|
||||
validate_script.py --force <script>
|
||||
|
||||
Examples:
|
||||
# Basic validation
|
||||
validate_script.py my_script.py
|
||||
|
||||
# Strict mode (all best practices)
|
||||
validate_script.py --strict my_script.py
|
||||
|
||||
# Validate executable Python script without .py extension
|
||||
validate_script.py my_script
|
||||
|
||||
# Force validation, skip extension check
|
||||
validate_script.py --force my_script
|
||||
|
||||
# Validate all scripts in directory
|
||||
find . -name '*.py' -exec python validate_script.py {} \\;
|
||||
"""
|
||||
|
||||
import ast
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import tomllib
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@dataclass
|
||||
class ValidationResult:
|
||||
"""Validation result"""
|
||||
valid: bool
|
||||
has_metadata: bool
|
||||
has_shebang: bool
|
||||
has_docstring: bool
|
||||
warnings: list[str]
|
||||
errors: list[str]
|
||||
|
||||
|
||||
def extract_metadata_block(content: str) -> str | None:
|
||||
"""Extract PEP 723 metadata block"""
|
||||
# Match metadata block with CRLF tolerance and flexible whitespace
|
||||
# Uses lookahead to allow last metadata line without trailing newline
|
||||
pattern = r'# /// script\r?\n((?:#.*(?:\r?\n|(?=\r?\n?#\s*///)))+)(?:\r?\n)?#\s*///'
|
||||
match = re.search(pattern, content, re.MULTILINE)
|
||||
|
||||
if not match:
|
||||
return None
|
||||
|
||||
# Extract TOML content (remove leading # and optional whitespace from each line)
|
||||
lines = match.group(1).splitlines()
|
||||
toml_lines = []
|
||||
for line in lines:
|
||||
if line.startswith('#'):
|
||||
# Strip '#' followed by optional space or tab
|
||||
stripped = re.sub(r'^#[ \t]?', '', line)
|
||||
toml_lines.append(stripped)
|
||||
else:
|
||||
# Preserve non-comment lines (shouldn't occur with our regex but be safe)
|
||||
toml_lines.append(line)
|
||||
return '\n'.join(toml_lines)
|
||||
|
||||
|
||||
def validate_toml_syntax(toml_content: str) -> list[str]:
|
||||
"""Validate TOML syntax using structured parsing"""
|
||||
errors = []
|
||||
|
||||
# Parse TOML content
|
||||
try:
|
||||
data = tomllib.loads(toml_content)
|
||||
except tomllib.TOMLDecodeError as e:
|
||||
errors.append(f"Invalid TOML syntax: {e}")
|
||||
return errors
|
||||
|
||||
# Validate required fields
|
||||
if 'requires-python' not in data:
|
||||
errors.append("Missing 'requires-python' field")
|
||||
elif not isinstance(data['requires-python'], str):
|
||||
errors.append("'requires-python' must be a string")
|
||||
|
||||
if 'dependencies' not in data:
|
||||
errors.append("Missing 'dependencies' field")
|
||||
else:
|
||||
dependencies = data['dependencies']
|
||||
# Dependencies should be a list/array
|
||||
if not isinstance(dependencies, list):
|
||||
errors.append("'dependencies' must be an array/list")
|
||||
else:
|
||||
# Validate each dependency item
|
||||
for idx, dep in enumerate(dependencies):
|
||||
if not isinstance(dep, str):
|
||||
errors.append(
|
||||
f"Dependency at index {idx} must be a string, got {type(dep).__name__}")
|
||||
|
||||
return errors
|
||||
|
||||
|
||||
def check_shebang(content: str) -> tuple[bool, list[str]]:
|
||||
"""Check shebang line"""
|
||||
warnings = []
|
||||
lines = content.split('\n')
|
||||
|
||||
if not lines:
|
||||
return False, ["Empty file"]
|
||||
|
||||
first_line = lines[0]
|
||||
|
||||
if not first_line.startswith('#!'):
|
||||
return False, []
|
||||
|
||||
# Check for recommended shebangs
|
||||
recommended = [
|
||||
'#!/usr/bin/env -S uv run --script',
|
||||
'#!/usr/bin/env -S uv run --script --quiet',
|
||||
]
|
||||
|
||||
if first_line not in recommended:
|
||||
warnings.append(f"Shebang not recommended. Use: {recommended[0]}")
|
||||
|
||||
return True, warnings
|
||||
|
||||
|
||||
def check_security_issues(content: str) -> list[str]:
|
||||
"""Check for common security issues"""
|
||||
warnings = []
|
||||
|
||||
# Check for hardcoded secrets
|
||||
secret_patterns = [
|
||||
(r'password\s*=\s*["\']', "Possible hardcoded password"),
|
||||
(r'api[_-]?key\s*=\s*["\']', "Possible hardcoded API key"),
|
||||
(r'secret\s*=\s*["\']', "Possible hardcoded secret"),
|
||||
(r'token\s*=\s*["\']', "Possible hardcoded token"),
|
||||
]
|
||||
|
||||
for pattern, message in secret_patterns:
|
||||
if re.search(pattern, content, re.IGNORECASE):
|
||||
warnings.append(f"Security: {message}")
|
||||
|
||||
# Check for shell=True
|
||||
if re.search(r'shell\s*=\s*True', content):
|
||||
warnings.append(
|
||||
"Security: subprocess.run with shell=True (command injection risk)")
|
||||
|
||||
# Check for eval/exec
|
||||
if re.search(r'\b(eval|exec)\s*\(', content):
|
||||
warnings.append(
|
||||
"Security: Use of eval() or exec() (code injection risk)")
|
||||
|
||||
return warnings
|
||||
|
||||
|
||||
def is_valid_python_file(script_path: Path) -> tuple[bool, str]:
|
||||
"""
|
||||
Check if a non-.py file is a valid Python script.
|
||||
|
||||
Returns:
|
||||
Tuple of (is_valid, reason) where reason describes why it's valid or invalid
|
||||
"""
|
||||
try:
|
||||
content = script_path.read_text(encoding='utf-8')
|
||||
except (FileNotFoundError, PermissionError, OSError, UnicodeDecodeError) as e:
|
||||
return False, f"Cannot read file: {e}"
|
||||
|
||||
# Check if file is executable with Python shebang
|
||||
is_executable = os.access(script_path, os.X_OK)
|
||||
lines = content.split('\n')
|
||||
has_python_shebang = False
|
||||
|
||||
if lines and lines[0].startswith('#!'):
|
||||
shebang = lines[0].lower()
|
||||
has_python_shebang = 'python' in shebang
|
||||
|
||||
if is_executable and has_python_shebang:
|
||||
return True, "executable with Python shebang"
|
||||
|
||||
# Try to parse as Python to confirm it's valid Python code
|
||||
try:
|
||||
ast.parse(content)
|
||||
return True, "valid Python syntax"
|
||||
except SyntaxError as e:
|
||||
return False, f"not valid Python: {e}"
|
||||
|
||||
|
||||
def validate_script(script_path: Path, strict: bool = False) -> ValidationResult:
|
||||
"""Validate Python script"""
|
||||
result = ValidationResult(
|
||||
valid=True,
|
||||
has_metadata=False,
|
||||
has_shebang=False,
|
||||
has_docstring=False,
|
||||
warnings=[],
|
||||
errors=[]
|
||||
)
|
||||
|
||||
# Read file
|
||||
try:
|
||||
content = script_path.read_text(encoding='utf-8')
|
||||
except (FileNotFoundError, PermissionError, OSError, UnicodeDecodeError) as e:
|
||||
result.valid = False
|
||||
result.errors.append(f"Failed to read file: {e}")
|
||||
return result
|
||||
|
||||
# Check shebang
|
||||
has_shebang, shebang_warnings = check_shebang(content)
|
||||
result.has_shebang = has_shebang
|
||||
result.warnings.extend(shebang_warnings)
|
||||
|
||||
if strict and not has_shebang:
|
||||
result.errors.append("Missing shebang (required in strict mode)")
|
||||
result.valid = False
|
||||
|
||||
# Check for metadata block
|
||||
metadata = extract_metadata_block(content)
|
||||
result.has_metadata = metadata is not None
|
||||
|
||||
if not metadata:
|
||||
result.errors.append("No PEP 723 metadata block found")
|
||||
result.valid = False
|
||||
return result
|
||||
|
||||
# Validate TOML syntax
|
||||
toml_errors = validate_toml_syntax(metadata)
|
||||
result.errors.extend(toml_errors)
|
||||
|
||||
if toml_errors:
|
||||
result.valid = False
|
||||
|
||||
# Check for module docstring using AST parsing
|
||||
try:
|
||||
module_node = ast.parse(content)
|
||||
module_docstring = ast.get_docstring(module_node)
|
||||
result.has_docstring = module_docstring is not None
|
||||
except SyntaxError as e:
|
||||
result.has_docstring = False
|
||||
result.warnings.append(
|
||||
f"Could not parse file for docstring check: {e}")
|
||||
|
||||
if strict and not result.has_docstring:
|
||||
result.warnings.append(
|
||||
"Missing module docstring (recommended in strict mode)")
|
||||
|
||||
# Security checks (always warnings, never errors - these are heuristic checks)
|
||||
security_warnings = check_security_issues(content)
|
||||
result.warnings.extend(security_warnings)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point"""
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Validate PEP 723 script metadata",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__
|
||||
)
|
||||
parser.add_argument('script', help='Python script to validate')
|
||||
parser.add_argument('--strict', action='store_true',
|
||||
help='Enable strict validation')
|
||||
parser.add_argument('--force', action='store_true',
|
||||
help='Skip Python file extension check')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
script_path = Path(args.script)
|
||||
|
||||
if not script_path.exists():
|
||||
print(f"Error: File not found: {script_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Check if file is a Python file
|
||||
if script_path.suffix != '.py':
|
||||
if args.force:
|
||||
print(
|
||||
f"Warning: File lacks .py extension, but --force was specified", file=sys.stderr)
|
||||
else:
|
||||
# Check if it's a valid Python file by other means
|
||||
is_valid, reason = is_valid_python_file(script_path)
|
||||
if not is_valid:
|
||||
print(
|
||||
f"Error: Not a Python file: {script_path}", file=sys.stderr)
|
||||
print(f" Reason: {reason}", file=sys.stderr)
|
||||
print(f" Hint: File must either:", file=sys.stderr)
|
||||
print(f" - Have a .py extension, OR", file=sys.stderr)
|
||||
print(f" - Be executable with a Python shebang, OR",
|
||||
file=sys.stderr)
|
||||
print(f" - Contain valid Python syntax", file=sys.stderr)
|
||||
print(f" Use --force to skip this check", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
else:
|
||||
print(
|
||||
f"Info: File accepted as Python ({reason})", file=sys.stderr)
|
||||
|
||||
# Validate
|
||||
result = validate_script(script_path, strict=args.strict)
|
||||
|
||||
# Print results
|
||||
print(f"Validating: {script_path}")
|
||||
print("=" * 60)
|
||||
|
||||
if result.has_shebang:
|
||||
print("✓ Has shebang")
|
||||
else:
|
||||
print("✗ Missing shebang")
|
||||
|
||||
if result.has_metadata:
|
||||
print("✓ Has PEP 723 metadata")
|
||||
else:
|
||||
print("✗ Missing PEP 723 metadata")
|
||||
|
||||
if result.has_docstring:
|
||||
print("✓ Has docstring")
|
||||
else:
|
||||
print("○ No docstring")
|
||||
|
||||
if result.warnings:
|
||||
print("\nWarnings:")
|
||||
for warning in result.warnings:
|
||||
print(f" ⚠ {warning}")
|
||||
|
||||
if result.errors:
|
||||
print("\nErrors:")
|
||||
for error in result.errors:
|
||||
print(f" ✗ {error}")
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
|
||||
if result.valid:
|
||||
print("Status: ✓ VALID")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print("Status: ✗ INVALID")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
67
skills/python-uv-scripts/workflows/ci-cd-integration.md
Normal file
67
skills/python-uv-scripts/workflows/ci-cd-integration.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# CI/CD Integration for UV Scripts
|
||||
|
||||
> **Status**: 🚧 Placeholder - Content in development
|
||||
|
||||
## Overview
|
||||
|
||||
Integrating UV single-file scripts into CI/CD pipelines with GitHub Actions, GitLab CI, and other platforms.
|
||||
|
||||
## Topics to Cover
|
||||
|
||||
- [ ] GitHub Actions workflows
|
||||
- [ ] GitLab CI configuration
|
||||
- [ ] Pre-commit hooks integration
|
||||
- [ ] Automated testing
|
||||
- [ ] Security scanning
|
||||
- [ ] Deployment strategies
|
||||
- [ ] Version management
|
||||
|
||||
## Quick Example
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
```yaml
|
||||
name: Test UV Scripts
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v3
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Run script tests
|
||||
run: |
|
||||
uv run scripts/check_health.py --validate
|
||||
uv run scripts/analyze_data.py --dry-run
|
||||
```
|
||||
|
||||
### Pre-commit Hook
|
||||
|
||||
```yaml
|
||||
# .pre-commit-config.yaml
|
||||
repos:
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: validate-uv-scripts
|
||||
name: Validate UV Scripts
|
||||
entry: uv run scripts/validate_all.py
|
||||
language: system
|
||||
pass_filenames: false
|
||||
```
|
||||
|
||||
## TODO
|
||||
|
||||
This file will be expanded to include:
|
||||
|
||||
- Complete GitHub Actions examples
|
||||
- GitLab CI patterns
|
||||
- Pre-commit hook configurations
|
||||
- Automated deployment workflows
|
||||
- Security scanning integration
|
||||
58
skills/python-uv-scripts/workflows/team-adoption.md
Normal file
58
skills/python-uv-scripts/workflows/team-adoption.md
Normal file
@@ -0,0 +1,58 @@
|
||||
# Team Adoption Guide
|
||||
|
||||
> **Status**: 🚧 Placeholder - Content in development
|
||||
|
||||
## Overview
|
||||
|
||||
Guide for rolling out UV single-file scripts across development teams, establishing standards, and ensuring adoption.
|
||||
|
||||
## Topics to Cover
|
||||
|
||||
- [ ] Introduction and training materials
|
||||
- [ ] Team standards and conventions
|
||||
- [ ] Code review guidelines
|
||||
- [ ] Migration from existing scripts
|
||||
- [ ] Tooling setup across platforms
|
||||
- [ ] Documentation requirements
|
||||
- [ ] Success metrics and tracking
|
||||
|
||||
## Adoption Roadmap
|
||||
|
||||
### Phase 1: Pilot (Week 1-2)
|
||||
|
||||
- Select 2-3 team members for initial trial
|
||||
- Convert 3-5 existing scripts
|
||||
- Gather feedback and iterate
|
||||
|
||||
### Phase 2: Expansion (Week 3-4)
|
||||
|
||||
- Team-wide training session
|
||||
- Establish coding standards
|
||||
- Set up CI/CD integration
|
||||
- Create internal documentation
|
||||
|
||||
### Phase 3: Full Adoption (Week 5+)
|
||||
|
||||
- All new scripts use UV format
|
||||
- Migration plan for legacy scripts
|
||||
- Regular reviews and improvements
|
||||
|
||||
## Team Standards Checklist
|
||||
|
||||
- [ ] Required shebang format: `#!/usr/bin/env -S uv run`
|
||||
- [ ] Minimum Python version: `>=3.11`
|
||||
- [ ] Required metadata fields
|
||||
- [ ] Naming conventions
|
||||
- [ ] Documentation requirements
|
||||
- [ ] Testing requirements
|
||||
- [ ] Security review process
|
||||
|
||||
## TODO
|
||||
|
||||
This file will be expanded to include:
|
||||
|
||||
- Detailed training materials
|
||||
- Complete adoption roadmap
|
||||
- Team communication templates
|
||||
- Migration guides
|
||||
- Success stories and case studies
|
||||
72
skills/python-uv-scripts/workflows/testing-strategies.md
Normal file
72
skills/python-uv-scripts/workflows/testing-strategies.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# Testing Strategies for UV Scripts
|
||||
|
||||
> **Status**: 🚧 Placeholder - Content in development
|
||||
|
||||
## Overview
|
||||
|
||||
Strategies for testing UV single-file scripts, from inline tests to pytest integration.
|
||||
|
||||
## Topics to Cover
|
||||
|
||||
- [ ] Inline test functions
|
||||
- [ ] pytest integration
|
||||
- [ ] Mock and fixtures for testing
|
||||
- [ ] Testing CLI applications
|
||||
- [ ] API client testing with mocks
|
||||
- [ ] Coverage reporting
|
||||
- [ ] Test automation in CI/CD
|
||||
|
||||
## Quick Example
|
||||
|
||||
### Inline Testing
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# ///
|
||||
def add(a: int, b: int) -> int:
|
||||
return a + b
|
||||
|
||||
def test_add():
|
||||
assert add(2, 3) == 5
|
||||
assert add(-1, 1) == 0
|
||||
print("✓ All tests passed")
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Run tests
|
||||
test_add()
|
||||
|
||||
# Or run main application
|
||||
result = add(10, 20)
|
||||
print(f"Result: {result}")
|
||||
```
|
||||
|
||||
### Pytest Integration
|
||||
|
||||
```python
|
||||
#!/usr/bin/env -S uv run
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["pytest>=7.0.0"]
|
||||
# ///
|
||||
"""
|
||||
Run with: pytest this_script.py
|
||||
"""
|
||||
def multiply(a: int, b: int) -> int:
|
||||
return a * b
|
||||
|
||||
def test_multiply():
|
||||
assert multiply(2, 3) == 6
|
||||
assert multiply(-1, 5) == -5
|
||||
```
|
||||
|
||||
## TODO
|
||||
|
||||
This file will be expanded to include:
|
||||
|
||||
- Complete testing patterns
|
||||
- Mocking external dependencies
|
||||
- Test organization strategies
|
||||
- CI/CD integration examples
|
||||
- Coverage tools and reporting
|
||||
Reference in New Issue
Block a user