Initial commit

This commit is contained in:
Zhongwei Li
2025-11-30 09:01:33 +08:00
commit 5115beb98d
8 changed files with 3836 additions and 0 deletions

View File

@@ -0,0 +1,782 @@
# Migration Guide: Moving to UV and Ruff
Complete guide for migrating from pip, conda, poetry, or pipx to UV, and from Flake8, Black, isort to Ruff.
## Table of Contents
1. [Why Migrate?](#why-migrate)
2. [From pip + virtualenv](#from-pip--virtualenv)
3. [From conda](#from-conda)
4. [From poetry](#from-poetry)
5. [From pipx](#from-pipx)
6. [From Flake8/Black/isort to Ruff](#from-flake8blackisort-to-ruff)
7. [Complete Workflow Migration](#complete-workflow-migration)
## Why Migrate?
### UV Benefits
- **10-100x faster** than pip for package installation
- **Single tool** replacing pip, pip-tools, pipx, poetry, pyenv, virtualenv
- **Automatic environment management** - no manual activation needed
- **Universal lockfiles** for cross-platform reproducibility
- **Python version management** built-in
- **Zero dependencies** - standalone binary
### Ruff Benefits
- **10-100x faster** than existing linters
- **Single tool** replacing Flake8, Black, isort, pyupgrade, autoflake
- **800+ lint rules** with auto-fix capabilities
- **Formatting** compatible with Black
- **Editor integration** with first-class support
- **Zero configuration** needed to get started
## From pip + virtualenv
### Current Workflow
```bash
# Create virtual environment
python -m venv .venv
source .venv/bin/activate # or .venv\Scripts\activate on Windows
# Install dependencies
pip install -r requirements.txt
# Install dev dependencies
pip install -r requirements-dev.txt
# Run application
python main.py
```
### New Workflow with UV
```bash
# Initialize project (one-time)
uv init my-project
cd my-project
# Add dependencies
uv add requests pandas numpy
# Add dev dependencies
uv add --dev pytest black ruff
# Run application (no activation needed!)
uv run python main.py
# Run tests
uv run pytest
```
### Migration Steps
**Step 1: Install UV**
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
**Step 2: Convert requirements.txt to pyproject.toml**
If you have `requirements.txt`:
```bash
# Create new project
uv init .
# Install from requirements.txt
uv pip install -r requirements.txt
# Generate pyproject.toml dependencies
uv add $(cat requirements.txt | grep -v '^#' | grep -v '^$')
```
Or manually create `pyproject.toml`:
```toml
[project]
name = "my-project"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"requests>=2.31.0",
"pandas>=2.0.0",
]
[tool.uv]
dev-dependencies = [
"pytest>=7.0.0",
"ruff>=0.1.0",
]
```
**Step 3: Create lockfile**
```bash
uv lock
```
**Step 4: Update CI/CD**
Before:
```yaml
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
```
After:
```yaml
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install dependencies
run: uv sync
```
**Step 5: Update Development Scripts**
Before:
```bash
#!/bin/bash
source .venv/bin/activate
python manage.py runserver
```
After:
```bash
#!/bin/bash
uv run python manage.py runserver
```
### Maintaining requirements.txt (Optional)
If you need to maintain `requirements.txt` for compatibility:
```bash
# Generate requirements.txt from lockfile
uv export -o requirements.txt
# Generate dev requirements
uv export --group dev -o requirements-dev.txt
```
## From conda
### Current Workflow
```bash
# Create environment
conda create -n myenv python=3.11
conda activate myenv
# Install dependencies
conda install numpy pandas scipy
pip install requests # Some packages not in conda
# Export environment
conda env export > environment.yml
```
### New Workflow with UV
```bash
# Initialize project
uv init my-project
cd my-project
# Pin Python version
uv python pin 3.11
# Add dependencies (all from PyPI)
uv add numpy pandas scipy requests
# All dependencies in one place
uv lock
```
### Migration Steps
**Step 1: Export conda dependencies**
```bash
# Get list of installed packages
conda list --export > conda-packages.txt
# Or just the package names
conda env export --from-history > environment.yml
```
**Step 2: Convert to pyproject.toml**
From `environment.yml`:
```yaml
name: myenv
dependencies:
- python=3.11
- numpy=1.24.0
- pandas=2.0.0
- pip:
- requests==2.31.0
```
To `pyproject.toml`:
```toml
[project]
name = "myproject"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"numpy>=1.24.0",
"pandas>=2.0.0",
"requests>=2.31.0",
]
```
**Step 3: Handle Conda-Only Packages**
Some packages are only available through conda. Options:
1. **Use PyPI alternatives**: Many packages are now on PyPI
2. **Keep conda for specific packages**: Use conda + uv hybrid
3. **Build from source**: UV can build packages if needed
**Step 4: Remove Conda Environment**
```bash
# Deactivate conda environment
conda deactivate
# Remove environment
conda env remove -n myenv
```
### Conda vs UV Comparison
| Feature | conda | UV |
|---------|-------|-----|
| Speed | Slow (10-30min) | Fast (10-30sec) |
| Python Versions | ✅ | ✅ |
| Non-Python Packages | ✅ | ❌ |
| PyPI Packages | Limited | Full |
| Lockfiles | ✅ | ✅ |
| Cross-platform | ✅ | ✅ |
| Memory Usage | High (1-2GB) | Low (<100MB) |
### When to Keep Conda
Keep conda if you need:
- Non-Python packages (R, Julia, C libraries)
- Specific binary distributions
- Legacy scientific computing workflows
You can use both:
```bash
# Use conda for system-level dependencies
conda install gcc openblas
# Use uv for Python packages
uv sync
```
## From poetry
### Current Workflow
```bash
# Create project
poetry new my-project
cd my-project
# Add dependencies
poetry add requests
# Install dependencies
poetry install
# Run scripts
poetry run python main.py
```
### New Workflow with UV
```bash
# Create project
uv init my-project
cd my-project
# Add dependencies
uv add requests
# Install dependencies (automatic with add)
# No separate install step needed!
# Run scripts
uv run python main.py
```
### Migration Steps
**Step 1: Convert pyproject.toml**
Poetry `pyproject.toml`:
```toml
[tool.poetry]
name = "my-project"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
[tool.poetry.dependencies]
python = "^3.11"
requests = "^2.31.0"
[tool.poetry.dev-dependencies]
pytest = "^7.0.0"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
```
UV `pyproject.toml`:
```toml
[project]
name = "my-project"
version = "0.1.0"
description = ""
authors = [{name = "Your Name", email = "[email protected]"}]
requires-python = ">=3.11"
dependencies = [
"requests>=2.31.0",
]
[tool.uv]
dev-dependencies = [
"pytest>=7.0.0",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
```
**Step 2: Convert Version Constraints**
Poetry uses caret (`^`) for version constraints:
- `^2.31.0` means `>=2.31.0, <3.0.0`
UV uses standard pip syntax:
- `>=2.31.0,<3.0.0` or `>=2.31.0`
**Step 3: Remove Poetry Files**
```bash
rm poetry.lock
rm -rf .venv
poetry env remove --all
```
**Step 4: Initialize UV**
```bash
uv lock
uv sync
```
**Step 5: Update Scripts**
Before:
```toml
[tool.poetry.scripts]
start = "my_project.main:main"
```
After:
```toml
[project.scripts]
start = "my_project.main:main"
```
Or just use `uv run`:
```bash
uv run python -m my_project.main
```
### Poetry vs UV Comparison
| Feature | Poetry | UV |
|---------|--------|-----|
| Speed | Medium | Very Fast |
| Lockfiles | ✅ | ✅ |
| Python Management | ❌ | ✅ |
| Tool Running | ❌ | ✅ (uvx) |
| Build Backend | poetry-core | Any (hatchling, setuptools) |
| Configuration | Opinionated | Flexible |
## From pipx
### Current Workflow
```bash
# Install tools globally
pipx install black
pipx install ruff
pipx install pytest
# Run tools
black .
ruff check .
```
### New Workflow with UV
```bash
# Install tools globally
uv tool install black
uv tool install ruff
uv tool install pytest
# Or run tools ephemerally
uvx black .
uvx ruff check .
uvx pytest
```
### Migration Steps
**Step 1: List pipx installations**
```bash
pipx list
```
**Step 2: Install with UV**
```bash
# For each tool in pipx list
uv tool install tool-name
```
**Step 3: Remove pipx tools**
```bash
pipx uninstall-all
```
**Step 4: Update PATH (if needed)**
Tools are installed in:
- **pipx**: `~/.local/bin/`
- **uv**: `~/.local/bin/` (same location!)
No PATH changes needed!
### pipx vs UV Tool Comparison
| Feature | pipx | UV Tool |
|---------|------|---------|
| Speed | Medium | Fast |
| Ephemeral runs | ❌ | ✅ (uvx) |
| Python Management | ❌ | ✅ |
| Isolated Environments | ✅ | ✅ |
| Upgrade Command | ✅ | ✅ |
## From Flake8/Black/isort to Ruff
### Current Workflow
```bash
# Multiple tools
isort .
black .
flake8 .
# With configuration in multiple files
# .flake8
# pyproject.toml [tool.black]
# pyproject.toml [tool.isort]
```
### New Workflow with Ruff
```bash
# Single command
ruff check --fix . && ruff format .
# All configuration in pyproject.toml
# [tool.ruff]
```
### Migration Steps
**Step 1: Install Ruff**
```bash
uv add --dev ruff
```
**Step 2: Convert Configuration**
From `.flake8`:
```ini
[flake8]
max-line-length = 88
extend-ignore = E203, W503
exclude = .git,__pycache__,build
per-file-ignores =
__init__.py:F401
```
From `pyproject.toml`:
```toml
[tool.black]
line-length = 88
target-version = ['py311']
[tool.isort]
profile = "black"
known_first_party = ["myproject"]
```
To unified Ruff config:
```toml
[tool.ruff]
line-length = 88
target-version = "py311"
[tool.ruff.lint]
ignore = ["E203", "W503"]
exclude = [".git", "__pycache__", "build"]
[tool.ruff.lint.per-file-ignores]
"__init__.py" = ["F401"]
[tool.ruff.lint.isort]
known-first-party = ["myproject"]
[tool.ruff.format]
quote-style = "double"
```
**Step 3: Test Ruff**
```bash
# Check for issues
ruff check .
# Auto-fix
ruff check --fix .
# Format
ruff format .
```
**Step 4: Remove Old Tools**
```bash
uv remove --dev black isort flake8
```
**Step 5: Update Pre-commit**
Before:
```yaml
repos:
- repo: https://github.com/PyCQA/isort
rev: 5.12.0
hooks:
- id: isort
- repo: https://github.com/psf/black
rev: 23.0.0
hooks:
- id: black
- repo: https://github.com/PyCQA/flake8
rev: 6.0.0
hooks:
- id: flake8
```
After:
```yaml
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.12.8
hooks:
- id: ruff
args: [--fix]
- id: ruff-format
```
**Step 6: Update CI/CD**
Before:
```yaml
- name: Lint
run: |
pip install black isort flake8
isort --check .
black --check .
flake8 .
```
After:
```yaml
- name: Lint
run: |
uv tool install ruff
ruff check .
ruff format --check .
```
## Complete Workflow Migration
### Before: Traditional Setup
**Project Structure:**
```
my-project/
├── .flake8
├── requirements.txt
├── requirements-dev.txt
├── setup.py
└── src/
```
**Development Workflow:**
```bash
# Setup
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -r requirements-dev.txt
# Development
isort .
black .
flake8 .
pytest
# Every day
source .venv/bin/activate # Easy to forget!
```
### After: Modern Setup with UV + Ruff
**Project Structure:**
```
my-project/
├── pyproject.toml # All configuration
├── uv.lock # Reproducible dependencies
└── src/
```
**Development Workflow:**
```bash
# Setup (one-time)
uv sync
# Development (no activation needed!)
uv run ruff check --fix .
uv run ruff format .
uv run pytest
# That's it!
```
### Migration Checklist
- [ ] Install UV
- [ ] Convert requirements to pyproject.toml
- [ ] Generate lockfile (`uv lock`)
- [ ] Install dependencies (`uv sync`)
- [ ] Install Ruff (`uv add --dev ruff`)
- [ ] Convert linter/formatter config
- [ ] Test Ruff (`ruff check . && ruff format .`)
- [ ] Update CI/CD pipelines
- [ ] Update pre-commit hooks
- [ ] Update documentation
- [ ] Remove old tools
- [ ] Update team workflows
- [ ] Celebrate faster builds! 🎉
## Rollback Plan
If you need to rollback:
**Save Old Configuration:**
```bash
# Before migration
cp requirements.txt requirements.txt.backup
cp .flake8 .flake8.backup
# etc.
```
**Keep Old Files:**
Don't delete old files until you're confident in the migration.
**Gradual Migration:**
You can run UV and pip side-by-side:
```bash
# Use both during transition
uv sync # For new workflow
pip install -r requirements.txt # For old workflow
```
## Common Issues
### UV Can't Find Python
```bash
# Install Python with UV
uv python install 3.12
# Or point to existing Python
uv python pin $(which python3.12)
```
### Ruff Too Strict
```bash
# Start with minimal rules
ruff check --select F . # Only Pyflakes
# Gradually add more
ruff check --select E,F . # Add pycodestyle
```
### Performance Issues
```bash
# Clear caches
uv cache clean
ruff clean
# Exclude large directories
# In pyproject.toml
[tool.ruff]
exclude = ["node_modules", "vendor"]
```
## Success Stories
**Typical Results After Migration:**
- **Installation time**: 5 minutes → 30 seconds (10x faster)
- **Linting time**: 15 seconds → 0.5 seconds (30x faster)
- **CI/CD time**: 10 minutes → 2 minutes (5x faster)
- **Tools to manage**: 7 → 2 (3.5x fewer)
- **Config files**: 4 → 1 (4x simpler)
- **Memory usage**: 2GB → 200MB (10x less)
## Next Steps
After migration:
1. Configure Ruff rules to your needs
2. Set up pre-commit hooks
3. Update team documentation
4. Train team on new workflow
5. Monitor CI/CD improvements
6. Consider adopting more modern Python features
## Resources
- [UV Documentation](https://docs.astral.sh/uv/)
- [Ruff Documentation](https://docs.astral.sh/ruff/)
- [UV Migration FAQ](https://docs.astral.sh/uv/guides/projects/)
- [Ruff Migration Guide](https://docs.astral.sh/ruff/formatter/)

View File

@@ -0,0 +1,892 @@
# Ruff: Complete Guide
Ruff is an extremely fast Python linter and code formatter written in Rust. It's 10-100x faster than existing linters and formatters, combining functionality from Flake8, Black, isort, and more into a single tool.
## Table of Contents
1. [Installation](#installation)
2. [Linting](#linting)
3. [Formatting](#formatting)
4. [Configuration](#configuration)
5. [Rule Selection](#rule-selection)
6. [Error Suppression](#error-suppression)
7. [Editor Integration](#editor-integration)
8. [CI/CD Integration](#cicd-integration)
9. [Migration Guide](#migration-guide)
## Installation
### With UV (Recommended)
```bash
# Install as a tool
uv tool install ruff
# Or add to project
uv add --dev ruff
```
### With pip
```bash
pip install ruff
```
### With Homebrew
```bash
brew install ruff
```
### With conda
```bash
conda install -c conda-forge ruff
```
### Verify Installation
```bash
ruff version
# Output: ruff 0.12.8
```
## Linting
### Basic Linting
```bash
# Check current directory
ruff check .
# Check specific files
ruff check src/main.py tests/
# Check and auto-fix
ruff check --fix .
# Preview changes without applying
ruff check --diff .
# Show fixes that would be applied
ruff check --show-fixes .
```
### Watch Mode
```bash
# Continuously check for errors
ruff check --watch .
```
### Output Formats
```bash
# Default (human-readable)
ruff check .
# JSON format
ruff check --output-format json .
# GitHub Actions format
ruff check --output-format github .
# GitLab format
ruff check --output-format gitlab .
# JUnit XML format
ruff check --output-format junit .
```
### Unsafe Fixes
```bash
# Include unsafe fixes
ruff check --fix --unsafe-fixes .
# Fix only (don't report remaining violations)
ruff check --fix-only .
```
### Statistics and Debugging
```bash
# Show statistics for each rule
ruff check --statistics .
# Show files that will be checked
ruff check --show-files .
# Show configuration being used
ruff check --show-settings .
```
## Formatting
### Basic Formatting
```bash
# Format current directory
ruff format .
# Format specific files
ruff format src/main.py
# Check formatting without changes
ruff format --check .
# Show diff of changes
ruff format --diff .
```
### Format and Lint Together
```bash
# Recommended workflow
ruff check --fix . && ruff format .
```
### Format Options
```bash
# Use preview features
ruff format --preview .
# Specific target Python version
ruff format --target-version py312 .
```
## Configuration
### Configuration Files
Ruff looks for configuration in (in order of precedence):
1. `.ruff.toml`
2. `ruff.toml`
3. `pyproject.toml`
### Basic Configuration
**pyproject.toml:**
```toml
[tool.ruff]
# Set line length
line-length = 88
indent-width = 4
# Set Python version
target-version = "py311"
# Exclude directories
exclude = [
".git",
".venv",
"__pycache__",
"build",
"dist",
"*.egg-info",
]
[tool.ruff.lint]
# Enable specific rules
select = [
"E", # pycodestyle errors
"W", # pycodestyle warnings
"F", # Pyflakes
"I", # isort
"B", # flake8-bugbear
"UP", # pyupgrade
"C4", # flake8-comprehensions
]
# Ignore specific rules
ignore = [
"E501", # line too long (handled by formatter)
]
# Allow auto-fixing
fixable = ["ALL"]
unfixable = []
# Allow unused variables prefixed with underscore
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
[tool.ruff.format]
# Use double quotes
quote-style = "double"
# Use spaces for indentation
indent-style = "space"
# Respect magic trailing commas
skip-magic-trailing-comma = false
# Auto-detect line endings
line-ending = "auto"
# Format code in docstrings
docstring-code-format = true
docstring-code-line-length = 72
```
### Per-File Configuration
```toml
[tool.ruff.lint.per-file-ignores]
# Ignore unused imports in __init__.py
"__init__.py" = ["F401"]
# Ignore assert statements in tests
"tests/*" = ["S101"]
# Ignore print statements in scripts
"scripts/*" = ["T201"]
# Ignore import order in migrations
"migrations/*" = ["I"]
```
### Import Sorting (isort)
```toml
[tool.ruff.lint.isort]
# Known first-party packages
known-first-party = ["myproject"]
# Known third-party packages
known-third-party = ["django", "requests"]
# Section order
section-order = [
"future",
"standard-library",
"third-party",
"first-party",
"local-folder"
]
# Combine as imports
combine-as-imports = true
# Split on trailing comma
split-on-trailing-comma = true
```
### Docstring Configuration
```toml
[tool.ruff.lint.pydocstyle]
# Use Google-style docstrings
convention = "google" # or "numpy", "pep257"
```
## Rule Selection
### Available Rule Sets
| Code | Name | Description |
|------|------|-------------|
| E | pycodestyle errors | PEP 8 error codes |
| W | pycodestyle warnings | PEP 8 warning codes |
| F | Pyflakes | Logical errors |
| I | isort | Import sorting |
| N | pep8-naming | Naming conventions |
| D | pydocstyle | Docstring style |
| UP | pyupgrade | Modern Python syntax |
| B | flake8-bugbear | Common bugs |
| A | flake8-builtins | Builtin shadowing |
| C4 | flake8-comprehensions | List/dict/set comprehensions |
| T20 | flake8-print | Print statements |
| PT | flake8-pytest-style | Pytest style |
| S | flake8-bandit | Security issues |
| Q | flake8-quotes | Quote style |
| RUF | Ruff-specific | Ruff custom rules |
### Selecting Rules
```bash
# Enable specific rule set
ruff check --select E,W,F .
# Enable all rules
ruff check --select ALL .
# Extend default rules
ruff check --extend-select B,I .
# Ignore specific rules
ruff check --ignore E501,W503 .
```
### Configuration
```toml
[tool.ruff.lint]
# Start with Flake8 defaults and add more
select = ["E", "F"]
extend-select = ["B", "I", "N"]
# Ignore specific rules
ignore = ["E501"]
extend-ignore = ["W503"]
# Make specific rules fixable
fixable = ["I", "F401"]
# Make specific rules non-fixable
unfixable = ["B"]
```
### Popular Rule Combinations
**Minimal (Default):**
```toml
select = ["E4", "E7", "E9", "F"]
```
**Standard:**
```toml
select = ["E", "W", "F", "I"]
```
**Strict:**
```toml
select = [
"E", # pycodestyle errors
"W", # pycodestyle warnings
"F", # Pyflakes
"I", # isort
"N", # pep8-naming
"D", # pydocstyle
"UP", # pyupgrade
"B", # flake8-bugbear
"C4", # flake8-comprehensions
"S", # flake8-bandit
"T20", # flake8-print
"PT", # flake8-pytest-style
]
```
## Error Suppression
### Inline Comments
```python
# Ignore specific rule on a line
import os # noqa: F401
# Ignore multiple rules
import sys, os # noqa: F401, E401
# Ignore all rules on a line
very_long_variable_name = "value" # noqa
# Ignore for next line
# noqa: E501
very_long_line = "This is a very long line that exceeds the limit"
```
### File-Level Suppression
```python
# At top of file
# ruff: noqa: F401, E402
import os
import sys
```
### Auto-Add noqa Comments
```bash
# Automatically add noqa comments
ruff check --add-noqa .
```
### Type-Aware Suppression
```python
# Type checkers only
import TYPE_CHECKING # type: ignore
# Ruff specific
from typing import Optional # ruff: noqa: UP007
```
## Editor Integration
### VS Code
Install the [Ruff VS Code extension](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff):
**.vscode/settings.json:**
```json
{
"[python]": {
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.fixAll.ruff": "explicit",
"source.organizeImports.ruff": "explicit"
},
"editor.defaultFormatter": "charliermarsh.ruff"
},
"ruff.lint.args": ["--config=pyproject.toml"],
"ruff.format.args": ["--config=pyproject.toml"]
}
```
### PyCharm/IntelliJ
1. Install Ruff via system package manager
2. Configure as external tool:
- File → Settings → Tools → External Tools
- Add Ruff with appropriate arguments
### Vim/Neovim
**With ALE:**
```vim
let g:ale_linters = {'python': ['ruff']}
let g:ale_fixers = {'python': ['ruff']}
```
**With nvim-lspconfig:**
```lua
require('lspconfig').ruff_lsp.setup{}
```
### Emacs
```elisp
;; With flycheck
(require 'flycheck)
(flycheck-define-checker python-ruff
"A Python checker using ruff."
:command ("ruff" "check" "--output-format" "text" source)
:error-patterns
((error line-start (file-name) ":" line ":" column ": " (message) line-end))
:modes python-mode)
(add-to-list 'flycheck-checkers 'python-ruff)
```
## CI/CD Integration
### GitHub Actions
```yaml
name: Lint with Ruff
on: [push, pull_request]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install dependencies
run: uv sync
- name: Lint with Ruff
run: uv run ruff check .
- name: Format check with Ruff
run: uv run ruff format --check .
```
**Using Ruff Action:**
```yaml
- uses: chartboost/ruff-action@v1
with:
args: check --output-format github
```
### GitLab CI
```yaml
ruff:
image: python:3.12
before_script:
- pip install ruff
script:
- ruff check .
- ruff format --check .
```
### Pre-commit Hooks
**.pre-commit-config.yaml:**
```yaml
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.12.8
hooks:
# Run the linter
- id: ruff
args: [--fix]
# Run the formatter
- id: ruff-format
```
Install:
```bash
pip install pre-commit
pre-commit install
```
### Docker
```dockerfile
FROM python:3.12-slim
# Install Ruff
RUN pip install ruff
# Copy code
COPY . /app
WORKDIR /app
# Run Ruff
RUN ruff check .
RUN ruff format --check .
```
## Migration Guide
### From Flake8
```bash
# Flake8 configuration
[flake8]
max-line-length = 88
extend-ignore = E203, W503
exclude = .git,__pycache__,build,dist
```
```toml
# Equivalent Ruff configuration
[tool.ruff]
line-length = 88
[tool.ruff.lint]
ignore = ["E203", "W503"]
exclude = [".git", "__pycache__", "build", "dist"]
```
### From Black
Ruff format is designed to be a drop-in replacement for Black:
```bash
# Replace
black .
# With
ruff format .
```
Configuration is compatible:
```toml
[tool.black]
line-length = 88
target-version = ["py311"]
# Becomes
[tool.ruff]
line-length = 88
target-version = "py311"
```
### From isort
```bash
# Replace
isort .
# With
ruff check --select I --fix .
# Or
ruff format . # Includes import sorting
```
Configuration:
```toml
[tool.isort]
profile = "black"
known_first_party = ["myproject"]
# Becomes
[tool.ruff.lint.isort]
known-first-party = ["myproject"]
```
### From pyupgrade
```bash
# Replace
pyupgrade --py311-plus **/*.py
# With
ruff check --select UP --fix .
```
### Complete Migration
**Before:**
```bash
# Multiple tools
isort .
black .
flake8 .
pyupgrade --py311-plus **/*.py
```
**After:**
```bash
# Single command
ruff check --fix . && ruff format .
```
**Configuration consolidation:**
```toml
# pyproject.toml - All in one place
[tool.ruff]
line-length = 88
target-version = "py311"
[tool.ruff.lint]
select = ["E", "W", "F", "I", "UP", "B"]
ignore = ["E501"]
[tool.ruff.format]
quote-style = "double"
```
## Common Rules Explained
### E/W (pycodestyle)
```python
# E501: Line too long
very_long_line = "This line exceeds 88 characters and will be flagged by E501"
# E401: Multiple imports on one line
import os, sys # Bad
# E402: Module level import not at top
def foo():
pass
import os # Bad
# W503: Line break before binary operator
result = (value
+ other_value) # Warning in older style guides
```
### F (Pyflakes)
```python
# F401: Imported but unused
import os # Not used anywhere
# F841: Local variable assigned but never used
def foo():
x = 10 # Never used
# F821: Undefined name
print(undefined_var) # NameError at runtime
```
### I (isort)
```python
# Incorrect import order
from myproject import foo
import os
import sys
from third_party import bar
# Correct
import os
import sys
from third_party import bar
from myproject import foo
```
### UP (pyupgrade)
```python
# UP006: Use list instead of List from typing
from typing import List # Old
def foo() -> List[int]: # Old
pass
# New
def foo() -> list[int]: # New (Python 3.9+)
pass
# UP032: Use f-string instead of format
"{} {}".format(a, b) # Old
f"{a} {b}" # New
```
### B (flake8-bugbear)
```python
# B006: Mutable default argument
def foo(items=[]): # Bad - mutable default
items.append(1)
return items
# B008: Do not perform function call in default argument
def foo(timestamp=datetime.now()): # Bad
pass
# B011: Do not use assert False
assert False, "This should not happen" # Use raise instead
```
## Advanced Features
### Rule Explainer
```bash
# Get detailed explanation of a rule
ruff rule E501
# List all rules
ruff rule --all
# Search for rules
ruff rule --filter "import"
```
### Custom Configuration per Directory
```toml
# Root pyproject.toml
[tool.ruff]
line-length = 88
# tests/ can have different config
```
Create `tests/pyproject.toml`:
```toml
[tool.ruff]
extend = "../pyproject.toml" # Inherit root config
line-length = 120 # Override for tests
```
### Preview Mode
```bash
# Enable preview features
ruff check --preview .
ruff format --preview .
```
```toml
[tool.ruff]
preview = true
```
### Unsafe Fixes
Some fixes are marked as "unsafe" because they might change semantics:
```bash
# Include unsafe fixes
ruff check --fix --unsafe-fixes .
```
### Cache Management
```bash
# Clear Ruff cache
ruff clean
# Disable cache
ruff check --no-cache .
```
## Performance Tips
1. **Use --no-cache for CI**: Ensures fresh analysis
2. **Run in parallel**: Ruff already parallelizes internally
3. **Use specific selects**: Only enable rules you need
4. **Exclude large directories**: Skip `node_modules`, `venv`, etc.
5. **Use fix in CI**: Auto-fix what you can, then check remaining
## Troubleshooting
### Ruff Not Found
```bash
# Check installation
which ruff
ruff version
# Reinstall
uv tool uninstall ruff
uv tool install ruff
```
### Configuration Not Loaded
```bash
# Show active configuration
ruff check --show-settings .
# Use specific config file
ruff check --config path/to/pyproject.toml .
```
### Rules Not Working
```bash
# Check which rules are enabled
ruff rule --all | grep E501
# Explain specific rule
ruff rule E501
```
### Performance Issues
```bash
# Profile ruff execution
time ruff check .
# Check file discovery
ruff check --show-files .
# Clear cache
ruff clean
```
## Best Practices
1. **Start with defaults**, add rules gradually
2. **Use format + lint together** in your workflow
3. **Configure in pyproject.toml** for centralized settings
4. **Use per-file-ignores** sparingly
5. **Enable auto-fix in pre-commit** hooks
6. **Run in CI/CD** to enforce standards
7. **Use --diff in CI** to show what would change
8. **Document rule exceptions** with inline comments
9. **Keep Ruff updated** for new features and fixes
10. **Use editor integration** for real-time feedback
## Resources
- [Ruff Documentation](https://docs.astral.sh/ruff/)
- [Ruff GitHub](https://github.com/astral-sh/ruff)
- [Ruff Rules](https://docs.astral.sh/ruff/rules/)
- [Ruff Settings](https://docs.astral.sh/ruff/settings/)
- [Ruff VS Code Extension](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff)

View File

@@ -0,0 +1,731 @@
# UV: Complete Guide
UV is an extremely fast Python package and project manager, written in Rust. It's 10-100x faster than pip and replaces multiple tools in the Python ecosystem.
## Table of Contents
1. [Installation](#installation)
2. [Project Management](#project-management)
3. [Dependency Management](#dependency-management)
4. [Python Version Management](#python-version-management)
5. [Tool Management](#tool-management)
6. [The pip Interface](#the-pip-interface)
7. [Scripts](#scripts)
8. [Building and Publishing](#building-and-publishing)
9. [Configuration](#configuration)
10. [Caching](#caching)
## Installation
### Standalone Installer (Recommended)
**macOS and Linux:**
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
**Windows:**
```powershell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```
### Alternative Methods
**With Homebrew:**
```bash
brew install uv
```
**With pip:**
```bash
pip install uv
```
**With pipx:**
```bash
pipx install uv
```
**With conda:**
```bash
conda install -c conda-forge uv
```
### Upgrading UV
**If installed with standalone installer:**
```bash
uv self update
```
**If installed with pipx:**
```bash
pipx upgrade uv
```
### Shell Autocompletion
**Bash:**
```bash
echo 'eval "$(uv generate-shell-completion bash)"' >> ~/.bashrc
```
**Zsh:**
```bash
echo 'eval "$(uv generate-shell-completion zsh)"' >> ~/.zshrc
```
**Fish:**
```bash
echo 'uv generate-shell-completion fish | source' > ~/.config/fish/completions/uv.fish
```
**PowerShell:**
```powershell
Add-Content -Path $PROFILE -Value '(& uv generate-shell-completion powershell) | Out-String | Invoke-Expression'
```
## Project Management
### Creating a New Project
```bash
# Initialize a new project
uv init my-project
cd my-project
# Project structure created:
# my-project/
# ├── .gitignore
# ├── .python-version
# ├── README.md
# ├── hello.py
# └── pyproject.toml
```
### Project Structure
UV creates a standardized project structure:
**pyproject.toml:**
```toml
[project]
name = "my-project"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.11"
dependencies = []
```
### Running Project Code
```bash
# Run a Python file in the project environment
uv run python hello.py
# Run a specific command
uv run pytest
# Run with specific Python version
uv run --python 3.12 python hello.py
```
### Project Commands
```bash
# Initialize existing directory as project
uv init .
# Sync environment with dependencies
uv sync
# Lock dependencies
uv lock
# Clean up project
uv cache clean
```
## Dependency Management
### Adding Dependencies
```bash
# Add a production dependency
uv add requests
# Add multiple dependencies
uv add requests pandas numpy
# Add with version constraint
uv add "requests>=2.31.0"
uv add "pandas<3.0.0"
# Add from git repository
uv add git+https://github.com/user/repo.git
# Add from local path
uv add ../path/to/package
```
### Development Dependencies
```bash
# Add development dependencies
uv add --dev pytest black ruff
# Add to specific dependency group
uv add --group docs sphinx sphinx-rtd-theme
```
### Optional Dependencies
```bash
# Add pandas with optional dependencies
uv add pandas --optional plot
# Install with optional dependencies
uv sync --extra plot
```
### Removing Dependencies
```bash
# Remove a dependency
uv remove requests
# Remove development dependency
uv remove --dev pytest
```
### Upgrading Dependencies
```bash
# Upgrade all dependencies
uv lock --upgrade
# Upgrade specific package
uv lock --upgrade-package requests
# Sync after upgrade
uv sync
```
### Dependency Groups
```toml
# In pyproject.toml
[project.optional-dependencies]
dev = ["pytest>=7.0.0", "ruff>=0.1.0"]
docs = ["sphinx>=5.0.0", "sphinx-rtd-theme"]
test = ["pytest", "pytest-cov", "pytest-mock"]
[tool.uv]
dev-dependencies = [
"black>=23.0.0",
]
```
```bash
# Install with specific groups
uv sync --group docs
uv sync --group test
# Install only specific group
uv sync --only-group dev
# Exclude a group
uv sync --no-group docs
```
### Lockfile Management
UV creates a `uv.lock` file that ensures reproducible environments:
```bash
# Create/update lockfile
uv lock
# Sync environment to match lockfile
uv sync
# Upgrade dependencies and update lockfile
uv lock --upgrade
# Export to requirements.txt
uv export -o requirements.txt
# Export for specific groups
uv export --group docs -o requirements-docs.txt
```
## Python Version Management
### Installing Python
```bash
# Install specific Python version
uv python install 3.12
# Install multiple versions
uv python install 3.11 3.12 3.13
# Install latest patch version
uv python install 3.12
```
### Listing Python Versions
```bash
# List installed Python versions
uv python list
# List only installed versions
uv python list --only-installed
# List all available versions
uv python list --all-versions
```
### Pinning Python Version
```bash
# Pin Python version for project
uv python pin 3.12
# This creates/updates .python-version file
# Pin to specific patch version
uv python pin 3.12.4
# Pin to latest compatible version
uv python pin 3.12
```
### Using Specific Python Version
```bash
# Run with specific Python
uv run --python 3.11 python script.py
# Create venv with specific Python
uv venv --python 3.12
# Use PyPy
uv run --python [email protected] python script.py
```
### Python Discovery
UV automatically discovers Python installations from:
- System Python
- Homebrew Python
- pyenv Python
- conda Python
- Downloaded UV-managed Python
## Tool Management
UV can run and install command-line tools like `pytest`, `black`, `ruff`, etc.
### Running Tools (uvx)
```bash
# Run tool in ephemeral environment
uvx ruff check .
uvx black .
uvx pytest
# Equivalent to:
uv tool run ruff check .
```
### Installing Tools
```bash
# Install tool globally
uv tool install ruff
# Install specific version
uv tool install "ruff==0.1.0"
# Upgrade installed tool
uv tool upgrade ruff
# List installed tools
uv tool list
# Uninstall tool
uv tool uninstall ruff
```
### Tool Directory
Tools are installed in:
- **macOS/Linux**: `~/.local/share/uv/tools`
- **Windows**: `%APPDATA%\uv\tools`
## The pip Interface
UV provides a drop-in replacement for pip commands:
### Installation Commands
```bash
# Install packages
uv pip install requests
# Install from requirements.txt
uv pip install -r requirements.txt
# Install in editable mode
uv pip install -e .
# Uninstall package
uv pip uninstall requests
```
### Environment Management
```bash
# Create virtual environment
uv venv
# Create with specific Python
uv venv --python 3.12
# Create with specific name
uv venv .venv-custom
# Activate (same as standard venv)
source .venv/bin/activate # macOS/Linux
.venv\Scripts\activate # Windows
```
### Dependency Resolution
```bash
# Compile requirements
uv pip compile requirements.in -o requirements.txt
# With upgrades
uv pip compile requirements.in --upgrade
# Platform-independent (universal)
uv pip compile requirements.in --universal
# Sync environment
uv pip sync requirements.txt
```
### Listing and Freezing
```bash
# List installed packages
uv pip list
# Freeze dependencies
uv pip freeze
# Freeze to file
uv pip freeze > requirements.txt
```
## Scripts
UV can manage dependencies for single-file scripts:
### Inline Script Dependencies
```python
# script.py
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "requests",
# "pandas",
# ]
# ///
import requests
import pandas as pd
response = requests.get("https://api.example.com/data")
df = pd.DataFrame(response.json())
print(df)
```
```bash
# Add dependencies to script
uv add --script script.py requests pandas
# Run script (UV auto-installs dependencies)
uv run script.py
```
### Script Management
```bash
# Run script with inline dependencies
uv run script.py
# Run with specific Python version
uv run --python 3.12 script.py
```
## Building and Publishing
### Building Packages
```bash
# Build wheel and sdist
uv build
# Build only wheel
uv build --wheel
# Build only sdist
uv build --sdist
# Build to specific directory
uv build --out-dir dist/
```
### Publishing Packages
```bash
# Publish to PyPI
uv publish
# Publish to TestPyPI
uv publish --publish-url https://test.pypi.org/legacy/
# Publish with token
uv publish --token <token>
# Publish with username/password
uv publish --username <user> --password <pass>
```
### Package Configuration
```toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "my-package"
version = "0.1.0"
description = "My awesome package"
readme = "README.md"
requires-python = ">=3.11"
license = {text = "MIT"}
authors = [
{name = "Your Name", email = "[email protected]"}
]
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
]
dependencies = [
"requests>=2.31.0",
]
[project.urls]
Homepage = "https://github.com/user/repo"
Documentation = "https://docs.example.com"
Repository = "https://github.com/user/repo.git"
```
## Configuration
### pyproject.toml Configuration
```toml
[tool.uv]
# Python version constraints
python = ">=3.11"
# Package indexes
index-url = "https://pypi.org/simple"
extra-index-url = ["https://example.com/simple"]
# Dependency resolution
resolution = "highest" # or "lowest", "lowest-direct"
# Package sources
[tool.uv.sources]
my-package = { git = "https://github.com/user/repo.git" }
local-package = { path = "../local-package" }
# Dependency groups
dev-dependencies = [
"pytest>=7.0.0",
"ruff>=0.1.0",
]
```
### Environment Variables
```bash
# Custom Python installation directory
export UV_PYTHON_INSTALL_DIR="$HOME/.python"
# Custom cache directory
export UV_CACHE_DIR="$HOME/.cache/uv"
# Disable cache
export UV_NO_CACHE=1
# Custom index URL
export UV_INDEX_URL="https://pypi.org/simple"
# Offline mode
export UV_OFFLINE=1
```
### UV Settings File
Create `~/.config/uv/uv.toml` (or `%APPDATA%\uv\uv.toml` on Windows):
```toml
# Global UV configuration
native-tls = true
no-cache = false
cache-dir = "~/.cache/uv"
[pip]
index-url = "https://pypi.org/simple"
```
## Caching
UV uses aggressive caching for speed:
### Cache Location
- **macOS/Linux**: `~/.cache/uv`
- **Windows**: `%LOCALAPPDATA%\uv\cache`
### Cache Commands
```bash
# View cache directory
uv cache dir
# Clean cache
uv cache clean
# Clean specific package
uv cache clean requests
# Prune old cache entries
uv cache prune
```
### Cache Behavior
UV caches:
- Downloaded packages and wheels
- Built wheels from source distributions
- Package metadata
- Git repositories
- Python installations
## Performance Tips
**Parallel Downloads:**
- UV downloads packages in parallel by default
- Uses global cache to avoid re-downloads
**Universal Resolution:**
- Create platform-independent lockfiles
- Faster CI/CD across different platforms
```bash
uv lock --universal
```
**Offline Mode:**
```bash
# Use only cached packages
uv sync --offline
```
**Pre-warming Cache:**
```bash
# Download all dependencies without installing
uv sync --no-install
```
## Common Issues and Solutions
### Permission Denied
```bash
# Change ownership of UV directory
sudo chown -R $USER ~/.local/share/uv # macOS/Linux
```
### Python Version Not Found
```bash
# Install the required Python version
uv python install 3.12
# Or point to existing Python
uv venv --python /usr/bin/python3.12
```
### Dependency Conflicts
```bash
# Try different resolution strategy
uv lock --resolution lowest
# Override specific package version
uv add "package==1.0.0"
# Check for conflicts
uv pip tree
```
### Cache Issues
```bash
# Clear cache and reinstall
uv cache clean
uv sync --reinstall
```
## Best Practices
1. **Always use lockfiles** for reproducible environments
2. **Pin Python versions** in projects with `.python-version`
3. **Use dependency groups** to organize dependencies
4. **Leverage the cache** - don't disable it unless necessary
5. **Use `uv run`** instead of activating virtualenvs
6. **Export requirements.txt** for compatibility with other tools
7. **Use `uvx`** for one-off tool executions
8. **Configure in pyproject.toml** for project-specific settings
## Comparison with Other Tools
| Feature | uv | pip | poetry | conda |
|---------|-----|-----|--------|-------|
| Speed | 10-100x | 1x | 2-5x | 0.5x |
| Lockfiles | ✅ | ❌ | ✅ | ✅ |
| Python Management | ✅ | ❌ | ❌ | ✅ |
| Project Init | ✅ | ❌ | ✅ | ❌ |
| Tool Running | ✅ | ❌ | ❌ | ❌ |
| Universal Locks | ✅ | ❌ | ❌ | ✅ |
| Written in | Rust | Python | Python | Python |
| Memory Usage | Low | Medium | Medium | High |
## Resources
- [UV Documentation](https://docs.astral.sh/uv/)
- [UV GitHub](https://github.com/astral-sh/uv)
- [UV Benchmarks](https://github.com/astral-sh/uv/blob/main/BENCHMARKS.md)
- [PEP 751 - Lockfile Standard](https://peps.python.org/pep-0751/)

View File

@@ -0,0 +1,934 @@
# Advanced Workflows with UV and Ruff
Comprehensive guide for advanced use cases, monorepos, Docker integration, and production deployments.
## Table of Contents
1. [Monorepo Management](#monorepo-management)
2. [Docker Integration](#docker-integration)
3. [CI/CD Pipelines](#cicd-pipelines)
4. [Development Workflows](#development-workflows)
5. [Production Deployments](#production-deployments)
6. [Team Collaboration](#team-collaboration)
## Monorepo Management
### Workspace Setup
UV supports Cargo-style workspaces for managing multiple packages in one repository.
**Project Structure:**
```
monorepo/
├── pyproject.toml # Workspace root
├── uv.lock # Shared lockfile
├── packages/
│ ├── core/
│ │ ├── pyproject.toml
│ │ └── src/
│ ├── api/
│ │ ├── pyproject.toml
│ │ └── src/
│ └── cli/
│ ├── pyproject.toml
│ └── src/
└── apps/
└── web/
├── pyproject.toml
└── src/
```
**Root pyproject.toml:**
```toml
[tool.uv.workspace]
members = [
"packages/*",
"apps/*"
]
[tool.uv]
dev-dependencies = [
"pytest>=7.0.0",
"ruff>=0.1.0",
]
```
**Package pyproject.toml (core):**
```toml
[project]
name = "myproject-core"
version = "0.1.0"
dependencies = [
"pydantic>=2.0.0",
]
[tool.uv.sources]
# No sources needed - uses workspace
```
**Package pyproject.toml (api):**
```toml
[project]
name = "myproject-api"
version = "0.1.0"
dependencies = [
"fastapi>=0.100.0",
"myproject-core", # Workspace dependency
]
[tool.uv.sources]
myproject-core = { workspace = true }
```
### Working with Workspaces
```bash
# Install all workspace packages
uv sync
# Run commands from root
uv run --package myproject-api python -m uvicorn main:app
# Run tests for specific package
uv run --package myproject-core pytest
# Add dependency to specific package
cd packages/api
uv add requests
```
### Shared Ruff Configuration
**Root pyproject.toml:**
```toml
[tool.ruff]
line-length = 88
target-version = "py311"
[tool.ruff.lint]
select = ["E", "W", "F", "I", "B"]
```
**Per-package overrides:**
```toml
# packages/cli/pyproject.toml
[tool.ruff.lint.per-file-ignores]
"src/cli/*.py" = ["T201"] # Allow prints in CLI
```
### Monorepo Scripts
**Makefile:**
```makefile
.PHONY: install lint format test
install:
\tuv sync
lint:
\truff check .
format:
\truff format .
test:
\tuv run pytest
# Per-package commands
test-core:
\tuv run --package myproject-core pytest
test-api:
\tuv run --package myproject-api pytest
```
## Docker Integration
### Multi-Stage Dockerfile
```dockerfile
# syntax=docker/dockerfile:1
# Stage 1: Build stage with UV
FROM python:3.12-slim AS builder
# Install UV
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
# Set working directory
WORKDIR /app
# Copy dependency files
COPY pyproject.toml uv.lock ./
# Install dependencies
RUN uv sync --frozen --no-dev --no-cache
# Stage 2: Runtime stage
FROM python:3.12-slim
WORKDIR /app
# Copy virtual environment from builder
COPY --from=builder /app/.venv /app/.venv
# Copy application code
COPY . .
# Set PATH to use virtual environment
ENV PATH="/app/.venv/bin:$PATH"
# Run application
CMD ["python", "-m", "myapp"]
```
### Development Dockerfile
```dockerfile
FROM python:3.12-slim
# Install UV
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app
# Copy dependency files
COPY pyproject.toml uv.lock ./
# Install all dependencies (including dev)
RUN uv sync --frozen --no-cache
# Copy code
COPY . .
# Development server
CMD ["uv", "run", "python", "-m", "uvicorn", "main:app", "--reload", "--host", "0.0.0.0"]
```
### Docker Compose
```yaml
version: '3.8'
services:
app:
build:
context: .
dockerfile: Dockerfile.dev
volumes:
- .:/app
- uv-cache:/root/.cache/uv
ports:
- "8000:8000"
environment:
- PYTHONUNBUFFERED=1
command: uv run python -m uvicorn main:app --reload --host 0.0.0.0
worker:
build:
context: .
dockerfile: Dockerfile.dev
volumes:
- .:/app
- uv-cache:/root/.cache/uv
command: uv run python -m celery worker
volumes:
uv-cache:
```
### Optimized Production Dockerfile
```dockerfile
FROM python:3.12-slim AS builder
# Install UV
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app
# Enable bytecode compilation
ENV UV_COMPILE_BYTECODE=1
# Copy files
COPY pyproject.toml uv.lock ./
COPY src ./src
# Install dependencies and application
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-dev
FROM python:3.12-slim
WORKDIR /app
# Copy only necessary files
COPY --from=builder /app/.venv /app/.venv
COPY --from=builder /app/src /app/src
# Create non-root user
RUN useradd -m -u 1000 appuser && \
chown -R appuser:appuser /app
USER appuser
ENV PATH="/app/.venv/bin:$PATH"
CMD ["python", "-m", "myapp"]
```
## CI/CD Pipelines
### GitHub Actions
**Complete Workflow:**
```yaml
name: CI/CD Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
quality:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install UV
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Set up Python
run: uv python install 3.12
- name: Install dependencies
run: uv sync --frozen
- name: Lint with Ruff
run: |
uv run ruff check .
uv run ruff format --check .
- name: Type check with mypy
run: uv run mypy src/
- name: Security check
run: uv run ruff check --select S .
test:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ['3.11', '3.12']
steps:
- uses: actions/checkout@v4
- name: Install UV
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Set up Python
run: uv python install ${{ matrix.python-version }}
- name: Install dependencies
run: uv sync --frozen
- name: Run tests
run: uv run pytest --cov=src --cov-report=xml
- name: Upload coverage
uses: codecov/codecov-action@v3
with:
files: ./coverage.xml
build:
needs: [quality, test]
runs-on: ubuntu-latest
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Install UV
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Build package
run: uv build
- name: Publish to PyPI
env:
UV_PUBLISH_TOKEN: ${{ secrets.PYPI_TOKEN }}
run: uv publish
```
### GitLab CI
```yaml
stages:
- lint
- test
- build
- deploy
variables:
UV_CACHE_DIR: ${CI_PROJECT_DIR}/.cache/uv
cache:
paths:
- .cache/uv
- .venv
before_script:
- curl -LsSf https://astral.sh/uv/install.sh | sh
- export PATH="$HOME/.local/bin:$PATH"
- uv sync --frozen
lint:
stage: lint
script:
- uv run ruff check .
- uv run ruff format --check .
test:
stage: test
parallel:
matrix:
- PYTHON_VERSION: ['3.11', '3.12']
script:
- uv python install $PYTHON_VERSION
- uv sync --frozen
- uv run pytest --cov=src
build:
stage: build
only:
- main
script:
- uv build
artifacts:
paths:
- dist/
deploy:
stage: deploy
only:
- main
script:
- uv publish --token $PYPI_TOKEN
```
### Circle CI
```yaml
version: 2.1
executors:
python-executor:
docker:
- image: python:3.12-slim
jobs:
lint:
executor: python-executor
steps:
- checkout
- run:
name: Install UV
command: curl -LsSf https://astral.sh/uv/install.sh | sh
- run:
name: Lint
command: |
export PATH="$HOME/.local/bin:$PATH"
uv sync --frozen
uv run ruff check .
uv run ruff format --check .
test:
executor: python-executor
steps:
- checkout
- run:
name: Install UV
command: curl -LsSf https://astral.sh/uv/install.sh | sh
- run:
name: Test
command: |
export PATH="$HOME/.local/bin:$PATH"
uv sync --frozen
uv run pytest
workflows:
main:
jobs:
- lint
- test
- deploy:
requires:
- lint
- test
filters:
branches:
only: main
```
## Development Workflows
### Pre-commit Integration
**.pre-commit-config.yaml:**
```yaml
repos:
# Ruff linting and formatting
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.12.8
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format
# Type checking
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.8.0
hooks:
- id: mypy
additional_dependencies: [types-all]
# Security
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.12.8
hooks:
- id: ruff
name: ruff-security
args: [--select, S]
# Standard hooks
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
```
**Install hooks:**
```bash
uv add --dev pre-commit
uv run pre-commit install
```
### VS Code Integration
**.vscode/settings.json:**
```json
{
"[python]": {
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.fixAll.ruff": "explicit",
"source.organizeImports.ruff": "explicit"
},
"editor.defaultFormatter": "charliermarsh.ruff"
},
"ruff.lint.args": ["--config=pyproject.toml"],
"ruff.format.args": ["--config=pyproject.toml"],
"python.testing.pytestEnabled": true,
"python.testing.pytestArgs": ["tests"]
}
```
**.vscode/tasks.json:**
```json
{
"version": "2.0.0",
"tasks": [
{
"label": "UV: Sync",
"type": "shell",
"command": "uv sync",
"group": "build"
},
{
"label": "Ruff: Check",
"type": "shell",
"command": "uv run ruff check .",
"group": "test"
},
{
"label": "Ruff: Format",
"type": "shell",
"command": "uv run ruff format .",
"group": "build"
},
{
"label": "Test",
"type": "shell",
"command": "uv run pytest",
"group": {
"kind": "test",
"isDefault": true
}
}
]
}
```
### Development Scripts
**justfile (like Makefile but better):**
```justfile
# Install dependencies
install:
uv sync
# Run development server
dev:
uv run python -m uvicorn main:app --reload
# Lint and format
lint:
uv run ruff check --fix .
uv run ruff format .
# Type check
typecheck:
uv run mypy src/
# Run tests
test:
uv run pytest -v
# Run tests with coverage
test-cov:
uv run pytest --cov=src --cov-report=html
# Security check
security:
uv run ruff check --select S .
# Update dependencies
update:
uv lock --upgrade
uv sync
# Clean caches
clean:
uv cache clean
ruff clean
find . -type d -name __pycache__ -exec rm -rf {} +
find . -type d -name .pytest_cache -exec rm -rf {} +
# All quality checks
check: lint typecheck security test
```
## Production Deployments
### AWS Lambda
```dockerfile
FROM public.ecr.aws/lambda/python:3.12
# Install UV
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
# Copy application
COPY pyproject.toml uv.lock ./
COPY src ${LAMBDA_TASK_ROOT}/src
# Install dependencies
RUN uv sync --frozen --no-dev --no-cache
# Lambda handler
CMD ["src.handler.lambda_handler"]
```
### Google Cloud Run
```dockerfile
FROM python:3.12-slim
# Install UV
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app
COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-dev --no-cache
COPY . .
ENV PORT=8080
CMD exec uv run python -m uvicorn main:app --host 0.0.0.0 --port ${PORT}
```
### Kubernetes Deployment
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 3
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: myapp:latest
ports:
- containerPort: 8000
env:
- name: PYTHONUNBUFFERED
value: "1"
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "200m"
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /ready
port: 8000
initialDelaySeconds: 5
periodSeconds: 5
```
### Helm Chart
**values.yaml:**
```yaml
image:
repository: myapp
tag: latest
pullPolicy: IfNotPresent
replicaCount: 3
service:
type: ClusterIP
port: 80
targetPort: 8000
env:
- name: PYTHONUNBUFFERED
value: "1"
- name: LOG_LEVEL
value: "info"
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "200m"
```
## Team Collaboration
### Shared Configuration
**pyproject.toml:**
```toml
[project]
name = "myproject"
version = "0.1.0"
requires-python = ">=3.11"
[tool.uv]
# Shared dev dependencies
dev-dependencies = [
"pytest>=7.0.0",
"pytest-cov>=4.0.0",
"ruff>=0.1.0",
"mypy>=1.8.0",
]
[tool.ruff]
# Team-wide code style
line-length = 88
target-version = "py311"
[tool.ruff.lint]
# Agreed upon rules
select = ["E", "W", "F", "I", "B", "UP"]
ignore = []
[tool.ruff.lint.per-file-ignores]
# Consistent exceptions
"tests/*" = ["S101"]
"__init__.py" = ["F401"]
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_functions = ["test_*"]
[tool.mypy]
python_version = "3.11"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
```
### Code Review Checklist
**CONTRIBUTING.md:**
```markdown
## Code Review Checklist
Before submitting a PR:
- [ ] Run `uv run ruff check --fix .`
- [ ] Run `uv run ruff format .`
- [ ] Run `uv run mypy src/`
- [ ] Run `uv run pytest`
- [ ] Update `uv.lock` if dependencies changed
- [ ] Add tests for new features
- [ ] Update documentation
- [ ] Ensure CI passes
```
### Onboarding Guide
**README.md:**
```markdown
## Getting Started
### Prerequisites
- Python 3.11+
- UV package manager
### Setup
1. Install UV:
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
2. Clone repository:
```bash
git clone https://github.com/org/repo.git
cd repo
```
3. Install dependencies:
```bash
uv sync
```
4. Run tests:
```bash
uv run pytest
```
5. Start development server:
```bash
uv run python -m uvicorn main:app --reload
```
### Development Commands
- `uv run ruff check --fix .` - Lint code
- `uv run ruff format .` - Format code
- `uv run pytest` - Run tests
- `uv run mypy src/` - Type check
### Adding Dependencies
```bash
uv add package-name
```
### Updating Dependencies
```bash
uv lock --upgrade
uv sync
```
```
## Performance Optimization
### Caching Strategies
```bash
# Pre-warm cache in CI
- name: Cache UV
uses: actions/cache@v3
with:
path: ~/.cache/uv
key: ${{ runner.os }}-uv-${{ hashFiles('uv.lock') }}
# Use frozen lockfile for reproducibility
uv sync --frozen
# Offline mode for airgapped environments
uv sync --offline
```
### Build Optimization
```bash
# Compile bytecode
export UV_COMPILE_BYTECODE=1
# Use system Python if available
export UV_SYSTEM_PYTHON=1
# Skip cache in CI
export UV_NO_CACHE=1
```
## Troubleshooting
### Common Issues
**Slow dependency resolution:**
```bash
# Use frozen lockfile
uv sync --frozen
# Clear cache
uv cache clean
```
**Out of disk space:**
```bash
# Prune old cache entries
uv cache prune
# Check cache size
du -sh ~/.cache/uv
```
**Permission errors:**
```bash
# Fix ownership
sudo chown -R $USER ~/.cache/uv
sudo chown -R $USER ~/.local/share/uv
```
## Resources
- [UV Documentation](https://docs.astral.sh/uv/)
- [Ruff Documentation](https://docs.astral.sh/ruff/)
- [Docker Best Practices](https://docs.docker.com/develop/dev-best-practices/)
- [GitHub Actions Documentation](https://docs.github.com/en/actions)