--- title: "Fabric: High-Level SSH Command Execution and Deployment" library_name: fabric pypi_package: fabric category: ssh-automation python_compatibility: "3.6+" last_updated: "2025-11-02" official_docs: "https://docs.fabfile.org" official_repository: "https://github.com/fabric/fabric" maintenance_status: "stable" --- # Fabric: High-Level SSH Command Execution and Deployment ## Core Purpose Fabric is a high-level Python library designed to execute shell commands remotely over SSH, yielding useful Python objects in return. It solves the problem of programmatic remote server management and deployment automation by providing a Pythonic interface to SSH operations. ### What Problem Does Fabric Solve? Fabric eliminates the need to manually SSH into multiple servers and run commands repeatedly. It provides: 1. **Programmatic SSH Execution**: Execute commands on remote servers from Python code 2. **Multi-Host Management**: Run commands across multiple servers in parallel or serially 3. **File Transfer**: Upload and download files over SSH/SFTP 4. **Deployment Automation**: Orchestrate complex deployment workflows 5. **Task Definition**: Define reusable deployment tasks with the `@task` decorator 6. **Connection Management**: Handle SSH authentication, connection pooling, and error handling ### When Should You Use Fabric? **Use Fabric when:** - You need to execute commands on **remote servers** over SSH - You're automating deployment processes (copying files, restarting services, running migrations) - You need to manage multiple servers programmatically - You want to define reusable deployment tasks in Python - You're building continuous integration/deployment pipelines - You need more than just subprocess (which only works locally) **Use subprocess when:** - You only need to run commands on your **local machine** - You don't need SSH connectivity to remote hosts - Your automation is purely local process execution **Use Ansible when:** - You need declarative configuration management across many hosts - You require idempotency guarantees - You need a large ecosystem of pre-built modules - Your team prefers YAML over Python - You're managing infrastructure state, not just running scripts **Use Paramiko directly when:** - You need low-level SSH protocol control - You're building custom SSH clients or servers - Fabric's higher-level abstractions are too restrictive ## Architecture and Dependencies Fabric is built on two core libraries: 1. **Invoke** (>=2.0): Subprocess command execution and command-line task features 2. **Paramiko** (>=2.4): SSH protocol implementation Fabric extends their APIs to provide: - Remote execution via `Connection.run()` - File transfer via `Connection.put()` and `Connection.get()` - Sudo support via `Connection.sudo()` - Group operations via `SerialGroup` and `ThreadingGroup` ## Python Version Compatibility | Python Version | Fabric 2.x | Fabric 3.x | Status | | -------------- | ---------- | ---------- | ---------------- | | 3.6 | ✓ | ✓ | Minimum version | | 3.7 | ✓ | ✓ | Supported | | 3.8 | ✓ | ✓ | Supported | | 3.9 | ✓ | ✓ | Supported | | 3.10 | ✓ | ✓ | Supported | | 3.11 | ✓ | ✓ | Supported | | 3.12 | ? | ? | Likely supported | | 3.13 | ? | ? | Likely supported | | 3.14 | ? | ? | Unknown | **Note**: Fabric follows semantic versioning. Fabric 2.x and 3.x share similar APIs with minor breaking changes. Fabric 1.x (legacy) is incompatible with 2.x/3.x. ### Fabric Version Differences - **Fabric 1.x** (legacy): Python 2.7 only, different API, no longer maintained - **Fabric 2.x**: Modern API, Python 3.6+, built on Invoke/Paramiko - **Fabric 3.x**: Current stable, incremental improvements over 2.x, Python 3.6+ ## Installation ```bash # Standard installation pip install fabric # For migration from Fabric 1.x (side-by-side installation) pip install fabric2 # Development installation pip install -e git+https://github.com/fabric/fabric # With pytest fixtures support pip install fabric[pytest] ``` ## Core Usage Patterns ### 1. Basic Remote Command Execution **@** ```python from fabric import Connection # Simple connection and command execution result = Connection('web1.example.com').run('uname -s', hide=True) print(f"Ran {result.command!r} on {result.connection.host}") print(f"Exit code: {result.exited}") print(f"Output: {result.stdout.strip()}") ``` **Result object attributes:** - `result.stdout`: Command output - `result.stderr`: Error output - `result.exited`: Exit code - `result.ok`: Boolean (True if exit code was 0) - `result.command`: The command that was run - `result.connection`: The Connection object used ### 2. Connection with Authentication **@** ```python from fabric import Connection # User@host:port format c = Connection('deploy@web1.example.com:2202') # Or explicit parameters c = Connection( host='web1.example.com', user='deploy', port=2202, connect_kwargs={ "key_filename": "/path/to/private/key", # or "password": "mypassword" } ) # Execute commands c.run('whoami') c.run('ls -la /var/www') ``` ### 3. File Transfer Operations **@** ```python from fabric import Connection c = Connection('web1') # Upload file result = c.put('myfiles.tgz', remote='/opt/mydata/') print(f"Uploaded {result.local} to {result.remote}") # Download file c.get('/var/log/app.log', local='./logs/') # Upload and extract c.put('myfiles.tgz', '/opt/mydata') c.run('tar -C /opt/mydata -xzvf /opt/mydata/myfiles.tgz') ``` ### 4. Sudo Operations **@** ```python import getpass from fabric import Connection, Config # Configure sudo password sudo_pass = getpass.getpass("What's your sudo password?") config = Config(overrides={'sudo': {'password': sudo_pass}}) c = Connection('db1', config=config) # Run with sudo using helper method c.sudo('whoami', hide='stderr') # Output: root c.sudo('useradd mydbuser') c.run('id -u mydbuser') # Verify user created # Alternative: Manual sudo with password responder from invoke import Responder sudopass = Responder( pattern=r'\[sudo\] password:', response=f'{sudo_pass}\n', ) c.run('sudo whoami', pty=True, watchers=[sudopass]) ``` ### 5. Multi-Host Execution (Serial) **@** ```python from fabric import SerialGroup as Group # Execute on multiple hosts serially pool = Group('web1', 'web2', 'web3') # Run command on all hosts results = pool.run('uname -s') for connection, result in results.items(): print(f"{connection.host}: {result.stdout.strip()}") # File operations on all hosts pool.put('myfiles.tgz', '/opt/mydata') pool.run('tar -C /opt/mydata -xzvf /opt/mydata/myfiles.tgz') ``` ### 6. Multi-Host Execution (Parallel) **@** ```python from fabric import ThreadingGroup as Group # Execute on multiple hosts in parallel pool = Group('web1', 'web2', 'web3', 'web4', 'web5') # Run command concurrently results = pool.run('hostname') # Process results for connection, result in results.items(): print(f"{connection.host}: {result.stdout.strip()}") ``` ### 7. Defining Reusable Tasks **@** ```python from fabric import task @task def deploy(c): """Deploy application to remote server""" code_dir = "/srv/django/myproject" # Check if directory exists if not c.run(f"test -d {code_dir}", warn=True): # Clone repository c.run(f"git clone user@vcshost:/path/to/repo/.git {code_dir}") # Update code c.run(f"cd {code_dir} && git pull") # Restart application c.run(f"cd {code_dir} && touch app.wsgi") @task def update_servers(c): """Run system updates""" c.sudo('apt update') c.sudo('apt upgrade -y') c.sudo('systemctl restart nginx') # Use with fab command: # fab -H web1,web2,web3 deploy ``` ### 8. Task Composition and Workflow **@** ```python from fabric import task from invoke import Exit from invocations.console import confirm @task def test(c): """Run local tests""" result = c.local("./manage.py test my_app", warn=True) if not result and not confirm("Tests failed. Continue anyway?"): raise Exit("Aborting at user request.") @task def commit(c): """Commit changes""" c.local("git add -p && git commit") @task def push(c): """Push to remote""" c.local("git push") @task def prepare_deploy(c): """Prepare for deployment""" test(c) commit(c) push(c) @task(hosts=['web1.example.com', 'web2.example.com']) def deploy(c): """Deploy to remote servers""" code_dir = "/srv/django/myproject" c.run(f"cd {code_dir} && git pull") c.run(f"cd {code_dir} && touch app.wsgi") # Usage: # fab prepare_deploy deploy ``` ### 9. Connection with Gateway/Bastion Host **@** ```python from fabric import Connection # Connect to internal host through gateway gateway = Connection('bastion.example.com') c = Connection('internal-db.local', gateway=gateway) # Now all operations go through the gateway c.run('hostname') c.run('df -h') ``` ### 10. Error Handling and Conditional Logic **@** ```python from fabric import SerialGroup as Group def upload_and_unpack(c): """Upload file only if it doesn't exist""" # Check if file exists (don't fail on non-zero exit) if c.run('test -f /opt/mydata/myfile', warn=True).failed: c.put('myfiles.tgz', '/opt/mydata') c.run('tar -C /opt/mydata -xzvf /opt/mydata/myfiles.tgz') else: print(f"File already exists on {c.host}, skipping upload") # Apply to group for connection in Group('web1', 'web2', 'web3'): upload_and_unpack(connection) ``` ## Real-World Integration Patterns ### Pattern 1: Django/Web Application Deployment **@** ```python from fabric import task, Connection @task def deploy_django(c): """Deploy Django application""" # Pull latest code c.run('cd /var/www/myapp && git pull origin main') # Install dependencies c.run('cd /var/www/myapp && pip install -r requirements.txt') # Run migrations c.run('cd /var/www/myapp && python manage.py migrate') # Collect static files c.run('cd /var/www/myapp && python manage.py collectstatic --noinput') # Restart services c.sudo('systemctl restart gunicorn') c.sudo('systemctl restart nginx') ``` ### Pattern 2: Database Backup and Restore **@Exa:fabric deployment examples** ```python from fabric import task from datetime import datetime @task def backup_database(c): """Create database backup""" timestamp = datetime.now().strftime('%Y%m%d_%H%M%S') backup_file = f"backup_{timestamp}.sql" # Create backup c.run(f"mysqldump -u dbuser -p database_name > /backups/{backup_file}") # Compress backup c.run(f"gzip /backups/{backup_file}") # Download backup c.get(f"/backups/{backup_file}.gz", local=f"./backups/{backup_file}.gz") print(f"Backup completed: {backup_file}.gz") ``` ### Pattern 3: Log Collection and Analysis ```python from fabric import SerialGroup as Group def collect_logs(c): """Collect application logs from remote server""" hostname = c.run('hostname', hide=True).stdout.strip() c.get('/var/log/app/error.log', local=f'logs/{hostname}_error.log') c.get('/var/log/app/access.log', local=f'logs/{hostname}_access.log') # Collect from all servers pool = Group('web1', 'web2', 'web3', 'web4') for conn in pool: collect_logs(conn) ``` ### Pattern 4: Service Health Check ```python from fabric import task, SerialGroup as Group @task def health_check(c): """Check service health across servers""" servers = Group('web1', 'web2', 'db1', 'cache1') for conn in servers: print(f"\nChecking {conn.host}...") # Check disk space result = conn.run("df -h / | tail -n1 | awk '{print $5}'", hide=True) disk_usage = result.stdout.strip() print(f" Disk usage: {disk_usage}") # Check memory result = conn.run("free -m | grep Mem | awk '{print $3/$2 * 100.0}'", hide=True) mem_usage = float(result.stdout.strip()) print(f" Memory usage: {mem_usage:.1f}%") # Check service status result = conn.run("systemctl is-active nginx", warn=True, hide=True) service_status = result.stdout.strip() print(f" Nginx status: {service_status}") ``` ## When NOT to Use Fabric ### 1. Simple Local Automation ```python # DON'T use Fabric for local operations from fabric import Connection c = Connection('localhost') c.run('ls -la') # DO use subprocess instead import subprocess subprocess.run(['ls', '-la']) ``` ### 2. Large-Scale Infrastructure Management If you need to manage hundreds of servers with complex configuration requirements, **Ansible** or **SaltStack** provide better: - Declarative configuration syntax - Idempotency guarantees - Large module ecosystem - Built-in inventory management - Role-based organization ### 3. Container Orchestration For Docker/Kubernetes deployments, use native orchestration tools: - Docker Compose - Kubernetes manifests - Helm charts - ArgoCD ### 4. Configuration Drift Detection Fabric executes commands but doesn't track state. For configuration management with drift detection, use: - Ansible - Chef - Puppet - Terraform (for infrastructure) ### 5. Windows Remote Management For Windows automation, use: - PowerShell Remoting - WinRM libraries - Ansible (with WinRM) ## Decision Matrix | Scenario | Fabric | Ansible | Subprocess | Paramiko | | ---------------------------- | ------ | ------- | ---------- | -------- | | Deploy to 1-10 Linux servers | ✓✓ | ✓ | ✗ | ✓ | | Deploy to 100+ servers | ✓ | ✓✓ | ✗ | ✗ | | Run local commands | ✗ | ✗ | ✓✓ | ✗ | | Configuration management | ✗ | ✓✓ | ✗ | ✗ | | Ad-hoc SSH automation | ✓✓ | ✓ | ✗ | ✓ | | Custom SSH protocol work | ✗ | ✗ | ✗ | ✓✓ | | Python-first workflow | ✓✓ | ✗ | ✓✓ | ✓✓ | | YAML-first workflow | ✗ | ✓✓ | ✗ | ✗ | | File transfer over SSH | ✓✓ | ✓ | ✗ | ✓ | | Parallel execution | ✓✓ | ✓✓ | ✓ | ✗ | | Windows targets | ✗ | ✓✓ | ✓ | ✗ | **Legend**: ✓✓ = Excellent fit, ✓ = Suitable, ✗ = Not appropriate ## Common Gotchas and Solutions ### 1. Separate Shell Sessions **@** ```python # WRONG: cd doesn't persist across run() calls @task def deploy(c): c.run("cd /path/to/application") c.run("./update.sh") # This runs in home directory! # CORRECT: Use shell && operator @task def deploy(c): c.run("cd /path/to/application && ./update.sh") # ALTERNATIVE: Use absolute paths @task def deploy(c): c.run("/path/to/application/update.sh") ``` ### 2. Sudo Password Prompts ```python # WRONG: Sudo hangs waiting for password c.run('sudo systemctl restart nginx') # CORRECT: Use pty=True and watchers from invoke import Responder sudopass = Responder( pattern=r'\[sudo\] password:', response='mypassword\n', ) c.run('sudo systemctl restart nginx', pty=True, watchers=[sudopass]) # BETTER: Use Connection.sudo() helper c.sudo('systemctl restart nginx') # Uses configured password ``` ### 3. Connection Reuse ```python # INEFFICIENT: Creates new connection each time for i in range(10): Connection('web1').run(f'echo {i}') # EFFICIENT: Reuse connection c = Connection('web1') for i in range(10): c.run(f'echo {i}') ``` ## Testing with Fabric **@** ```python from fabric.testing import MockRemote def test_deployment(): """Test deployment logic without real SSH""" with MockRemote(commands={ 'test -d /srv/app': (1, '', ''), # Exit 1 = doesn't exist 'git clone ...': (0, 'Cloning...', ''), 'cd /srv/app && git pull': (0, 'Already up to date', ''), }) as remote: c = remote.connection deploy(c) # Verify commands were called assert 'git clone' in remote.calls ``` ## Migration from Fabric 1.x to 2.x/3.x **@** Key changes: 1. No more `env` global dictionary 2. Tasks must accept `Connection` or `Context` as first argument 3. No more `@hosts` decorator (use `@task(hosts=[...])`) 4. `run()` is now `c.run()` on Connection object 5. Import from `fabric` not `fabric.api` ```python # Fabric 1.x (OLD) from fabric.api import env, run, task env.hosts = ['web1', 'web2'] @task def deploy(): run('git pull') # Fabric 2.x/3.x (NEW) from fabric import task @task(hosts=['web1', 'web2']) def deploy(c): c.run('git pull') ``` ## Performance Considerations 1. **Parallel vs Serial Execution**: - Use `ThreadingGroup` for I/O-bound tasks (network operations) - Consider `SerialGroup` for order-dependent operations - Default thread pool size is 10 connections 2. **Connection Pooling**: - Reuse `Connection` objects when possible - Close connections explicitly with `c.close()` or use context managers 3. **Output Buffering**: - Use `hide=True` to suppress output and improve performance - Large output can slow down execution ## Resources and Examples ### Official Documentation - Main site: @ - Getting Started: @ - API Reference: @ - FAQ: @ - Upgrading Guide: @ ### GitHub Examples - Official repository: @ - Example fabfiles: @ - Integration tests: @ ### Community Resources - Fabricio (Docker automation): @ - Linux Journal tutorial: @ - Medium tutorials: @ ## Summary **Use Fabric when you need to:** - Execute commands on remote Linux servers via SSH - Automate deployment of web applications - Manage small to medium server fleets (1-50 servers) - Transfer files between local and remote systems - Define reusable deployment tasks in Python - Integrate deployment into CI/CD pipelines **Don't use Fabric when:** - You only need local command execution (use subprocess) - You're managing large infrastructure (>100 servers, use Ansible) - You need configuration drift detection (use Ansible/Chef/Puppet) - You're working with Windows servers primarily - You need declarative infrastructure as code (use Terraform/Ansible) Fabric excels at programmatic SSH automation for deployment workflows where you want the full power of Python combined with remote execution capabilities. It's the sweet spot between low-level Paramiko and heavyweight configuration management tools.