zhongwei/gh-basher83-lunar-claude-plugins-homelab-netbox-powerdns-integration
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 18:00:21 +08:00
commit 26377bd9be
20 changed files with 8845 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

469
skills/netbox-powerdns-integration/SKILL.md Normal file
View File

@@ -0,0 +1,469 @@
---
name: netbox-powerdns-integration
description: NetBox IPAM and PowerDNS integration for automated DNS record management.
---
# NetBox PowerDNS Integration
Expert guidance for implementing NetBox as your source of truth for infrastructure documentation and
automating DNS record management with PowerDNS.
## Quick Start
### Common Tasks
**Query NetBox API:**
```bash
# List all sites
./tools/netbox_api_client.py sites list
# Get device details
./tools/netbox_api_client.py devices get --name foxtrot
# List VMs in cluster
./tools/netbox_api_client.py vms list --cluster matrix
# Query IPs
./tools/netbox_api_client.py ips query --dns-name docker-01
```
**Create VM in NetBox:**
```bash
# Create VM with auto-assigned IP
./tools/netbox_vm_create.py --name docker-02 --cluster matrix --vcpus 4 --memory 8192
# Create VM with specific IP
./tools/netbox_vm_create.py --name k8s-01-master --cluster matrix --ip 192.168.3.50/24
```
**IPAM Queries:**
```bash
# Get available IPs
./tools/netbox_ipam_query.py available --prefix 192.168.3.0/24
# Check prefix utilization
./tools/netbox_ipam_query.py utilization --site matrix
# View IP assignments
./tools/netbox_ipam_query.py assignments --prefix 192.168.3.0/24
```
**Validate DNS Naming:**
```bash
./tools/validate_dns_naming.py --name "docker-01-nexus.spaceships.work"
```
**Deploy from NetBox Inventory:**
```bash
cd ansible && uv run ansible-playbook -i tools/netbox-dynamic-inventory.yml deploy-from-netbox.yml
```
## When to Use This Skill
Activate this skill when:
- **Querying NetBox API** - Sites, devices, VMs, IPs, prefixes, VLANs
- **Setting up NetBox IPAM** - Prefixes, IP management, VRFs
- **Implementing automated DNS** - PowerDNS sync plugin configuration
- **Creating DNS naming conventions** - `service-NN-purpose.domain` pattern
- **Managing VMs in NetBox** - Creating, updating, IP assignment
- **Using Terraform with NetBox** - Provider configuration and resources
- **Setting up Ansible dynamic inventory** - NetBox as inventory source
- **Troubleshooting NetBox-PowerDNS sync** - Tag matching, zone configuration
- **Migrating to NetBox** - From manual DNS or spreadsheet tracking
- **Infrastructure documentation** - Using NetBox as source of truth
## Core Workflows
### 1. NetBox API Usage
**Query infrastructure data:**
```python
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = ["pynetbox>=7.0.0", "infisical-python>=2.3.3"]
# ///
import pynetbox
from infisical import InfisicalClient
# Get token from Infisical
client = InfisicalClient()
token = client.get_secret(
secret_name="NETBOX_API_TOKEN",
project_id="7b832220-24c0-45bc-a5f1-ce9794a31259",
environment="prod",
path="/matrix"
).secret_value
# Connect to NetBox
nb = pynetbox.api('https://netbox.spaceships.work', token=token)
# Query devices in Matrix cluster
site = nb.dcim.sites.get(slug='matrix')
devices = nb.dcim.devices.filter(site='matrix')
for device in devices:
print(f"{device.name}: {device.primary_ip4.address if device.primary_ip4 else 'No IP'}")
```
See [reference/netbox-api-guide.md](reference/netbox-api-guide.md) for complete API reference.
### 2. DNS Naming Convention
This infrastructure uses the pattern: `<service>-<number>-<purpose>.<domain>`
**Examples:**
- `docker-01-nexus.spaceships.work` - Docker host #1 running Nexus
- `proxmox-foxtrot-mgmt.spaceships.work` - Proxmox node Foxtrot management interface
- `k8s-01-master.spaceships.work` - Kubernetes cluster master node #1
**Implementation:**
```python
# tools/validate_dns_naming.py validates this pattern
pattern = r'^[a-z0-9-]+-\d{2}-[a-z0-9-]+\.[a-z0-9.-]+$'
```
See [workflows/naming-conventions.md](workflows/naming-conventions.md) for complete rules.
### 3. NetBox + PowerDNS Sync Setup
#### Step 1: Install Plugin
```bash
# In NetBox virtualenv
pip install netbox-powerdns-sync
```
#### Step 2: Configure Plugin
```python
# /opt/netbox/netbox/netbox/configuration.py
PLUGINS = ['netbox_powerdns_sync']
PLUGINS_CONFIG = {
"netbox_powerdns_sync": {
"powerdns_managed_record_comment": "netbox-managed",
"post_save_enabled": True, # Real-time sync
},
}
```
#### Step 3: Create Zones in NetBox
Configure zones with:
- Zone name (e.g., `spaceships.work`)
- PowerDNS server connection
- Tag matching rules (e.g., `production-dns`)
- DNS name generation method
See [reference/sync-plugin-reference.md](reference/sync-plugin-reference.md) for details.
### 4. Terraform Integration
**Provider Setup:**
```hcl
terraform {
required_providers {
netbox = {
source = "e-breuninger/netbox"
version = "~> 5.0.0"
}
}
}
provider "netbox" {
server_url = "https://netbox.spaceships.work"
api_token = var.netbox_api_token
}
```
**Create IP with Auto-DNS:**
```hcl
resource "netbox_ip_address" "docker_host" {
ip_address = "192.168.1.100/24"
dns_name = "docker-01-nexus.spaceships.work"
description = "Docker host for Nexus registry"
tags = [
"terraform",
"production-dns" # Triggers auto DNS sync
]
}
```
DNS records created automatically by plugin!
See [reference/terraform-provider-guide.md](reference/terraform-provider-guide.md) and [examples/netbox-terraform-provider.tf](examples/netbox-terraform-provider.tf).
### 5. Ansible Dynamic Inventory
**Use NetBox as Inventory Source:**
```yaml
# tools/netbox-dynamic-inventory.yml
plugin: netbox.netbox.nb_inventory
api_endpoint: https://netbox.spaceships.work
token: !vault |
$ANSIBLE_VAULT;...
group_by:
- device_roles
- tags
```
**Deploy Using NetBox Data:**
```bash
ansible-playbook -i tools/netbox-dynamic-inventory.yml deploy-from-netbox.yml
```
See [workflows/ansible-dynamic-inventory.md](workflows/ansible-dynamic-inventory.md).
## Architecture Reference
### DNS Automation Flow
```text
1. Create/Update resource in NetBox
└→ IP Address with dns_name and tags
2. NetBox PowerDNS Sync Plugin activates
└→ Matches IP to zone based on tags
└→ Generates DNS records
3. PowerDNS API called
└→ A record: docker-01-nexus.spaceships.work → 192.168.1.100
└→ PTR record: 100.1.168.192.in-addr.arpa → docker-01-nexus.spaceships.work
4. DNS propagates automatically
└→ No manual DNS configuration needed
```
### Integration with Proxmox
```text
Terraform/Ansible
↓
Creates VM in Proxmox
↓
Registers in NetBox (via API)
├→ Device object
├→ IP Address with dns_name
└→ Tags (production-dns)
↓
NetBox PowerDNS Sync
↓
DNS Records in PowerDNS
↓
Ansible Dynamic Inventory
↓
Automated configuration management
```
## Tools Available
### NetBox API Tools (Python + uv)
**netbox_api_client.py** - Comprehensive NetBox API client
```bash
# List sites, devices, VMs, IPs
./tools/netbox_api_client.py sites list
./tools/netbox_api_client.py devices get --name foxtrot
./tools/netbox_api_client.py vms list --cluster matrix
./tools/netbox_api_client.py ips query --dns-name docker-01
./tools/netbox_api_client.py prefixes available --prefix 192.168.3.0/24
```
**netbox_vm_create.py** - Create VMs in NetBox with IP assignment
```bash
# Create VM with auto IP
./tools/netbox_vm_create.py --name docker-02 --cluster matrix --vcpus 4 --memory 8192
# Create VM with specific IP
./tools/netbox_vm_create.py --name k8s-01-master --cluster matrix --ip 192.168.3.50/24
```
**netbox_ipam_query.py** - Advanced IPAM queries
```bash
# Available IPs
./tools/netbox_ipam_query.py available --prefix 192.168.3.0/24
# Prefix utilization
./tools/netbox_ipam_query.py utilization --site matrix
# IP assignments
./tools/netbox_ipam_query.py assignments --prefix 192.168.3.0/24
# IPAM summary
./tools/netbox_ipam_query.py summary --site matrix
```
**validate_dns_naming.py** - Validate DNS naming conventions
```bash
./tools/validate_dns_naming.py --name "docker-01-nexus.spaceships.work"
```
### Terraform Modules
**netbox-data-sources.tf** - Examples using NetBox provider
- Query existing NetBox resources
- Use as data sources for other resources
### Ansible Playbooks
**deploy-from-netbox.yml** - Deploy using NetBox inventory
- Dynamic inventory from NetBox
- Tag-based host selection
- Automatic IP and hostname discovery
See [examples/](examples/) directory.
## Troubleshooting
### DNS Records Not Syncing
#### Check 1: Tag Matching
```bash
# Verify IP has correct tags
./tools/netbox_query.py --ip 192.168.1.100 | jq '.tags'
```
#### Check 2: Zone Configuration
- Ensure zone exists in NetBox
- Verify tag rules match
- Check PowerDNS server connectivity
#### Check 3: Sync Results
```bash
./tools/powerdns_sync_check.py --zone spaceships.work --verbose
```
### Naming Convention Violations
**Validate names before creating:**
```bash
./tools/validate_dns_naming.py --name "my-proposed-name.domain"
```
**Common mistakes:**
- Uppercase letters (use lowercase only)
- Missing service number (must be XX format)
- Wrong domain
- Special characters (use hyphens only)
### Terraform Provider Issues
**Version mismatch:**
```text
Warning: NetBox version 4.3.0 not supported by provider 3.9.0
```
**Solution:** Update provider version:
```hcl
version = "~> 5.0.0" # Match NetBox 4.3.x
```
### Dynamic Inventory Not Working
**Check API connectivity:**
```bash
curl -H "Authorization: Token YOUR_TOKEN" \
https://netbox.spaceships.work/api/dcim/devices/
```
**Verify inventory plugin:**
```bash
ansible-inventory -i tools/netbox-dynamic-inventory.yml --list
```
See [troubleshooting/](reference/) for more details.
## Best Practices
1. **Consistent naming** - Always follow `service-NN-purpose.domain` pattern
2. **Use tags** - Tag resources for auto-DNS (`production-dns`, `lab-dns`)
3. **Document in NetBox** - Single source of truth for all infrastructure
4. **Real-time sync** - Enable `post_save_enabled` for immediate DNS updates
5. **Terraform everything** - Manage NetBox resources as IaC
6. **Dynamic inventory** - Never maintain static Ansible inventory
7. **Audit regularly** - Run `dns_record_audit.py` to verify sync
## DNS Naming Patterns
### Service Types
- `docker-NN-<app>` - Docker hosts
- `k8s-NN-<role>` - Kubernetes nodes
- `proxmox-<node>-<iface>` - Proxmox infrastructure
- `storage-NN-<purpose>` - Storage systems
- `db-NN-<dbtype>` - Database servers
### Examples from This Infrastructure
```text
docker-01-nexus.spaceships.work # Nexus container registry
k8s-01-master.spaceships.work # K8s control plane
k8s-02-worker.spaceships.work # K8s worker node
proxmox-foxtrot-mgmt.spaceships.work # Proxmox mgmt interface
proxmox-foxtrot-ceph.spaceships.work # Proxmox CEPH interface
storage-01-nas.spaceships.work # NAS storage
db-01-postgres.spaceships.work # PostgreSQL database
```
## Progressive Disclosure
For deeper knowledge:
### NetBox API & Integration
- [NetBox API Guide](reference/netbox-api-guide.md) - Complete REST API reference with pynetbox examples
- [NetBox Data Models](reference/netbox-data-models.md) - Data model relationships and hierarchy
- [NetBox Best Practices](reference/netbox-best-practices.md) - Security, performance, automation patterns
### DNS & PowerDNS Integration
- [Sync Plugin Reference](reference/sync-plugin-reference.md) - PowerDNS sync plugin installation and config
- [Terraform Provider Guide](reference/terraform-provider-guide.md) - Complete provider documentation
- [Naming Conventions](workflows/naming-conventions.md) - Detailed DNS naming rules
- [DNS Automation](workflows/dns-automation.md) - End-to-end automation workflows
### Ansible Integration
- [Ansible Dynamic Inventory](workflows/ansible-dynamic-inventory.md) - Using NetBox as inventory source
### Anti-Patterns & Common Mistakes
- [Common Mistakes](anti-patterns/common-mistakes.md) - DNS naming violations, cluster domain errors, master node targeting,
and NetBox integration pitfalls for spaceships.work infrastructure
## Related Skills
- **Proxmox Infrastructure** - VMs auto-registered in NetBox with DNS
- **Ansible Best Practices** - Dynamic inventory and secrets management

441
skills/netbox-powerdns-integration/anti-patterns/common-mistakes.md Normal file
View File

@@ -0,0 +1,441 @@
# Common Mistakes and Anti-Patterns
DNS naming convention violations and NetBox/PowerDNS integration pitfalls based on the `spaceships.work` infrastructure.
## DNS Naming Convention Violations
### Infrastructure Overview
**Root Domain**: `spaceships.work`
**Cluster Domains**:
- `matrix.spaceships.work` - Nexus cluster (3 nodes)
- `quantum.spaceships.work` - Quantum cluster (3 nodes)
- `nexus.spaceships.work` - (Legacy naming reference)
**Proxmox Node Domains** (with master node designations):
**Matrix Cluster** (nexus.spaceships.work):
- `foxtrot.nexus.spaceships.work` - **Master Node** (API Target)
- `golf.nexus.spaceships.work`
- `hotel.nexus.spaceships.work`
**Quantum Cluster** (quantum.spaceships.work):
- `charlie.quantum.spaceships.work`
- `delta.quantum.spaceships.work` - **Master Node** (API Target)
- `echo.quantum.spaceships.work`
**Matrix Cluster** (matrix.spaceships.work):
- `alpha.matrix.spaceships.work`
- `bravo.matrix.spaceships.work` - **Master Node** (API Target)
- `charlie.matrix.spaceships.work`
---
## ❌ Wrong Root Domain
**Problem**: Using incorrect root domain in DNS records.
```python
# BAD - Wrong domain
hostname = "docker-01-nexus.internal.lan"
hostname = "k8s-master.homelab.local"
```
**Solution**: Always use `spaceships.work` as root domain.
```python
# GOOD
hostname = "docker-01-nexus.spaceships.work"
hostname = "k8s-01-master.matrix.spaceships.work"
```
---
## ❌ Wrong Cluster Subdomain
**Problem**: Using non-existent cluster domains.
```python
# BAD - Invalid cluster domains
hostname = "docker-01.homelab.spaceships.work" # No 'homelab' cluster
hostname = "vm-01.lab.spaceships.work" # No 'lab' cluster
hostname = "test.prod.spaceships.work" # No 'prod' cluster
```
**Solution**: Use only valid cluster domains.
```python
# GOOD - Valid cluster domains
hostname = "docker-01-nexus.matrix.spaceships.work"
hostname = "k8s-01-worker.quantum.spaceships.work"
hostname = "storage-01-ceph.nexus.spaceships.work"
```
**Valid Clusters**:
- `matrix.spaceships.work`
- `quantum.spaceships.work`
- `nexus.spaceships.work`
---
## ❌ Wrong Node Domain
**Problem**: Using incorrect Proxmox node FQDNs.
```python
# BAD - Incorrect node names
proxmox_host = "node1.spaceships.work" # Not the naming pattern
proxmox_host = "foxtrot.spaceships.work" # Missing cluster subdomain
proxmox_host = "foxtrot.matrix.spaceships.org" # Wrong TLD
```
**Solution**: Use correct node FQDN pattern.
```python
# GOOD - Correct node FQDNs
proxmox_host = "foxtrot.nexus.spaceships.work" # Nexus cluster master
proxmox_host = "delta.quantum.spaceships.work" # Quantum cluster master
proxmox_host = "bravo.matrix.spaceships.work" # Matrix cluster master
```
---
## ❌ Targeting Non-Master Nodes
**Problem**: Sending API calls to non-master nodes in cluster.
```python
# BAD - Not a master node
from proxmoxer import ProxmoxAPI
proxmox = ProxmoxAPI(
'golf.nexus.spaceships.work', # ❌ Not the master!
user='root@pam',
password='...'
)
```
**Solution**: Always target the designated master node for API operations.
```python
# GOOD - Target master nodes
from proxmoxer import ProxmoxAPI
# Nexus cluster
proxmox_nexus = ProxmoxAPI(
'foxtrot.nexus.spaceships.work', # ✅ Master node
user='root@pam',
password='...'
)
# Quantum cluster
proxmox_quantum = ProxmoxAPI(
'delta.quantum.spaceships.work', # ✅ Master node
user='root@pam',
password='...'
)
# Matrix cluster
proxmox_matrix = ProxmoxAPI(
'bravo.matrix.spaceships.work', # ✅ Master node
user='root@pam',
password='...'
)
```
**Master Nodes Quick Reference**:
- **Nexus**: `foxtrot.nexus.spaceships.work`
- **Quantum**: `delta.quantum.spaceships.work`
- **Matrix**: `bravo.matrix.spaceships.work`
---
## ❌ Violating Service-Number-Purpose Pattern
**Problem**: Not following the `service-NN-purpose` naming convention.
```python
# BAD - Violates naming pattern
hostname = "nexus.spaceships.work" # Missing service number
hostname = "docker-nexus.spaceships.work" # Missing number
hostname = "docker1-nexus.spaceships.work" # Wrong number format (1 vs 01)
hostname = "nexus-docker-01.spaceships.work" # Wrong order
```
**Solution**: Follow `service-NN-purpose.domain` pattern.
```python
# GOOD - Correct naming pattern
hostname = "docker-01-nexus.spaceships.work" # ✅ service-01-purpose
hostname = "k8s-02-worker.matrix.spaceships.work" # ✅ service-02-purpose
hostname = "storage-03-ceph.nexus.spaceships.work"# ✅ service-03-purpose
```
**Pattern**: `<service>-<NN>-<purpose>.<cluster>.<root-domain>`
Components:
- **service**: Infrastructure type (`docker`, `k8s`, `proxmox`, `storage`, `db`)
- **NN**: Two-digit number (`01`, `02`, `03`, ... `99`)
- **purpose**: Specific role (`nexus`, `master`, `worker`, `ceph`, `postgres`)
- **cluster**: Cluster subdomain (`matrix`, `quantum`, `nexus`)
- **root-domain**: `spaceships.work`
---
## ❌ Improper Case in Hostnames
**Problem**: Using uppercase letters in DNS names.
```python
# BAD - Uppercase not allowed
hostname = "Docker-01-Nexus.spaceships.work"
hostname = "K8S-01-MASTER.matrix.spaceships.work"
```
**Solution**: Always use lowercase.
```python
# GOOD
hostname = "docker-01-nexus.spaceships.work"
hostname = "k8s-01-master.matrix.spaceships.work"
```
---
## NetBox Integration Issues
### ❌ Not Setting DNS Name in NetBox
**Problem**: Creating IP address in NetBox without `dns_name` field.
```python
# BAD - Missing DNS name
ip = nb.ipam.ip_addresses.create(
address="192.168.3.100/24",
description="Docker host",
# Missing dns_name!
)
```
**Solution**: Always set `dns_name` when creating IPs.
```python
# GOOD
ip = nb.ipam.ip_addresses.create(
address="192.168.3.100/24",
dns_name="docker-01-nexus.matrix.spaceships.work", # ✅
description="Docker host for Nexus registry",
tags=["production-dns"] # Triggers PowerDNS sync
)
```
---
### ❌ Missing PowerDNS Sync Tags
**Problem**: DNS record not created automatically because missing trigger tag.
```python
# BAD - No sync tag
ip = nb.ipam.ip_addresses.create(
address="192.168.3.100/24",
dns_name="docker-01-nexus.matrix.spaceships.work",
tags=["docker", "production"] # Missing 'production-dns' tag!
)
# PowerDNS record NOT created automatically
```
**Solution**: Include appropriate sync tag.
```python
# GOOD
ip = nb.ipam.ip_addresses.create(
address="192.168.3.100/24",
dns_name="docker-01-nexus.matrix.spaceships.work",
tags=[
"docker",
"production",
"production-dns" # ✅ Triggers PowerDNS sync
]
)
```
**Sync Tags**:
- `production-dns` - Auto-create in PowerDNS production zone
- `lab-dns` - Auto-create in PowerDNS lab zone
---
### ❌ Inconsistent DNS Naming Between Tools
**Problem**: Different naming in OpenTofu vs NetBox vs Proxmox.
**Note**: Use `tofu` CLI (not `terraform`).
```hcl
# OpenTofu
resource "proxmox_virtual_environment_vm" "docker_host" {
name = "docker-nexus-01" # ❌ Wrong order
}
# NetBox
dns_name = "nexus-docker-01.spaceships.work" # ❌ Different order
# Proxmox
hostname = "docker01-nexus" # ❌ Missing hyphen
```
**Solution**: Consistent naming everywhere.
```hcl
# OpenTofu
resource "proxmox_virtual_environment_vm" "docker_host" {
name = "docker-01-nexus" # ✅ Consistent
}
```
```python
# NetBox
dns_name = "docker-01-nexus.matrix.spaceships.work" # ✅ Consistent
```
```yaml
# Proxmox (via OpenTofu)
initialization {
user_data_file_id = "local:snippets/user-data.yaml"
# Inside user-data.yaml:
# hostname: docker-01-nexus # ✅ Consistent
# fqdn: docker-01-nexus.matrix.spaceships.work
}
```
---
## Validation Tools
### Check DNS Naming Convention
```bash
# Validate hostname format
./tools/validate_dns_naming.py --name "docker-01-nexus.matrix.spaceships.work"
# ✅ Valid
./tools/validate_dns_naming.py --name "docker-nexus.spaceships.work"
# ❌ Invalid: Missing number in service-NN-purpose pattern
```
### Check NetBox DNS Records
```bash
# Query NetBox for DNS records
./tools/netbox_api_client.py ips query --dns-name docker-01
# Verify PowerDNS sync
./tools/powerdns_sync_check.py --zone spaceships.work --verbose
```
---
## Quick Reference
### Correct Naming Patterns
**VM Hostnames**:
```text
docker-01-nexus.matrix.spaceships.work
k8s-01-master.quantum.spaceships.work
storage-01-ceph.nexus.spaceships.work
db-01-postgres.matrix.spaceships.work
```
**Proxmox Nodes**:
```text
foxtrot.nexus.spaceships.work (master)
delta.quantum.spaceships.work (master)
bravo.matrix.spaceships.work (master)
```
**Service Types**:
- `docker-NN-<app>` - Docker hosts
- `k8s-NN-<role>` - Kubernetes nodes (master, worker)
- `proxmox-<node>-<iface>` - Proxmox infrastructure
- `storage-NN-<type>` - Storage systems (ceph, nas)
- `db-NN-<dbtype>` - Database servers (postgres, mysql)
### Valid Domains
**Clusters**:
- `matrix.spaceships.work`
- `quantum.spaceships.work`
- `nexus.spaceships.work`
**Master Nodes** (API targets):
- Nexus: `foxtrot.nexus.spaceships.work`
- Quantum: `delta.quantum.spaceships.work`
- Matrix: `bravo.matrix.spaceships.work`
---
## Troubleshooting
### DNS Record Not Created in PowerDNS
**Check**:
1. ✅ DNS name follows pattern? `service-NN-purpose.cluster.spaceships.work`
2. ✅ Has `production-dns` or `lab-dns` tag in NetBox?
3. ✅ NetBox PowerDNS sync plugin enabled?
4. ✅ Zone exists in NetBox and matches domain?
### Can't Connect to Proxmox API
**Check**:
1. ✅ Using master node FQDN?
- Nexus: `foxtrot.nexus.spaceships.work`
- Quantum: `delta.quantum.spaceships.work`
- Matrix: `bravo.matrix.spaceships.work`
2. ✅ DNS resolves correctly? `dig <master-node-fqdn>`
3. ✅ Cluster subdomain included? (not just `<node>.spaceships.work`)
### Validation Script Fails
**Common Issues**:
```bash
# Missing number
❌ docker-nexus.spaceships.work
✅ docker-01-nexus.matrix.spaceships.work
# Wrong number format
❌ docker-1-nexus.spaceships.work
✅ docker-01-nexus.matrix.spaceships.work
# Missing cluster subdomain
❌ docker-01-nexus.spaceships.work
✅ docker-01-nexus.matrix.spaceships.work
# Wrong domain
❌ docker-01-nexus.local
✅ docker-01-nexus.matrix.spaceships.work
```
Run validation:
```bash
./tools/validate_dns_naming.py --name "your-hostname-here"
```

385
skills/netbox-powerdns-integration/examples/01-vm-with-dns/README.md Normal file
View File

@@ -0,0 +1,385 @@
# VM with Automatic DNS Registration
**Learning objective:** Experience the full infrastructure automation workflow from VM creation to DNS resolution.
## What This Example Demonstrates
This is the "holy grail" of infrastructure automation - deploy infrastructure and have DNS automatically configured:
```text
Terraform Deploy → Proxmox VM Created → NetBox IP Registered
→ PowerDNS Records Auto-Created → Ready for Ansible
```
**Benefits:**
- ✅ No manual DNS configuration
- ✅ Single source of truth (NetBox)
- ✅ Audit trail for all infrastructure changes
- ✅ Automatic forward + reverse DNS
- ✅ Ready for dynamic inventory
## Architecture
```text
┌──────────────┐
│ Terraform │
└──────┬───────┘
│
├─ Creates VM in Proxmox ─────────────► VM: docker-01-nexus
│ IP: 192.168.1.100
│
└─ Registers IP in NetBox ────────────► NetBox IPAM
│ Tags: ["production-dns"]
│
│ netbox-powerdns-sync plugin (automatic)
│
▼
┌────────────┐
│ PowerDNS │
└─────┬──────┘
│
├─► A record: docker-01-nexus.spaceships.work → 192.168.1.100
└─► PTR record: 100.1.168.192.in-addr.arpa → docker-01-nexus.spaceships.work
```
## Prerequisites
### 1. Proxmox Template
Template must exist (default: VMID 9000):
```bash
# Create using Virgo-Core template deployment
cd ../../../terraform/netbox-template
tofu apply
```
### 2. NetBox Configuration
**NetBox PowerDNS Sync Plugin** must be configured with:
- Zone: `spaceships.work`
- Tag rule matching: `production-dns`
- PowerDNS server connection configured
See: [../../reference/sync-plugin-reference.md](../../reference/sync-plugin-reference.md)
### 3. Environment Variables
```bash
# Proxmox
export PROXMOX_VE_ENDPOINT="https://192.168.3.5:8006"
export PROXMOX_VE_API_TOKEN="user@realm!token-id=secret"
# NetBox
export NETBOX_API_TOKEN="your-netbox-token"
export TF_VAR_netbox_api_token="$NETBOX_API_TOKEN"
# SSH Key
export TF_VAR_ssh_public_key="$(cat ~/.ssh/id_rsa.pub)"
```
### 4. DNS Naming Convention
FQDN must follow: `<service>-<NN>-<purpose>.<domain>`
**Validate name first:**
```bash
../../tools/validate_dns_naming.py docker-01-nexus.spaceships.work
```
## Quick Start
### 1. Initialize
```bash
tofu init
```
### 2. Plan
```bash
tofu plan
```
**Expected resources:**
- 1 Proxmox VM (docker-01-nexus)
- 1 NetBox IP address (192.168.1.100)
- DNS records created automatically (not shown in plan)
### 3. Deploy
```bash
tofu apply
```
**Wait 30 seconds** for DNS propagation (netbox-powerdns-sync runs async).
### 4. Verify Complete Workflow
#### Step 1: Check VM exists
```bash
# On Proxmox node
qm status $(terraform output -raw vm_id)
```
#### Step 2: Check NetBox registration
```bash
curl -H "Authorization: Token $NETBOX_API_TOKEN" \
"https://netbox.spaceships.work/api/ipam/ip-addresses/?address=192.168.1.100" | jq
```
#### Step 3: Verify DNS forward resolution
```bash
dig @192.168.3.1 docker-01-nexus.spaceships.work +short
# Expected: 192.168.1.100
```
#### Step 4: Verify DNS reverse resolution
```bash
dig @192.168.3.1 -x 192.168.1.100 +short
# Expected: docker-01-nexus.spaceships.work.
```
#### Step 5: SSH into VM
```bash
ssh ansible@docker-01-nexus.spaceships.work
# or
ssh ansible@192.168.1.100
```
### 5. Cleanup
```bash
tofu destroy
```
**Note:** DNS records are automatically removed when NetBox IP is deleted.
## How It Works
### The Magic Tag: `production-dns`
```hcl
resource "netbox_ip_address" "docker_host" {
tags = [
"terraform",
"production-dns", # ← This triggers DNS automation
"docker-host"
]
}
```
The `production-dns` tag matches a zone rule in NetBox PowerDNS Sync plugin configuration.
### NetBox PowerDNS Sync Plugin
**Configuration (in NetBox):**
```python
# Plugins → NetBox PowerDNS Sync → Zones → spaceships.work
zone_config = {
"name": "spaceships.work",
"powerdns_server": "PowerDNS Server 1",
"tag_match": ["production-dns"], # ← Matches our tag
"naming_method": "device", # Use device/IP name
"auto_sync": True,
"sync_schedule": "*/5 * * * *" # Every 5 minutes
}
```
**What happens:**
1. IP address created in NetBox with `production-dns` tag
2. Plugin detects new IP matches zone rule
3. Plugin calls PowerDNS API to create records:
- A record from `dns_name` field
- PTR record from IP address
4. Records appear in DNS within 30 seconds
## Customization
### Different Service Type
```hcl
# In terraform.tfvars or as -var
vm_name = "k8s-01-master"
fqdn = "k8s-01-master.spaceships.work"
```
### Development Environment
Use different domain and tags:
```hcl
variable "dns_domain" {
default = "dev.spaceships.work"
}
resource "netbox_ip_address" "docker_host" {
tags = ["terraform", "dev-dns", "docker-host"] # Different tag for dev zone
}
```
### Multiple Environments
```bash
# Production
tofu apply -var-file=prod.tfvars
# Development
tofu apply -var-file=dev.tfvars
```
## Troubleshooting
### DNS Records Not Created
**1. Check NetBox IP registration:**
```bash
curl -H "Authorization: Token $NETBOX_API_TOKEN" \
"https://netbox.spaceships.work/api/ipam/ip-addresses/?dns_name=docker-01-nexus.spaceships.work" | jq
```
Verify:
- IP exists in NetBox
- Has `production-dns` tag
- `dns_name` field is set
**2. Check zone configuration:**
- NetBox → Plugins → NetBox PowerDNS Sync → Zones
- Verify `spaceships.work` zone exists
- Check tag rules match `production-dns`
**3. Check sync results:**
- NetBox → Plugins → NetBox PowerDNS Sync → Zones → spaceships.work
- Click "Sync Now" to manually trigger
- Review sync results for errors
**4. Check PowerDNS:**
```bash
# Query PowerDNS API directly
curl -H "X-API-Key: $POWERDNS_API_KEY" \
http://192.168.3.1:8081/api/v1/servers/localhost/zones/spaceships.work | jq
```
### SSH Connection Fails
**Try IP first:**
```bash
ssh ansible@192.168.1.100
```
If IP works but FQDN doesn't:
- DNS not propagated yet (wait 30s)
- Check DNS resolution: `dig @192.168.3.1 docker-01-nexus.spaceships.work`
### NetBox Provider Authentication
**Error:** `authentication failed`
**Solution:**
```bash
# Test API token
curl -H "Authorization: Token $NETBOX_API_TOKEN" \
https://netbox.spaceships.work/api/status/ | jq
# Regenerate token if needed
# NetBox → Admin → API Tokens → Create
```
## Integration with Ansible
After deployment, use for Ansible configuration:
```yaml
---
# Ansible playbook using NetBox dynamic inventory
- name: Configure Docker hosts
hosts: tag_docker_host # From NetBox tags
become: true
tasks:
- name: Install Docker
ansible.builtin.apt:
name: docker.io
state: present
```
**Run with dynamic inventory:**
```bash
cd ../../../ansible
ansible-playbook -i inventory/netbox.yml playbooks/configure-docker.yml
```
See: [../../workflows/ansible-dynamic-inventory.md](../../workflows/ansible-dynamic-inventory.md)
## Next Steps
### Learn More Workflows
- **Bulk Deployment:** `../02-bulk-deployment/` (coming soon)
- **DNS Automation Guide:** [../../workflows/dns-automation.md](../../workflows/dns-automation.md)
- **Naming Conventions:** [../../workflows/naming-conventions.md](../../workflows/naming-conventions.md)
### Production Checklist
Before using in production:
- [ ] NetBox PowerDNS Sync plugin tested and working
- [ ] DNS naming convention documented
- [ ] Zone rules configured for all environments
- [ ] API tokens secured (not in version control)
- [ ] Backup/restore procedures for NetBox
- [ ] Monitoring for DNS sync failures
- [ ] Documentation for team
## Benefits of This Approach
**Single Source of Truth:**
- All infrastructure documented in NetBox
- DNS automatically matches reality
- Easy to audit what exists
**Automation:**
- No manual DNS configuration
- Reduced human error
- Faster deployments
**Consistency:**
- Naming convention enforced
- Tag-based organization
- Audit trail via Terraform
**Integration:**
- Ansible dynamic inventory
- Monitoring integrations
- IPAM + DNS + DCIM in one place
## Related Documentation
- [NetBox Provider Guide](../../reference/terraform-provider-guide.md)
- [PowerDNS Sync Plugin](../../reference/sync-plugin-reference.md)
- [DNS Naming Conventions](../../workflows/naming-conventions.md)
- [DNS Automation Workflows](../../workflows/dns-automation.md)

191
skills/netbox-powerdns-integration/examples/01-vm-with-dns/main.tf Normal file
View File

@@ -0,0 +1,191 @@
# =============================================================================
# VM with Automatic DNS Registration
# =============================================================================
# This example demonstrates the complete infrastructure automation workflow:
# 1. Create VM in Proxmox (using unified module)
# 2. Register IP address in NetBox with DNS name
# 3. DNS records automatically created in PowerDNS (via netbox-powerdns-sync plugin)
# 4. Ready for Ansible configuration management
#
# Result: Fully automated infrastructure from VM → DNS → Configuration
terraform {
required_version = ">= 1.0"
required_providers {
proxmox = {
source = "bpg/proxmox"
version = "~> 0.69"
}
netbox = {
source = "e-breuninger/netbox"
version = "~> 5.0.0"
}
}
}
# === Proxmox Provider ===
provider "proxmox" {
endpoint = var.proxmox_endpoint
# Credentials from environment:
# PROXMOX_VE_API_TOKEN or PROXMOX_VE_USERNAME/PASSWORD
}
# === NetBox Provider ===
provider "netbox" {
server_url = var.netbox_url
api_token = var.netbox_api_token
# Or use environment: NETBOX_API_TOKEN
}
# =============================================================================
# Step 1: Create VM in Proxmox
# =============================================================================
module "docker_host" {
source = "github.com/basher83/Triangulum-Prime//terraform-bgp-vm?ref=vm/1.0.1"
# VM Configuration
vm_type = "clone"
pve_node = var.proxmox_node
vm_name = var.vm_name
# Clone from template
src_clone = {
datastore_id = "local-lvm"
tpl_id = var.template_id
}
# Production-ready resources
vm_cpu = {
cores = 4 # Docker workload
}
vm_mem = {
dedicated = 8192 # 8GB for containers
}
# Disk configuration
vm_disk = {
scsi0 = {
datastore_id = "local-lvm"
size = 100 # Larger for Docker images
main_disk = true
}
}
# Network with VLAN
vm_net_ifaces = {
net0 = {
bridge = var.network_bridge
vlan_id = var.vlan_id
ipv4_addr = "${var.ip_address}/24"
ipv4_gw = var.gateway
}
}
# Cloud-init
vm_init = {
datastore_id = "local-lvm"
user = {
name = "ansible"
keys = [var.ssh_public_key]
}
dns = {
domain = var.dns_domain
servers = [var.dns_server]
}
}
# EFI disk
vm_efi_disk = {
datastore_id = "local-lvm"
}
# Tags for organization
vm_tags = ["terraform", "docker-host", "production"]
}
# =============================================================================
# Step 2: Register IP in NetBox with DNS Name
# =============================================================================
resource "netbox_ip_address" "docker_host" {
ip_address = "${var.ip_address}/24"
dns_name = var.fqdn # e.g., docker-01-nexus.spaceships.work
status = "active"
description = var.vm_description
# CRITICAL: This tag triggers automatic DNS sync via netbox-powerdns-sync plugin
tags = [
"terraform",
"production-dns", # ← Matches zone rule in NetBox PowerDNS Sync plugin
"docker-host"
]
# Ensure VM is created first
depends_on = [module.docker_host]
lifecycle {
# Prevent accidental deletion of IP registration
prevent_destroy = false # Set to true for production
}
}
# =============================================================================
# Step 3: DNS Records Created Automatically
# =============================================================================
#
# The netbox-powerdns-sync plugin automatically creates:
# - A record: docker-01-nexus.spaceships.work → 192.168.1.100
# - PTR record: 100.1.168.192.in-addr.arpa → docker-01-nexus.spaceships.work
#
# No manual DNS configuration needed!
#
# Verify with:
# dig @192.168.3.1 docker-01-nexus.spaceships.work +short
# dig @192.168.3.1 -x 192.168.1.100 +short
# =============================================================================
# Outputs
# =============================================================================
output "vm_id" {
description = "Proxmox VM ID"
value = module.docker_host.vm_id
}
output "vm_name" {
description = "VM name"
value = module.docker_host.vm_name
}
output "vm_ip" {
description = "VM IP address"
value = var.ip_address
}
output "fqdn" {
description = "Fully qualified domain name (with automatic DNS)"
value = var.fqdn
}
output "netbox_ip_id" {
description = "NetBox IP address ID"
value = netbox_ip_address.docker_host.id
}
output "ssh_command" {
description = "SSH command to access the VM"
value = "ssh ansible@${var.fqdn}"
}
output "verification_commands" {
description = "Commands to verify DNS resolution"
value = {
forward = "dig @${var.dns_server} ${var.fqdn} +short"
reverse = "dig @${var.dns_server} -x ${var.ip_address} +short"
}
}

96
skills/netbox-powerdns-integration/examples/01-vm-with-dns/variables.tf Normal file
View File

@@ -0,0 +1,96 @@
variable "proxmox_endpoint" {
description = "Proxmox API endpoint"
type = string
default = "https://192.168.3.5:8006"
}
variable "proxmox_node" {
description = "Proxmox node for deployment"
type = string
default = "foxtrot"
}
variable "netbox_url" {
description = "NetBox server URL"
type = string
default = "https://netbox.spaceships.work"
}
variable "netbox_api_token" {
description = "NetBox API token"
type = string
sensitive = true
# Set via: export TF_VAR_netbox_api_token="your-token"
# Or use: NETBOX_API_TOKEN environment variable
}
variable "template_id" {
description = "Proxmox template VMID to clone from"
type = number
default = 9000
}
variable "vm_name" {
description = "VM name (short, used in Proxmox)"
type = string
default = "docker-01-nexus"
}
variable "fqdn" {
description = "Fully qualified domain name (must follow naming convention: <service>-<NN>-<purpose>.<domain>)"
type = string
default = "docker-01-nexus.spaceships.work"
validation {
condition = can(regex("^[a-z0-9-]+-\\d{2}-[a-z0-9-]+\\.[a-z0-9.-]+$", var.fqdn))
error_message = "FQDN must follow naming convention: <service>-<NN>-<purpose>.<domain> (e.g., docker-01-nexus.spaceships.work)"
}
}
variable "vm_description" {
description = "VM description (shown in both Proxmox and NetBox)"
type = string
default = "Docker host for Nexus container registry"
}
variable "ip_address" {
description = "Static IP address (without CIDR)"
type = string
default = "192.168.1.100"
}
variable "gateway" {
description = "Network gateway"
type = string
default = "192.168.1.1"
}
variable "network_bridge" {
description = "Proxmox network bridge"
type = string
default = "vmbr0"
}
variable "vlan_id" {
description = "VLAN ID (null for no VLAN)"
type = number
default = 30
}
variable "dns_domain" {
description = "DNS domain for cloud-init"
type = string
default = "spaceships.work"
}
variable "dns_server" {
description = "DNS server IP"
type = string
default = "192.168.3.1"
}
variable "ssh_public_key" {
description = "SSH public key for VM access"
type = string
# Set via: export TF_VAR_ssh_public_key="$(cat ~/.ssh/id_rsa.pub)"
}

1092
skills/netbox-powerdns-integration/reference/netbox-api-guide.md Normal file
View File

File diff suppressed because it is too large Load Diff

997
skills/netbox-powerdns-integration/reference/netbox-best-practices.md Normal file
View File

@@ -0,0 +1,997 @@
# NetBox Best Practices for Virgo-Core
**NetBox Version:** 4.3.0
**Audience:** Infrastructure automation engineers
Comprehensive best practices for using NetBox as the source of truth for the Matrix cluster infrastructure, including data organization, security, performance, and integration patterns.
---
## Table of Contents
- [Data Organization](#data-organization)
- [Naming Conventions](#naming-conventions)
- [IP Address Management](#ip-address-management)
- [Device Management](#device-management)
- [Virtualization](#virtualization)
- [Tagging Strategy](#tagging-strategy)
- [Security](#security)
- [Performance](#performance)
- [API Integration](#api-integration)
- [Automation Patterns](#automation-patterns)
- [Troubleshooting](#troubleshooting)
---
## Data Organization
### Hierarchical Structure
Follow this order when setting up infrastructure in NetBox:
```text
1. Sites → Create physical locations first
2. Prefixes → Define IP networks (IPAM)
3. VLANs → Network segmentation
4. Device Types → Hardware models
5. Device Roles → Purpose categories
6. Clusters → Virtualization clusters
7. Devices → Physical hardware
8. Interfaces → Network interfaces
9. IP Addresses → Assign IPs to interfaces
10. VMs → Virtual machines
```
**Why this order?**
- Parent objects must exist before children
- Avoids circular dependencies
- Enables atomic operations
### Site Organization
**✅ Good:**
```python
# One site per physical location
site = nb.dcim.sites.create(
name="Matrix Cluster",
slug="matrix",
description="3-node Proxmox VE cluster at home lab",
tags=[{"name": "production"}, {"name": "homelab"}]
)
```
**❌ Bad:**
```python
# Don't create separate sites for logical groupings
site_proxmox = nb.dcim.sites.create(name="Proxmox Nodes", ...)
site_vms = nb.dcim.sites.create(name="Virtual Machines", ...)
```
Use **device roles** and **tags** for logical grouping, not separate sites.
### Consistent Data Entry
**Required fields:** Always populate these
```python
device = nb.dcim.devices.create(
name="foxtrot", # ✅ Required
device_type=device_type.id, # ✅ Required
device_role=role.id, # ✅ Required
site=site.id, # ✅ Required
status="active", # ✅ Required
description="AMD Ryzen 9 9955HX", # ✅ Recommended
tags=[{"name": "proxmox-node"}] # ✅ Recommended
)
```
**Optional but recommended:**
- `description` - Hardware specs, purpose
- `tags` - For filtering and automation
- `comments` - Additional notes
- `custom_fields` - Serial numbers, purchase dates
---
## Naming Conventions
### Device Names
**✅ Use hostname only (no domain):**
```python
device = nb.dcim.devices.create(name="foxtrot", ...) # ✅ Good
device = nb.dcim.devices.create(name="foxtrot.spaceships.work", ...) # ❌ Bad
```
**Rationale:** Domain goes in DNS name field, not device name.
### Interface Names
**✅ Match actual OS interface names:**
```python
# Linux
interface = nb.dcim.interfaces.create(name="enp1s0f0", ...) # ✅ Good
# Not generic names
interface = nb.dcim.interfaces.create(name="eth0", ...) # ❌ Bad (unless actually eth0)
```
**Why?** Enables automation that references interfaces by name.
### DNS Naming Convention
Follow the Matrix cluster pattern: **`<service>-<number>-<purpose>.<domain>`**
```python
# ✅ Good examples
dns_name="docker-01-nexus.spaceships.work"
dns_name="k8s-01-master.spaceships.work"
dns_name="proxmox-foxtrot-mgmt.spaceships.work"
# ❌ Bad examples
dns_name="server1.spaceships.work" # Not descriptive
dns_name="nexus.spaceships.work" # Missing number
dns_name="DOCKER-01.spaceships.work" # Uppercase not allowed
```
See [../workflows/naming-conventions.md](../workflows/naming-conventions.md) for complete rules.
### Slugs
**✅ Lowercase with hyphens:**
```python
site = nb.dcim.sites.create(slug="matrix", ...) # ✅ Good
site = nb.dcim.sites.create(slug="Matrix_Cluster", ...) # ❌ Bad
```
**Pattern:** `^[a-z0-9-]+$`
---
## IP Address Management
### Plan IP Hierarchy
**Matrix cluster example:**
```text
192.168.0.0/16 (Home network supernet)
├── 192.168.3.0/24 (Management)
│ ├── 192.168.3.1 (Gateway)
│ ├── 192.168.3.5-7 (Proxmox nodes)
│ ├── 192.168.3.10+ (VMs)
│ └── 192.168.3.200+ (Reserved for future)
├── 192.168.5.0/24 (CEPH Public, MTU 9000)
├── 192.168.7.0/24 (CEPH Private, MTU 9000)
└── 192.168.8.0/24 (Corosync, VLAN 9)
```
### Use Prefix Roles
Create roles for clarity:
```python
# Create roles
role_mgmt = nb.ipam.roles.create(name='Management', slug='management')
role_storage = nb.ipam.roles.create(name='Storage', slug='storage')
role_cluster = nb.ipam.roles.create(name='Cluster', slug='cluster')
# Apply to prefixes
prefix = nb.ipam.prefixes.create(
prefix='192.168.3.0/24',
role=role_mgmt.id,
description='Management network for Matrix cluster'
)
```
### Reserve Important IPs
**✅ Explicitly reserve gateway, broadcast, network addresses:**
```python
# Gateway
gateway = nb.ipam.ip_addresses.create(
address='192.168.3.1/24',
status='active',
role='anycast',
description='Management network gateway'
)
# DNS servers
dns1 = nb.ipam.ip_addresses.create(
address='192.168.3.2/24',
status='reserved',
description='Primary DNS server'
)
```
### Use Prefixes as IP Pools
**✅ Enable automatic IP assignment:**
```python
prefix = nb.ipam.prefixes.create(
prefix='192.168.3.0/24',
is_pool=True, # ✅ Allow automatic IP assignment
...
)
# Get next available IP
ip = prefix.available_ips.create(dns_name='docker-02.spaceships.work')
```
**❌ Don't manually track available IPs** - let NetBox do it.
### IP Status Values
Use appropriate status:
| Status | Use Case |
|--------|----------|
| `active` | Currently in use |
| `reserved` | Reserved for specific purpose |
| `deprecated` | Planned for decommission |
| `dhcp` | Managed by DHCP server |
```python
# Production VM
ip = nb.ipam.ip_addresses.create(address='192.168.3.10/24', status='active')
# Future expansion
ip = nb.ipam.ip_addresses.create(address='192.168.3.50/24', status='reserved')
```
### VRF for Isolation
**Use VRFs for true isolation:**
```python
# Management VRF (enforce unique IPs)
vrf_mgmt = nb.ipam.vrfs.create(
name='management',
enforce_unique=True,
description='Management network VRF'
)
# Lab VRF (allow overlapping IPs)
vrf_lab = nb.ipam.vrfs.create(
name='lab',
enforce_unique=False,
description='Lab/testing VRF'
)
```
**When to use VRFs:**
- Multiple environments (prod, dev, lab)
- Overlapping IP ranges
- Security isolation
---
## Device Management
### Create Device Types First
**✅ Always create device type before devices:**
```python
# 1. Create manufacturer
manufacturer = nb.dcim.manufacturers.get(slug='minisforum')
if not manufacturer:
manufacturer = nb.dcim.manufacturers.create(
name='MINISFORUM',
slug='minisforum'
)
# 2. Create device type
device_type = nb.dcim.device_types.create(
manufacturer=manufacturer.id,
model='MS-A2',
slug='ms-a2',
u_height=0, # Not rack mounted
is_full_depth=False
)
# 3. Create device
device = nb.dcim.devices.create(
name='foxtrot',
device_type=device_type.id,
...
)
```
### Use Device Roles Consistently
**Create specific roles:**
```python
roles = [
('Proxmox Node', 'proxmox-node', '2196f3'), # Blue
('Docker Host', 'docker-host', '4caf50'), # Green
('K8s Master', 'k8s-master', 'ff9800'), # Orange
('K8s Worker', 'k8s-worker', 'ffc107'), # Amber
('Storage', 'storage', '9c27b0'), # Purple
]
for name, slug, color in roles:
nb.dcim.device_roles.create(
name=name,
slug=slug,
color=color,
vm_role=True # If role applies to VMs too
)
```
**✅ Consistent naming helps automation:**
```python
# Get all Proxmox nodes
proxmox_nodes = nb.dcim.devices.filter(role='proxmox-node')
# Get all Kubernetes workers
k8s_workers = nb.virtualization.virtual_machines.filter(role='k8s-worker')
```
### Always Set Primary IP
**✅ Set primary IP after creating device and IPs:**
```python
# Create device
device = nb.dcim.devices.create(name='foxtrot', ...)
# Create interface
iface = nb.dcim.interfaces.create(device=device.id, name='enp2s0', ...)
# Create IP
ip = nb.ipam.ip_addresses.create(
address='192.168.3.5/24',
assigned_object_type='dcim.interface',
assigned_object_id=iface.id
)
# ✅ Set as primary (critical for automation!)
device.primary_ip4 = ip.id
device.save()
```
**Why?** Primary IP is used by:
- Ansible dynamic inventory
- Monitoring tools
- DNS automation
### Document Interfaces
**✅ Include descriptions:**
```python
# Management
mgmt = nb.dcim.interfaces.create(
device=device.id,
name='enp2s0',
type='2.5gbase-t',
mtu=1500,
description='Management interface (vmbr0)',
tags=[{'name': 'management'}]
)
# CEPH public
ceph_pub = nb.dcim.interfaces.create(
device=device.id,
name='enp1s0f0',
type='10gbase-x-sfpp',
mtu=9000,
description='CEPH public network (vmbr1)',
tags=[{'name': 'ceph-public'}, {'name': 'jumbo-frames'}]
)
```
---
## Virtualization
### Create Cluster First
**✅ Create cluster before VMs:**
```python
# 1. Get/create cluster type
cluster_type = nb.virtualization.cluster_types.get(slug='proxmox')
if not cluster_type:
cluster_type = nb.virtualization.cluster_types.create(
name='Proxmox VE',
slug='proxmox'
)
# 2. Create cluster
cluster = nb.virtualization.clusters.create(
name='Matrix',
type=cluster_type.id,
site=site.id,
description='3-node Proxmox VE 9.x cluster'
)
# 3. Create VMs in cluster
vm = nb.virtualization.virtual_machines.create(
name='docker-01',
cluster=cluster.id,
...
)
```
### Standardize VM Sizing
**✅ Use consistent resource allocations:**
| Role | vCPUs | Memory (MB) | Disk (GB) |
|------|-------|-------------|-----------|
| Small (dev) | 2 | 2048 | 20 |
| Medium (app) | 4 | 8192 | 100 |
| Large (database) | 8 | 16384 | 200 |
| XL (compute) | 16 | 32768 | 500 |
```python
VM_SIZES = {
'small': {'vcpus': 2, 'memory': 2048, 'disk': 20},
'medium': {'vcpus': 4, 'memory': 8192, 'disk': 100},
'large': {'vcpus': 8, 'memory': 16384, 'disk': 200},
}
# Create VM with standard size
vm = nb.virtualization.virtual_machines.create(
name='docker-01',
cluster=cluster.id,
**VM_SIZES['medium']
)
```
### VM Network Configuration
**✅ Complete network setup:**
```python
# 1. Create VM
vm = nb.virtualization.virtual_machines.create(...)
# 2. Create interface
vm_iface = nb.virtualization.interfaces.create(
virtual_machine=vm.id,
name='eth0',
type='virtual',
enabled=True,
mtu=1500
)
# 3. Assign IP from pool
prefix = nb.ipam.prefixes.get(prefix='192.168.3.0/24')
vm_ip = prefix.available_ips.create(
dns_name='docker-01-nexus.spaceships.work',
assigned_object_type='virtualization.vminterface',
assigned_object_id=vm_iface.id,
tags=[{'name': 'production-dns'}] # ✅ Triggers PowerDNS sync
)
# 4. Set as primary IP
vm.primary_ip4 = vm_ip.id
vm.save()
```
---
## Tagging Strategy
### Tag Categories
Organize tags by purpose:
**Infrastructure Type:**
- `proxmox-node`, `ceph-node`, `docker-host`, `k8s-master`, `k8s-worker`
**Environment:**
- `production`, `staging`, `development`, `lab`
**DNS Automation:**
- `production-dns`, `lab-dns` (triggers PowerDNS sync)
**Management:**
- `terraform`, `ansible`, `manual`
**Networking:**
- `management`, `ceph-public`, `ceph-private`, `jumbo-frames`
### Tag Naming Convention
**✅ Lowercase with hyphens:**
```python
tags = [
{'name': 'proxmox-node'}, # ✅ Good
{'name': 'production-dns'}, # ✅ Good
{'name': 'Proxmox Node'}, # ❌ Bad (spaces, capitals)
{'name': 'production_dns'}, # ❌ Bad (underscores)
]
```
### Apply Tags Consistently
**✅ Tag at multiple levels:**
```python
# Tag device
device = nb.dcim.devices.create(
name='foxtrot',
tags=[{'name': 'proxmox-node'}, {'name': 'ceph-node'}, {'name': 'production'}]
)
# Tag interface
iface = nb.dcim.interfaces.create(
device=device.id,
name='enp1s0f0',
tags=[{'name': 'ceph-public'}, {'name': 'jumbo-frames'}]
)
# Tag IP
ip = nb.ipam.ip_addresses.create(
address='192.168.3.5/24',
tags=[{'name': 'production-dns'}, {'name': 'terraform'}]
)
```
**Why?** Enables granular filtering:
```bash
# Get all CEPH nodes
ansible-playbook -i netbox-inventory.yml setup-ceph.yml --limit tag_ceph_node
# Get all production DNS-enabled IPs
ips = nb.ipam.ip_addresses.filter(tag='production-dns')
```
---
## Security
### API Token Management
**✅ Store tokens in Infisical (Virgo-Core standard):**
```python
from infisical import InfisicalClient
def get_netbox_token() -> str:
"""Get NetBox API token from Infisical."""
client = InfisicalClient()
secret = client.get_secret(
secret_name="NETBOX_API_TOKEN",
project_id="7b832220-24c0-45bc-a5f1-ce9794a31259",
environment="prod",
path="/matrix"
)
return secret.secret_value
# Use token
nb = pynetbox.api('https://netbox.spaceships.work', token=get_netbox_token())
```
**❌ Never hardcode tokens:**
```python
# ❌ NEVER DO THIS
token = "a1b2c3d4e5f6..."
nb = pynetbox.api(url, token=token)
```
### Use Minimal Permissions
Create tokens with appropriate scopes:
| Use Case | Permissions |
|----------|-------------|
| Read-only queries | Read only |
| Terraform automation | Read + Write (DCIM, IPAM, Virtualization) |
| Full automation | Read + Write (all) |
| Emergency admin | Full access |
**✅ Create separate tokens for different purposes:**
```text
NETBOX_API_TOKEN_READONLY → Read-only queries
NETBOX_API_TOKEN_TERRAFORM → Terraform automation
NETBOX_API_TOKEN_ANSIBLE → Ansible dynamic inventory
```
### HTTPS Only
**✅ Always use HTTPS in production:**
```python
# ✅ Production
nb = pynetbox.api('https://netbox.spaceships.work', token=token)
# ❌ Never HTTP in production
nb = pynetbox.api('http://netbox.spaceships.work', token=token)
```
**For self-signed certs (dev/lab only):**
```python
# ⚠️ Dev/testing only
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
nb = pynetbox.api(
'https://netbox.local',
token=token,
ssl_verify=False # Only for self-signed certs in lab
)
```
### Rotate Tokens Regularly
**Best practice:** Rotate every 90 days
```bash
# 1. Create new token in NetBox UI
# 2. Update Infisical secret
infisical secrets set NETBOX_API_TOKEN="new-token-here"
# 3. Test new token
./tools/netbox_api_client.py sites list
# 4. Delete old token in NetBox UI
```
### Audit API Usage
**✅ Log API calls in production:**
```python
import logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
filename='/var/log/netbox-api.log'
)
logger = logging.getLogger(__name__)
def audit_api_call(action: str, resource: str, details: dict):
"""Log API calls for security audit."""
logger.info(f"API Call: {action} {resource} - User: {os.getenv('USER')} - {details}")
# Usage
ip = nb.ipam.ip_addresses.create(address='192.168.1.1/24')
audit_api_call('CREATE', 'ip-address', {'address': '192.168.1.1/24'})
```
---
## Performance
### Use Filtering Server-Side
**✅ Filter on server:**
```python
# ✅ Efficient: Server filters results
devices = nb.dcim.devices.filter(site='matrix', status='active')
```
**❌ Don't filter client-side:**
```python
# ❌ Inefficient: Downloads all devices then filters
all_devices = nb.dcim.devices.all()
matrix_devices = [d for d in all_devices if d.site.slug == 'matrix']
```
### Request Only Needed Fields
**✅ Use field selection:**
```python
# Get only specific fields
devices = nb.dcim.devices.filter(site='matrix', fields=['name', 'primary_ip4'])
```
### Use Pagination for Large Datasets
**✅ Process in batches:**
```python
# Paginate automatically
for device in nb.dcim.devices.filter(site='matrix'):
process_device(device) # pynetbox handles pagination
# Manual pagination for control
page_size = 100
offset = 0
while True:
devices = nb.dcim.devices.filter(limit=page_size, offset=offset)
if not devices:
break
for device in devices:
process_device(device)
offset += page_size
```
### Cache Lookups
**✅ Cache static data:**
```python
from functools import lru_cache
@lru_cache(maxsize=128)
def get_site(site_slug: str):
"""Cached site lookup."""
return nb.dcim.sites.get(slug=site_slug)
@lru_cache(maxsize=256)
def get_device_type(slug: str):
"""Cached device type lookup."""
return nb.dcim.device_types.get(slug=slug)
```
### Use Bulk Operations
**✅ Bulk create is faster:**
```python
# ✅ Fast: Bulk create
ips = [
{'address': f'192.168.3.{i}/24', 'status': 'active'}
for i in range(10, 20)
]
nb.ipam.ip_addresses.create(ips)
# ❌ Slow: Loop with individual creates
for i in range(10, 20):
nb.ipam.ip_addresses.create(address=f'192.168.3.{i}/24', status='active')
```
---
## API Integration
### Error Handling
**✅ Always handle errors:**
```python
import pynetbox
from requests.exceptions import HTTPError
try:
device = nb.dcim.devices.get(name='foxtrot')
if not device:
console.print("[yellow]Device not found[/yellow]")
return None
except HTTPError as e:
if e.response.status_code == 404:
console.print("[red]Resource not found[/red]")
elif e.response.status_code == 403:
console.print("[red]Permission denied[/red]")
else:
console.print(f"[red]HTTP Error: {e}[/red]")
sys.exit(1)
except pynetbox.RequestError as e:
console.print(f"[red]NetBox API Error: {e.error}[/red]")
sys.exit(1)
except Exception as e:
console.print(f"[red]Unexpected error: {e}[/red]")
sys.exit(1)
```
### Validate Before Creating
**✅ Validate input before API calls:**
```python
import ipaddress
import re
def validate_ip(ip_str: str) -> bool:
"""Validate IP address format."""
try:
ipaddress.ip_interface(ip_str)
return True
except ValueError:
return False
def validate_dns_name(name: str) -> bool:
"""Validate DNS naming convention."""
pattern = r'^[a-z0-9-]+-\d{2}-[a-z0-9-]+\.[a-z0-9.-]+$'
return bool(re.match(pattern, name))
# Use before API calls
if not validate_ip(ip_address):
raise ValueError(f"Invalid IP address: {ip_address}")
if not validate_dns_name(dns_name):
raise ValueError(f"Invalid DNS name: {dns_name}")
ip = nb.ipam.ip_addresses.create(address=ip_address, dns_name=dns_name)
```
### Check Before Create
**✅ Check existence before creating:**
```python
# Check if device exists
device = nb.dcim.devices.get(name='foxtrot')
if device:
console.print("[yellow]Device already exists, updating...[/yellow]")
device.status = 'active'
device.save()
else:
console.print("[green]Creating new device...[/green]")
device = nb.dcim.devices.create(name='foxtrot', ...)
```
---
## Automation Patterns
### Idempotent Operations
**✅ Design operations to be safely re-run:**
```python
def ensure_vm_exists(name: str, cluster: str, **kwargs) -> pynetbox.core.response.Record:
"""Ensure VM exists (idempotent)."""
# Check if exists
vm = nb.virtualization.virtual_machines.get(name=name)
if vm:
# Update if needed
updated = False
for key, value in kwargs.items():
if getattr(vm, key) != value:
setattr(vm, key, value)
updated = True
if updated:
vm.save()
console.print(f"[yellow]Updated VM: {name}[/yellow]")
else:
console.print(f"[green]VM unchanged: {name}[/green]")
return vm
else:
# Create new
vm = nb.virtualization.virtual_machines.create(
name=name,
cluster=nb.virtualization.clusters.get(name=cluster).id,
**kwargs
)
console.print(f"[green]Created VM: {name}[/green]")
return vm
```
### Terraform Integration
See [terraform-provider-guide.md](terraform-provider-guide.md) for complete examples.
**Key pattern:**
```hcl
# Use NetBox as data source
data "netbox_prefix" "management" {
prefix = "192.168.3.0/24"
}
# Create IP in NetBox via Terraform
resource "netbox_ip_address" "vm_ip" {
ip_address = cidrhost(data.netbox_prefix.management.prefix, 10)
dns_name = "docker-01-nexus.spaceships.work"
status = "active"
tags = ["terraform", "production-dns"]
}
```
### Ansible Dynamic Inventory
See [../workflows/ansible-dynamic-inventory.md](../workflows/ansible-dynamic-inventory.md).
**Key pattern:**
```yaml
# netbox-dynamic-inventory.yml
plugin: netbox.netbox.nb_inventory
api_endpoint: https://netbox.spaceships.work
token: !vault |
$ANSIBLE_VAULT;...
group_by:
- device_roles
- tags
- site
```
---
## Troubleshooting
### Common Issues
**Problem:** "Permission denied" errors
**Solution:** Check API token permissions
```bash
# Test token
curl -H "Authorization: Token YOUR_TOKEN" \
https://netbox.spaceships.work/api/
```
**Problem:** IP not syncing to PowerDNS
**Solution:** Check tags
```python
# IP must have tag matching zone rules
ip = nb.ipam.ip_addresses.get(address='192.168.3.10/24')
print(f"Tags: {[tag.name for tag in ip.tags]}")
# Must include 'production-dns' or matching tag
```
**Problem:** Slow API queries
**Solution:** Use filtering and pagination
```python
# ❌ Slow
all_devices = nb.dcim.devices.all()
# ✅ Fast
devices = nb.dcim.devices.filter(site='matrix', limit=50)
```
### Debug Mode
**Enable verbose logging:**
```python
import logging
# Enable debug logging
logging.basicConfig(level=logging.DEBUG)
# Now pynetbox will log all API calls
nb = pynetbox.api('https://netbox.spaceships.work', token=token)
devices = nb.dcim.devices.all()
```
---
## Related Documentation
- [NetBox API Guide](netbox-api-guide.md) - Complete API reference
- [NetBox Data Models](netbox-data-models.md) - Data model relationships
- [DNS Naming Conventions](../workflows/naming-conventions.md) - Naming rules
- [Terraform Provider Guide](terraform-provider-guide.md) - Terraform integration
- [Tools: netbox_api_client.py](../tools/netbox_api_client.py) - Working examples
---
**Next:** Review [API Integration Patterns](netbox-api-guide.md#api-integration)

1039
skills/netbox-powerdns-integration/reference/netbox-data-models.md Normal file
View File

File diff suppressed because it is too large Load Diff

300
skills/netbox-powerdns-integration/reference/sync-plugin-reference.md Normal file
View File

@@ -0,0 +1,300 @@
# NetBox PowerDNS Sync Plugin Reference
*Source: <https://github.com/ArnesSI/netbox-powerdns-sync*>
## Overview
A NetBox plugin that automatically generates DNS records in PowerDNS based on NetBox IP Address and Device objects.
## Features
- Automatically generates A, AAAA & PTR records
- Manages multiple DNS Zones across multiple PowerDNS servers
- Flexible rules to match NetBox IP Addresses into DNS zones
- Multiple options to generate DNS names from IP address or Device
- Scheduled sync of DNS zones
- Can add DNS records for new zones immediately
- Per-zone synchronization schedule
## DNS Name Generation
### Zone Matching Priority
When determining the zone for an IP Address, match rules are evaluated in this order:
1. Check if `IPAddress.dns_name` matches any zone
2. Check if IPAddress is assigned to Device/VirtualMachine and if its name matches any zone
3. Check if IPAddress is assigned to FHRPGroup and if its name matches any zone
4. Try to match based on assigned tags (in order):
- `IPAddress.tags`
- `Interface.tags`
- `VMInterface.tags`
- `Device.tags`
- `VirtualMachine.tags`
- `Device.device_role`
- `VM.role`
5. Use default zone if configured
### Name Generation Methods
Each zone can use multiple naming methods (tried in order):
1. **IP naming method** - Generate name from IP address
2. **Device naming method** - Generate name from Device/VirtualMachine
3. **FHRP group method** - Generate name from FHRP Group
## Installation
### Via pip
```bash
# Activate NetBox virtual environment
source /opt/netbox/venv/bin/activate
# Install plugin
pip install netbox-powerdns-sync
```
### From GitHub
```bash
pip install git+https://github.com/ArnesSI/netbox-powerdns-sync.git@master
```
### Configuration
Add to `/opt/netbox/netbox/netbox/configuration.py`:
```python
PLUGINS = [
'netbox_powerdns_sync'
]
PLUGINS_CONFIG = {
"netbox_powerdns_sync": {
"ttl_custom_field": "",
"powerdns_managed_record_comment": "netbox-powerdns-sync",
"post_save_enabled": False,
},
}
```
### Apply Migrations
```bash
cd /opt/netbox/netbox/
python3 manage.py migrate
python3 manage.py reindex --lazy
```
## Configuration Settings
| Setting | Default | Description |
|---------|---------|-------------|
| `ttl_custom_field` | `None` | Name of NetBox Custom field applied to IP Address objects for TTL |
| `powerdns_managed_record_comment` | `"netbox-powerdns-sync"` | Plugin only touches records with this comment. Set to `None` to manage all records |
| `post_save_enabled` | `False` | Immediately create DNS records when creating/updating IP Address, Device, or FHRP Group |
### Custom TTL Field
To set TTL per DNS record:
1. Create NetBox Custom Field:
- Type: Integer
- Apply to: IP Address objects
- Name: e.g., "dns_ttl"
2. Set in plugin config:
```python
"ttl_custom_field": "dns_ttl"
```
3. Set TTL value on IP Address objects in NetBox
## Compatibility
| NetBox Version | Plugin Version |
|---------------|----------------|
| 3.5.0-7 | 0.0.1 - 0.0.6 |
| 3.5.8 | 0.0.7 |
| 3.6.x | 0.8.0 |
## Usage Workflow
### 1. Configure DNS Zones in NetBox
Create zones in the plugin interface with:
- Zone name (e.g., `spaceships.work`)
- PowerDNS server connection
- Tag matching rules
- DNS name generation method
### 2. Tag Resources
Apply tags to IP Addresses, Devices, or Interfaces to match zones:
```python
# Example: Tag IP for specific zone
ipaddress.tags.add("production-dns")
```
### 3. Schedule Sync
Configure sync schedule for each zone:
- Immediate (on save)
- Scheduled (cron-style)
- Manual only
### 4. Monitor Sync Results
View sync results in NetBox:
- Records created
- Records updated
- Records deleted
- Sync errors
## Best Practices
### DNS Naming Conventions
For homelab naming like `docker-01-nexus.spaceships.work`:
1. Use Device name as base: `docker-01-nexus`
2. Zone maps to domain: `spaceships.work`
3. Set device naming method in zone config
### Tag Organization
```python
# Production resources
tags: ["production", "dns-auto"]
# Development resources
tags: ["development", "dns-dev"]
```
### TTL Strategy
- Default TTL in zone: 300 (5 minutes)
- Override with custom field for specific records
- Longer TTL for stable infrastructure (3600)
- Shorter TTL for dynamic services (60-300)
### PowerDNS Server Management
- Configure multiple PowerDNS servers for HA
- Use different servers for different zones
- Monitor PowerDNS API connectivity
## Integration Patterns
### With Terraform
Use NetBox as data source, sync DNS automatically:
```hcl
# Terraform creates resource in NetBox
resource "netbox_ip_address" "server" {
ip_address = "192.168.1.100/24"
dns_name = "docker-01-nexus"
tags = ["production-dns"]
}
# Plugin automatically creates DNS in PowerDNS
# A record: docker-01-nexus.spaceships.work -> 192.168.1.100
# PTR record: 100.1.168.192.in-addr.arpa -> docker-01-nexus.spaceships.work
```
### With Ansible
Use NetBox dynamic inventory with automatic DNS:
```yaml
---
# Ansible creates VM in Proxmox
- name: Create VM
proxmox_kvm:
name: docker-01-nexus
# ... vm config ...
# Add to NetBox via API
- name: Register in NetBox
netbox.netbox.netbox_ip_address:
data:
address: "192.168.1.100/24"
dns_name: "docker-01-nexus"
tags:
- production-dns
# DNS records created automatically by plugin
```
## Troubleshooting
### Records Not Syncing
1. Check zone matching rules
2. Verify tags applied correctly
3. Check PowerDNS API connectivity
4. Review sync results for errors
### Duplicate Records
If `powerdns_managed_record_comment` is `None`, plugin manages ALL records. Set a comment to limit scope:
```python
"powerdns_managed_record_comment": "netbox-managed"
```
### Performance Issues
- Disable `post_save_enabled` for large environments
- Use scheduled sync instead
- Batch changes before sync
### Name Generation Not Working
1. Check zone name generation method configuration
2. Verify Device/IP naming follows expected pattern
3. Test with manual sync first
## API Endpoints
Plugin adds REST API endpoints:
- `/api/plugins/netbox-powerdns-sync/zones/` - List/manage zones
- `/api/plugins/netbox-powerdns-sync/servers/` - PowerDNS servers
- `/api/plugins/netbox-powerdns-sync/sync-results/` - Sync history
## Example Configuration
### Zone for Production
```python
zone_config = {
"name": "spaceships.work",
"server": powerdns_server_prod,
"default_ttl": 300,
"naming_methods": ["device", "ip"],
"tag_match": ["production-dns"],
"auto_sync": True,
"sync_schedule": "*/15 * * * *" # Every 15 minutes
}
```
### Zone for Lab
```python
zone_config = {
"name": "lab.spaceships.work",
"server": powerdns_server_dev,
"default_ttl": 60,
"naming_methods": ["ip", "device"],
"tag_match": ["lab-dns"],
"auto_sync": False # Manual sync only
}
```

429
skills/netbox-powerdns-integration/reference/terraform-provider-guide.md Normal file
View File

@@ -0,0 +1,429 @@
# Terraform NetBox Provider Guide
*Source: <https://registry.terraform.io/providers/e-breuninger/netbox/latest/docs*>
## Overview
The Terraform NetBox provider enables full lifecycle management of NetBox resources using Infrastructure as Code.
## Version Compatibility
| NetBox Version | Provider Version |
|---------------|------------------|
| v4.3.0 - 4.4.0 | v5.0.0 and up |
| v4.2.2 - 4.2.9 | v4.0.0 - 4.3.1 |
| v4.1.0 - 4.1.11 | v3.10.0 - 3.11.1 |
| v4.0.0 - 4.0.11 | v3.9.0 - 3.9.2 |
| v3.7.0 - 3.7.8 | v3.8.0 - 3.8.9 |
| v3.6.0 - 3.6.9 | v3.7.0 - 3.7.7 |
| v3.5.1 - 3.5.9 | v3.6.x |
**Important**: NetBox makes breaking API changes even in non-major releases. Match provider version to NetBox version.
## Provider Configuration
### Basic Setup
```hcl
terraform {
required_providers {
netbox = {
source = "e-breuninger/netbox"
version = "~> 5.0.0" # Match your NetBox version
}
}
}
provider "netbox" {
server_url = "https://netbox.spaceships.work"
api_token = var.netbox_api_token
}
```
### Environment Variables
Configure via environment instead of hard-coding:
```bash
export NETBOX_SERVER_URL="https://netbox.spaceships.work"
export NETBOX_API_TOKEN="your-api-token-here"
```
```hcl
# Provider auto-reads from environment
provider "netbox" {}
```
## Configuration Schema
### Required
- `api_token` (String) - NetBox API authentication token
- Environment: `NETBOX_API_TOKEN`
- `server_url` (String) - NetBox server URL (with scheme and port)
- Environment: `NETBOX_SERVER_URL`
### Optional
- `allow_insecure_https` (Boolean) - Allow invalid certificates
- Environment: `NETBOX_ALLOW_INSECURE_HTTPS`
- Default: `false`
- `ca_cert_file` (String) - Path to PEM-encoded CA certificate
- Environment: `NETBOX_CA_CERT_FILE`
- `default_tags` (Set of String) - Tags added to every resource
- Useful for tracking Terraform-managed resources
- `headers` (Map of String) - Custom headers for all requests
- Environment: `NETBOX_HEADERS`
- `request_timeout` (Number) - HTTP request timeout (seconds)
- Environment: `NETBOX_REQUEST_TIMEOUT`
- `skip_version_check` (Boolean) - Skip NetBox version validation
- Environment: `NETBOX_SKIP_VERSION_CHECK`
- Default: `false`
- Useful for: Testing, unsupported versions
- `strip_trailing_slashes_from_url` (Boolean) - Auto-fix URL format
- Environment: `NETBOX_STRIP_TRAILING_SLASHES_FROM_URL`
- Default: `true`
## Usage Examples
### Create IP Address
```hcl
resource "netbox_ip_address" "server_ip" {
ip_address = "192.168.1.100/24"
dns_name = "docker-01-nexus.spaceships.work"
status = "active"
description = "Docker host - Nexus container registry"
tags = [
"terraform",
"production",
"dns-auto"
]
}
```
### Create Device
```hcl
resource "netbox_device" "proxmox_node" {
name = "foxtrot"
device_type = netbox_device_type.minisforum_ms_a2.id
role = netbox_device_role.hypervisor.id
site = netbox_site.homelab.id
primary_ip4 = netbox_ip_address.foxtrot_mgmt.id
tags = [
"terraform",
"proxmox",
"cluster-matrix"
]
comments = "Proxmox node in Matrix cluster - AMD Ryzen 9 9955HX"
}
```
### Create Prefix
```hcl
resource "netbox_prefix" "vlan_30_mgmt" {
prefix = "192.168.3.0/24"
vlan = netbox_vlan.management.id
status = "active"
description = "Management network for Proxmox cluster"
tags = [
"terraform",
"mgmt-network"
]
}
```
### Create VLAN
```hcl
resource "netbox_vlan" "management" {
vid = 30
name = "MGMT"
site = netbox_site.homelab.id
status = "active"
description = "Management VLAN for infrastructure"
tags = ["terraform"]
}
```
## Integration Patterns
### With Proxmox Provider
```hcl
# Create VM in Proxmox
resource "proxmox_vm_qemu" "docker_host" {
name = "docker-01-nexus"
target_node = "foxtrot"
# ... vm config ...
}
# Register in NetBox
resource "netbox_ip_address" "docker_host_ip" {
ip_address = "192.168.1.100/24"
dns_name = "${proxmox_vm_qemu.docker_host.name}.spaceships.work"
description = "Docker host for Nexus registry"
tags = [
"terraform",
"production-dns",
"docker-host"
]
}
# DNS record auto-created by netbox-powerdns-sync plugin
```
### Data Sources
Query existing NetBox data:
```hcl
# Get all production IPs
data "netbox_ip_addresses" "production" {
filter {
name = "tag"
value = "production"
}
}
# Get device details
data "netbox_device" "proxmox_node" {
name = "foxtrot"
}
# Use in other resources
resource "proxmox_vm_qemu" "new_vm" {
target_node = data.netbox_device.proxmox_node.name
# ... config ...
}
```
### Dynamic Inventory for Ansible
```hcl
# Export NetBox data for Ansible
output "ansible_inventory" {
value = {
for device in data.netbox_devices.all.devices :
device.name => {
ansible_host = device.primary_ip4_address
device_role = device.role
site = device.site
tags = device.tags
}
}
}
```
Save to file:
```bash
terraform output -json ansible_inventory > inventory.json
```
## Best Practices
### 1. Use Default Tags
Track all Terraform-managed resources:
```hcl
provider "netbox" {
server_url = var.netbox_url
api_token = var.netbox_token
default_tags = ["terraform", "iac"]
}
```
### 2. Organize with Modules
```hcl
module "vm_network" {
source = "./modules/netbox-vm"
vm_name = "docker-01"
ip_address = "192.168.1.100/24"
vlan_id = 30
dns_zone = "spaceships.work"
}
```
### 3. Use Variables for Secrets
Never hard-code tokens:
```hcl
variable "netbox_api_token" {
description = "NetBox API token"
type = string
sensitive = true
}
```
### 4. State Management
Use remote state for team collaboration:
```hcl
terraform {
backend "s3" {
bucket = "terraform-state"
key = "netbox/terraform.tfstate"
region = "us-east-1"
}
}
```
### 5. Version Pinning
Pin provider version to prevent breaking changes:
```hcl
terraform {
required_providers {
netbox = {
source = "e-breuninger/netbox"
version = "= 5.0.0" # Exact version
}
}
}
```
## Common Workflows
### 1. VM Provisioning Workflow
```hcl
# 1. Reserve IP in NetBox
resource "netbox_ip_address" "vm_ip" {
ip_address = "192.168.1.100/24"
dns_name = "app-server.spaceships.work"
status = "reserved"
description = "Reserved for new application server"
}
# 2. Create VM in Proxmox
resource "proxmox_vm_qemu" "app_server" {
# ... config using netbox_ip_address.vm_ip.ip_address ...
}
# 3. Mark IP as active
resource "netbox_ip_address" "vm_ip_active" {
ip_address = netbox_ip_address.vm_ip.ip_address
status = "active" # Update status
description = "Application server - deployed ${timestamp()}"
}
```
### 2. DNS Automation Workflow
```hcl
# Create IP with DNS name and auto-DNS tag
resource "netbox_ip_address" "service" {
ip_address = "192.168.1.200/24"
dns_name = "service-01-api.spaceships.work"
tags = [
"terraform",
"production-dns" # Triggers netbox-powerdns-sync
]
}
# DNS records created automatically by plugin
# No manual DNS configuration needed
```
### 3. Network Documentation Workflow
```hcl
# Document entire network in NetBox
module "network_documentation" {
source = "./modules/network"
site_name = "homelab"
vlans = {
"mgmt" = { vid = 30, prefix = "192.168.3.0/24" }
"storage" = { vid = 40, prefix = "192.168.5.0/24" }
"ceph" = { vid = 50, prefix = "192.168.7.0/24" }
}
devices = var.proxmox_nodes
}
```
## Troubleshooting
### Version Mismatch Warning
```text
Warning: NetBox version X.Y.Z is not officially supported by provider version A.B.C
```
**Solution**: Use matching provider version or set `skip_version_check = true`
### API Authentication Errors
```text
Error: authentication failed
```
**Solution**:
1. Verify `api_token` is valid
2. Check token has required permissions
3. Ensure `server_url` includes scheme (`https://`)
### SSL Certificate Errors
```text
Error: x509: certificate signed by unknown authority
```
**Solution**:
```hcl
provider "netbox" {
server_url = var.netbox_url
api_token = var.netbox_token
ca_cert_file = "/path/to/ca.pem"
# OR for dev/testing only:
# allow_insecure_https = true
}
```
### Trailing Slash Issues
```text
Error: invalid URL format
```
**Solution**: Remove trailing slashes from `server_url` or let provider auto-fix:
```hcl
provider "netbox" {
server_url = "https://netbox.example.com" # No trailing slash
strip_trailing_slashes_from_url = true # Auto-fix if present
}
```
## Further Resources
- [Provider GitHub Repository](https://github.com/e-breuninger/terraform-provider-netbox)
- [NetBox Official Documentation](https://docs.netbox.dev/)
- [NetBox API Reference](https://demo.netbox.dev/api/docs/)

657
skills/netbox-powerdns-integration/tools/netbox_api_client.py Executable file
View File

@@ -0,0 +1,657 @@
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "pynetbox>=7.0.0",
# "infisicalsdk>=1.0.3",
# "rich>=13.0.0",
# "typer>=0.9.0",
# ]
# ///
"""
NetBox API Client
Complete working example of NetBox REST API usage with pynetbox.
Demonstrates authentication, queries, filtering, error handling, and best practices.
Usage:
# List all sites
./netbox_api_client.py sites list
# Get specific device
./netbox_api_client.py devices get --name foxtrot
# List VMs in cluster
./netbox_api_client.py vms list --cluster matrix
# Query IPs by DNS name
./netbox_api_client.py ips query --dns-name docker-01
# Get available IPs in prefix
./netbox_api_client.py prefixes available --prefix 192.168.3.0/24
# Create VM with IP
./netbox_api_client.py vms create --name test-vm --cluster matrix --ip 192.168.3.100
Example:
./netbox_api_client.py devices get --name foxtrot --output json
"""
import os
import sys
from typing import Optional
from dataclasses import dataclass
import typer
from rich import print as rprint
from rich.console import Console
from rich.table import Table
from rich.panel import Panel
import pynetbox
from infisical_sdk import InfisicalSDKClient
app = typer.Typer(help="NetBox API Client for Matrix cluster infrastructure")
console = Console()
# ============================================================================
# Configuration and Authentication
# ============================================================================
@dataclass
class NetBoxConfig:
"""NetBox connection configuration."""
url: str
token: str
ssl_verify: bool = True
def get_netbox_client() -> pynetbox.api:
"""
Get authenticated NetBox API client.
Uses Infisical SDK with Universal Auth to securely retrieve API token.
Requires INFISICAL_CLIENT_ID and INFISICAL_CLIENT_SECRET environment variables.
Returns:
pynetbox.api: Authenticated NetBox client
Raises:
ValueError: If token cannot be retrieved or is empty
typer.Exit: On connection or authentication errors (CLI exits)
"""
try:
# Initialize Infisical SDK client
client = InfisicalSDKClient(host="https://app.infisical.com")
# Authenticate using Universal Auth (machine identity)
client_id = os.getenv("INFISICAL_CLIENT_ID")
client_secret = os.getenv("INFISICAL_CLIENT_SECRET")
if not client_id or not client_secret:
console.print(
"[red]INFISICAL_CLIENT_ID and INFISICAL_CLIENT_SECRET environment variables required[/red]")
raise typer.Exit(1)
client.auth.universal_auth.login(
client_id=client_id,
client_secret=client_secret
)
# Get NetBox API token from Infisical
secret = client.secrets.get_secret_by_name(
secret_name="NETBOX_API_TOKEN",
project_id="7b832220-24c0-45bc-a5f1-ce9794a31259",
environment_slug="prod",
secret_path="/matrix"
)
token = secret.secretValue
if not token:
console.print("[red]NETBOX_API_TOKEN is empty in Infisical[/red]")
raise ValueError("NETBOX_API_TOKEN is empty")
config = NetBoxConfig(
url="https://netbox.spaceships.work",
token=token,
ssl_verify=True
)
return pynetbox.api(config.url, token=config.token)
except ValueError:
# ValueError already logged above, re-raise to propagate
raise
except Exception as e:
console.print(f"[red]Failed to connect to NetBox: {e}[/red]")
raise typer.Exit(1)
# ============================================================================
# Sites Commands
# ============================================================================
sites_app = typer.Typer(help="Manage NetBox sites")
@sites_app.command("list")
def sites_list(
output: str = typer.Option("table", help="Output format: table or json")
):
"""List all sites in NetBox."""
nb = get_netbox_client()
try:
sites = nb.dcim.sites.all()
if output == "json":
import json
sites_data = [
{
"id": s.id,
"name": s.name,
"slug": s.slug,
"status": s.status.value if hasattr(s.status, 'value') else str(s.status),
"description": s.description or ""
}
for s in sites
]
print(json.dumps(sites_data, indent=2))
else:
table = Table(title="NetBox Sites")
table.add_column("ID", style="cyan")
table.add_column("Name", style="green")
table.add_column("Slug", style="yellow")
table.add_column("Status")
table.add_column("Description")
for site in sites:
table.add_row(
str(site.id),
site.name,
site.slug,
site.status.value if hasattr(
site.status, 'value') else str(site.status),
site.description or ""
)
console.print(table)
console.print(f"\n[green]Total sites: {len(sites)}[/green]")
except Exception as e:
console.print(f"[red]Error listing sites: {e}[/red]")
sys.exit(1)
@sites_app.command("get")
def sites_get(
slug: str = typer.Option(..., help="Site slug"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""Get specific site by slug."""
nb = get_netbox_client()
try:
site = nb.dcim.sites.get(slug=slug)
if not site:
console.print(f"[red]Site '{slug}' not found[/red]")
sys.exit(1)
if output == "json":
import json
site_data = {
"id": site.id,
"name": site.name,
"slug": site.slug,
"status": site.status.value if hasattr(site.status, 'value') else str(site.status),
"description": site.description or "",
"tags": [tag.name for tag in site.tags] if site.tags else []
}
print(json.dumps(site_data, indent=2))
else:
console.print(Panel(
f"[green]Name:[/green] {site.name}\n"
f"[green]Slug:[/green] {site.slug}\n"
f"[green]Status:[/green] {site.status}\n"
f"[green]Description:[/green] {site.description or 'N/A'}\n"
f"[green]Tags:[/green] {', '.join([tag.name for tag in site.tags]) if site.tags else 'None'}",
title=f"Site: {site.name}",
border_style="green"
))
except Exception as e:
console.print(f"[red]Error getting site: {e}[/red]")
sys.exit(1)
# ============================================================================
# Devices Commands
# ============================================================================
devices_app = typer.Typer(help="Manage NetBox devices")
@devices_app.command("list")
def devices_list(
site: Optional[str] = typer.Option(None, help="Filter by site slug"),
role: Optional[str] = typer.Option(None, help="Filter by device role"),
tag: Optional[str] = typer.Option(None, help="Filter by tag"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""List devices with optional filtering."""
nb = get_netbox_client()
try:
# Build filter
filters = {}
if site:
filters['site'] = site
if role:
filters['role'] = role
if tag:
filters['tag'] = tag
devices = nb.dcim.devices.filter(
**filters) if filters else nb.dcim.devices.all()
if output == "json":
import json
devices_data = [
{
"id": d.id,
"name": d.name,
"site": d.site.name if d.site else None,
"role": d.device_role.name if d.device_role else None,
"status": d.status.value if hasattr(d.status, 'value') else str(d.status),
"primary_ip": str(d.primary_ip4.address) if d.primary_ip4 else None
}
for d in devices
]
print(json.dumps(devices_data, indent=2))
else:
table = Table(title="NetBox Devices")
table.add_column("ID", style="cyan")
table.add_column("Name", style="green")
table.add_column("Site", style="yellow")
table.add_column("Role")
table.add_column("Status")
table.add_column("Primary IP")
for device in devices:
table.add_row(
str(device.id),
device.name,
device.site.name if device.site else "N/A",
device.device_role.name if device.device_role else "N/A",
device.status.value if hasattr(
device.status, 'value') else str(device.status),
str(device.primary_ip4.address) if device.primary_ip4 else "N/A"
)
console.print(table)
console.print(
f"\n[green]Total devices: {len(list(devices))}[/green]")
except Exception as e:
console.print(f"[red]Error listing devices: {e}[/red]")
sys.exit(1)
@devices_app.command("get")
def devices_get(
name: str = typer.Option(..., help="Device name"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""Get device details including interfaces."""
nb = get_netbox_client()
try:
device = nb.dcim.devices.get(name=name)
if not device:
console.print(f"[red]Device '{name}' not found[/red]")
sys.exit(1)
# Get interfaces
interfaces = nb.dcim.interfaces.filter(device=name)
if output == "json":
import json
device_data = {
"id": device.id,
"name": device.name,
"site": device.site.name if device.site else None,
"role": device.device_role.name if device.device_role else None,
"status": device.status.value if hasattr(device.status, 'value') else str(device.status),
"primary_ip4": str(device.primary_ip4.address) if device.primary_ip4 else None,
"interfaces": [
{
"name": iface.name,
"type": iface.type.value if hasattr(iface.type, 'value') else str(iface.type),
"enabled": iface.enabled,
"mtu": iface.mtu
}
for iface in interfaces
]
}
print(json.dumps(device_data, indent=2))
else:
# Device info
console.print(Panel(
f"[green]Name:[/green] {device.name}\n"
f"[green]Site:[/green] {device.site.name if device.site else 'N/A'}\n"
f"[green]Role:[/green] {device.device_role.name if device.device_role else 'N/A'}\n"
f"[green]Status:[/green] {device.status}\n"
f"[green]Primary IP:[/green] {device.primary_ip4.address if device.primary_ip4 else 'N/A'}",
title=f"Device: {device.name}",
border_style="green"
))
# Interfaces table
if interfaces:
iface_table = Table(title="Interfaces")
iface_table.add_column("Name", style="cyan")
iface_table.add_column("Type")
iface_table.add_column("Enabled")
iface_table.add_column("MTU")
for iface in interfaces:
iface_table.add_row(
iface.name,
iface.type.value if hasattr(
iface.type, 'value') else str(iface.type),
"✓" if iface.enabled else "✗",
str(iface.mtu) if iface.mtu else "default"
)
console.print("\n", iface_table)
except Exception as e:
console.print(f"[red]Error getting device: {e}[/red]")
sys.exit(1)
# ============================================================================
# Virtual Machines Commands
# ============================================================================
vms_app = typer.Typer(help="Manage NetBox virtual machines")
@vms_app.command("list")
def vms_list(
cluster: Optional[str] = typer.Option(None, help="Filter by cluster"),
tag: Optional[str] = typer.Option(None, help="Filter by tag"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""List virtual machines."""
nb = get_netbox_client()
try:
filters = {}
if cluster:
filters['cluster'] = cluster
if tag:
filters['tag'] = tag
vms = nb.virtualization.virtual_machines.filter(
**filters) if filters else nb.virtualization.virtual_machines.all()
if output == "json":
import json
vms_data = [
{
"id": vm.id,
"name": vm.name,
"cluster": vm.cluster.name if vm.cluster else None,
"vcpus": vm.vcpus,
"memory": vm.memory,
"status": vm.status.value if hasattr(vm.status, 'value') else str(vm.status),
"primary_ip": str(vm.primary_ip4.address) if vm.primary_ip4 else None
}
for vm in vms
]
print(json.dumps(vms_data, indent=2))
else:
table = Table(title="NetBox Virtual Machines")
table.add_column("ID", style="cyan")
table.add_column("Name", style="green")
table.add_column("Cluster", style="yellow")
table.add_column("vCPUs")
table.add_column("Memory (MB)")
table.add_column("Status")
table.add_column("Primary IP")
for vm in vms:
table.add_row(
str(vm.id),
vm.name,
vm.cluster.name if vm.cluster else "N/A",
str(vm.vcpus) if vm.vcpus else "N/A",
str(vm.memory) if vm.memory else "N/A",
vm.status.value if hasattr(
vm.status, 'value') else str(vm.status),
str(vm.primary_ip4.address) if vm.primary_ip4 else "N/A"
)
console.print(table)
console.print(f"\n[green]Total VMs: {len(list(vms))}[/green]")
except Exception as e:
console.print(f"[red]Error listing VMs: {e}[/red]")
sys.exit(1)
@vms_app.command("get")
def vms_get(
name: str = typer.Option(..., help="VM name"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""Get VM details including interfaces and IPs."""
nb = get_netbox_client()
try:
vm = nb.virtualization.virtual_machines.get(name=name)
if not vm:
console.print(f"[red]VM '{name}' not found[/red]")
sys.exit(1)
# Get interfaces
interfaces = nb.virtualization.interfaces.filter(
virtual_machine_id=vm.id)
if output == "json":
import json
vm_data = {
"id": vm.id,
"name": vm.name,
"cluster": vm.cluster.name if vm.cluster else None,
"vcpus": vm.vcpus,
"memory": vm.memory,
"disk": vm.disk,
"status": vm.status.value if hasattr(vm.status, 'value') else str(vm.status),
"primary_ip4": str(vm.primary_ip4.address) if vm.primary_ip4 else None,
"interfaces": [
{
"name": iface.name,
"enabled": iface.enabled,
"mtu": iface.mtu
}
for iface in interfaces
]
}
print(json.dumps(vm_data, indent=2))
else:
console.print(Panel(
f"[green]Name:[/green] {vm.name}\n"
f"[green]Cluster:[/green] {vm.cluster.name if vm.cluster else 'N/A'}\n"
f"[green]vCPUs:[/green] {vm.vcpus or 'N/A'}\n"
f"[green]Memory:[/green] {vm.memory or 'N/A'} MB\n"
f"[green]Disk:[/green] {vm.disk or 'N/A'} GB\n"
f"[green]Status:[/green] {vm.status}\n"
f"[green]Primary IP:[/green] {vm.primary_ip4.address if vm.primary_ip4 else 'N/A'}",
title=f"VM: {vm.name}",
border_style="green"
))
if interfaces:
iface_table = Table(title="Interfaces")
iface_table.add_column("Name", style="cyan")
iface_table.add_column("Enabled")
iface_table.add_column("MTU")
for iface in interfaces:
iface_table.add_row(
iface.name,
"✓" if iface.enabled else "✗",
str(iface.mtu) if iface.mtu else "default"
)
console.print("\n", iface_table)
except Exception as e:
console.print(f"[red]Error getting VM: {e}[/red]")
sys.exit(1)
# ============================================================================
# IP Addresses Commands
# ============================================================================
ips_app = typer.Typer(help="Manage NetBox IP addresses")
@ips_app.command("query")
def ips_query(
dns_name: Optional[str] = typer.Option(
None, help="Filter by DNS name (partial match)"),
address: Optional[str] = typer.Option(None, help="Filter by IP address"),
tag: Optional[str] = typer.Option(None, help="Filter by tag"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""Query IP addresses."""
nb = get_netbox_client()
try:
filters = {}
if dns_name:
filters['dns_name__ic'] = dns_name # Case-insensitive contains
if address:
filters['address'] = address
if tag:
filters['tag'] = tag
if not filters:
console.print(
"[yellow]Please provide at least one filter[/yellow]")
sys.exit(0)
ips = nb.ipam.ip_addresses.filter(**filters)
if output == "json":
import json
ips_data = [
{
"id": ip.id,
"address": str(ip.address),
"dns_name": ip.dns_name or "",
"status": ip.status.value if hasattr(ip.status, 'value') else str(ip.status),
"assigned_to": ip.assigned_object.name if ip.assigned_object else None
}
for ip in ips
]
print(json.dumps(ips_data, indent=2))
else:
table = Table(title="IP Addresses")
table.add_column("ID", style="cyan")
table.add_column("Address", style="green")
table.add_column("DNS Name", style="yellow")
table.add_column("Status")
table.add_column("Assigned To")
for ip in ips:
table.add_row(
str(ip.id),
str(ip.address),
ip.dns_name or "",
ip.status.value if hasattr(
ip.status, 'value') else str(ip.status),
ip.assigned_object.name if ip.assigned_object else "N/A"
)
console.print(table)
console.print(f"\n[green]Total IPs: {len(list(ips))}[/green]")
except Exception as e:
console.print(f"[red]Error querying IPs: {e}[/red]")
sys.exit(1)
# ============================================================================
# Prefixes Commands
# ============================================================================
prefixes_app = typer.Typer(help="Manage NetBox prefixes")
@prefixes_app.command("available")
def prefixes_available(
prefix: str = typer.Option(..., help="Prefix (e.g., 192.168.3.0/24)"),
limit: int = typer.Option(10, help="Max results to show"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""Get available IPs in a prefix."""
nb = get_netbox_client()
try:
prefix_obj = nb.ipam.prefixes.get(prefix=prefix)
if not prefix_obj:
console.print(f"[red]Prefix '{prefix}' not found[/red]")
sys.exit(1)
available = prefix_obj.available_ips.list()[:limit]
if output == "json":
import json
print(json.dumps([str(ip.address) for ip in available], indent=2))
else:
console.print(f"\n[green]Prefix:[/green] {prefix}")
console.print(
f"[green]Available IPs (showing first {limit}):[/green]\n")
for ip in available:
console.print(f" • {ip.address}")
console.print(
f"\n[yellow]Total available: {len(available)}+[/yellow]")
except Exception as e:
console.print(f"[red]Error getting available IPs: {e}[/red]")
sys.exit(1)
# ============================================================================
# Main App
# ============================================================================
app.add_typer(sites_app, name="sites")
app.add_typer(devices_app, name="devices")
app.add_typer(vms_app, name="vms")
app.add_typer(ips_app, name="ips")
app.add_typer(prefixes_app, name="prefixes")
@app.command("version")
def version():
"""Show version information."""
console.print("[green]NetBox API Client v1.0.0[/green]")
console.print("Part of Virgo-Core infrastructure automation")
if __name__ == "__main__":
app()

451
skills/netbox-powerdns-integration/tools/netbox_ipam_query.py Executable file
View File

@@ -0,0 +1,451 @@
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "pynetbox>=7.0.0",
# "infisicalsdk>=1.0.3",
# "rich>=13.0.0",
# "typer>=0.9.0",
# ]
# ///
"""
NetBox IPAM Query Tool
Advanced IPAM queries for the Matrix cluster infrastructure.
Query available IPs, prefix utilization, IP assignments, and VLAN information.
Usage:
# Get available IPs in prefix
./netbox_ipam_query.py available --prefix 192.168.3.0/24
# Check prefix utilization
./netbox_ipam_query.py utilization --site matrix
# Find IP assignments
./netbox_ipam_query.py assignments --prefix 192.168.3.0/24
# List VLANs
./netbox_ipam_query.py vlans --site matrix
Example:
./netbox_ipam_query.py available --prefix 192.168.3.0/24 --limit 5
./netbox_ipam_query.py utilization --site matrix --output json
"""
import os
import sys
from typing import Optional
from dataclasses import dataclass
import typer
from rich.console import Console
from rich.table import Table
from rich.panel import Panel
from rich.progress import Progress, SpinnerColumn, TextColumn
import pynetbox
from infisical_sdk import InfisicalSDKClient
app = typer.Typer()
console = Console()
# ============================================================================
# Configuration and Authentication
# ============================================================================
@dataclass
class NetBoxConfig:
"""NetBox connection configuration."""
url: str
token: str
ssl_verify: bool = True
def get_netbox_client() -> pynetbox.api:
"""
Get authenticated NetBox API client.
Uses Infisical SDK with Universal Auth to securely retrieve API token.
Requires INFISICAL_CLIENT_ID and INFISICAL_CLIENT_SECRET environment variables.
Returns:
pynetbox.api: Authenticated NetBox client
Raises:
ValueError: If token cannot be retrieved or is empty
typer.Exit: On connection or authentication errors (CLI exits)
"""
try:
# Initialize Infisical SDK client
client = InfisicalSDKClient(host="https://app.infisical.com")
# Authenticate using Universal Auth (machine identity)
client_id = os.getenv("INFISICAL_CLIENT_ID")
client_secret = os.getenv("INFISICAL_CLIENT_SECRET")
if not client_id or not client_secret:
console.print("[red]INFISICAL_CLIENT_ID and INFISICAL_CLIENT_SECRET environment variables required[/red]")
raise typer.Exit(1)
client.auth.universal_auth.login(
client_id=client_id,
client_secret=client_secret
)
# Get NetBox API token from Infisical
secret = client.secrets.get_secret_by_name(
secret_name="NETBOX_API_TOKEN",
project_id="7b832220-24c0-45bc-a5f1-ce9794a31259",
environment_slug="prod",
secret_path="/matrix"
)
token = secret.secretValue
if not token:
console.print("[red]NETBOX_API_TOKEN is empty in Infisical[/red]")
raise ValueError("NETBOX_API_TOKEN is empty")
config = NetBoxConfig(
url="https://netbox.spaceships.work",
token=token,
ssl_verify=True
)
return pynetbox.api(config.url, token=config.token)
except ValueError:
# ValueError already logged above, re-raise to propagate
raise
except Exception as e:
console.print(f"[red]Failed to connect to NetBox: {e}[/red]")
raise typer.Exit(1)
@app.command("available")
def available_ips(
prefix: str = typer.Option(..., help="Prefix (e.g., 192.168.3.0/24)"),
limit: int = typer.Option(10, help="Number of IPs to show"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""Get available IPs in a prefix."""
nb = get_netbox_client()
try:
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
console=console
) as progress:
task = progress.add_task(
f"Querying prefix {prefix}...", total=None)
prefix_obj = nb.ipam.prefixes.get(prefix=prefix)
if not prefix_obj:
console.print(f"[red]Prefix '{prefix}' not found[/red]")
sys.exit(1)
progress.update(task, description="Getting available IPs...")
available = prefix_obj.available_ips.list()[:limit]
if output == "json":
import json
data = {
"prefix": str(prefix),
"total_shown": len(available),
"available_ips": [str(ip.address) for ip in available]
}
print(json.dumps(data, indent=2))
else:
console.print(f"\n[green]Prefix:[/green] {prefix}")
console.print(
f"[green]Site:[/green] {prefix_obj.site.name if prefix_obj.site else 'N/A'}")
console.print(
f"[green]Description:[/green] {prefix_obj.description or 'N/A'}\n")
table = Table(title="Available IP Addresses")
table.add_column("IP Address", style="cyan")
table.add_column("Ready for Assignment")
for ip in available:
table.add_row(str(ip.address), "✓")
console.print(table)
console.print(
f"\n[yellow]Showing first {limit} available IPs[/yellow]")
except Exception as e:
console.print(f"[red]Error: {e}[/red]")
sys.exit(1)
@app.command("utilization")
def prefix_utilization(
site: Optional[str] = typer.Option(None, help="Filter by site slug"),
role: Optional[str] = typer.Option(None, help="Filter by prefix role"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""Show prefix utilization statistics."""
nb = get_netbox_client()
try:
filters = {}
if site:
filters['site'] = site
if role:
filters['role'] = role
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
console=console
) as progress:
task = progress.add_task("Querying prefixes...", total=None)
prefixes = nb.ipam.prefixes.filter(
**filters) if filters else nb.ipam.prefixes.all()
if output == "json":
import json
data = [
{
"prefix": str(p.prefix),
"site": p.site.name if p.site else None,
"role": p.role.name if p.role else None,
"utilization": float(p.utilization) if hasattr(p, 'utilization') else 0.0,
"description": p.description or ""
}
for p in prefixes
]
print(json.dumps(data, indent=2))
else:
table = Table(title="Prefix Utilization")
table.add_column("Prefix", style="cyan")
table.add_column("Site", style="yellow")
table.add_column("Role")
table.add_column("Utilization", justify="right")
table.add_column("Description")
for p in prefixes:
utilization = p.utilization if hasattr(p, 'utilization') else 0
util_pct = f"{utilization}%"
# Color code based on utilization
if utilization >= 90:
util_color = "red"
elif utilization >= 75:
util_color = "yellow"
else:
util_color = "green"
table.add_row(
str(p.prefix),
p.site.name if p.site else "N/A",
p.role.name if p.role else "N/A",
f"[{util_color}]{util_pct}[/{util_color}]",
p.description or ""
)
console.print(table)
except Exception as e:
console.print(f"[red]Error: {e}[/red]")
sys.exit(1)
@app.command("assignments")
def ip_assignments(
prefix: str = typer.Option(..., help="Prefix (e.g., 192.168.3.0/24)"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""Show IP assignments in a prefix."""
nb = get_netbox_client()
try:
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
console=console
) as progress:
task = progress.add_task(
f"Querying prefix {prefix}...", total=None)
prefix_obj = nb.ipam.prefixes.get(prefix=prefix)
if not prefix_obj:
console.print(f"[red]Prefix '{prefix}' not found[/red]")
sys.exit(1)
progress.update(task, description="Getting IP assignments...")
# Get IPs by parent prefix (materialize once to avoid re-fetching)
ips = list(nb.ipam.ip_addresses.filter(parent=prefix))
if output == "json":
import json
data = [
{
"address": str(ip.address),
"dns_name": ip.dns_name or "",
"status": ip.status.value if hasattr(ip.status, 'value') else str(ip.status),
"assigned_to": {
"type": ip.assigned_object_type if ip.assigned_object else None,
"name": ip.assigned_object.name if ip.assigned_object else None
},
"description": ip.description or ""
}
for ip in ips
]
print(json.dumps(data, indent=2))
else:
table = Table(title=f"IP Assignments in {prefix}")
table.add_column("IP Address", style="cyan")
table.add_column("DNS Name", style="green")
table.add_column("Status")
table.add_column("Assigned To")
table.add_column("Description")
for ip in ips:
assigned_to = "N/A"
if ip.assigned_object:
obj_type = ip.assigned_object_type.split(
'.')[-1] if ip.assigned_object_type else "unknown"
assigned_to = f"{ip.assigned_object.name} ({obj_type})"
table.add_row(
str(ip.address),
ip.dns_name or "",
ip.status.value if hasattr(
ip.status, 'value') else str(ip.status),
assigned_to,
ip.description or ""
)
console.print(table)
console.print(f"\n[green]Total IPs: {len(ips)}[/green]")
except Exception as e:
console.print(f"[red]Error: {e}[/red]")
sys.exit(1)
@app.command("vlans")
def list_vlans(
site: Optional[str] = typer.Option(None, help="Filter by site slug"),
output: str = typer.Option("table", help="Output format: table or json")
):
"""List VLANs."""
nb = get_netbox_client()
try:
filters = {}
if site:
filters['site'] = site
# Materialize once to avoid re-fetching on len() call
vlans = list(nb.ipam.vlans.filter(**filters)
if filters else nb.ipam.vlans.all())
if output == "json":
import json
data = [
{
"id": vlan.id,
"vid": vlan.vid,
"name": vlan.name,
"site": vlan.site.name if vlan.site else None,
"status": vlan.status.value if hasattr(vlan.status, 'value') else str(vlan.status),
"description": vlan.description or ""
}
for vlan in vlans
]
print(json.dumps(data, indent=2))
else:
table = Table(title="VLANs")
table.add_column("ID", style="cyan")
table.add_column("VID", justify="right", style="yellow")
table.add_column("Name", style="green")
table.add_column("Site")
table.add_column("Status")
table.add_column("Description")
for vlan in vlans:
table.add_row(
str(vlan.id),
str(vlan.vid),
vlan.name,
vlan.site.name if vlan.site else "N/A",
vlan.status.value if hasattr(
vlan.status, 'value') else str(vlan.status),
vlan.description or ""
)
console.print(table)
console.print(f"\n[green]Total VLANs: {len(vlans)}[/green]")
except Exception as e:
console.print(f"[red]Error: {e}[/red]")
sys.exit(1)
@app.command("summary")
def ipam_summary(
site: str = typer.Option(..., help="Site slug")
):
"""Show IPAM summary for a site."""
nb = get_netbox_client()
try:
site_obj = nb.dcim.sites.get(slug=site)
if not site_obj:
console.print(f"[red]Site '{site}' not found[/red]")
sys.exit(1)
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
console=console
) as progress:
task = progress.add_task("Gathering IPAM data...", total=None)
# Get prefixes
prefixes = list(nb.ipam.prefixes.filter(site=site))
# Get IPs
ips = list(nb.ipam.ip_addresses.filter(site=site)
if hasattr(nb.ipam.ip_addresses, 'filter') else [])
# Get VLANs
vlans = list(nb.ipam.vlans.filter(site=site))
# Display summary
console.print(Panel(
f"[green]Site:[/green] {site_obj.name}\n"
f"[green]Prefixes:[/green] {len(prefixes)}\n"
f"[green]IP Addresses:[/green] {len(ips)}\n"
f"[green]VLANs:[/green] {len(vlans)}",
title="IPAM Summary",
border_style="cyan"
))
# Prefix details
if prefixes:
console.print("\n[yellow]Prefixes:[/yellow]")
for p in prefixes:
utilization = p.utilization if hasattr(p, 'utilization') else 0
console.print(
f" • {p.prefix} - {p.description or 'No description'} ({utilization}% used)")
# VLAN details
if vlans:
console.print("\n[yellow]VLANs:[/yellow]")
for v in vlans:
console.print(
f" • VLAN {v.vid} ({v.name}) - {v.description or 'No description'}")
except Exception as e:
console.print(f"[red]Error: {e}[/red]")
sys.exit(1)
if __name__ == "__main__":
app()

369
skills/netbox-powerdns-integration/tools/netbox_vm_create.py Executable file
View File

@@ -0,0 +1,369 @@
#!/usr/bin/env -S uv run --script --quiet
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "pynetbox>=7.0.0",
# "infisicalsdk>=1.0.3",
# "rich>=13.0.0",
# "typer>=0.9.0",
# ]
# ///
"""
NetBox VM Creator
Create a complete VM in NetBox with automatic IP assignment and DNS configuration.
Follows Matrix cluster naming conventions and integrates with PowerDNS sync.
Usage:
# Create VM with auto-assigned IP
./netbox_vm_create.py --name docker-02 --cluster matrix --vcpus 4 --memory 8192
# Create VM with specific IP
./netbox_vm_create.py --name k8s-01-master --cluster matrix --ip 192.168.3.50
# Create VM with custom DNS name
./netbox_vm_create.py --name app-01 --cluster matrix --dns-name app-01-web.spaceships.work
Example:
./netbox_vm_create.py \\
--name docker-02 \\
--cluster matrix \\
--vcpus 4 \\
--memory 8192 \\
--disk 100 \\
--description "Docker host for GitLab"
"""
import ipaddress
import os
import sys
import re
from typing import Optional
from dataclasses import dataclass
import typer
from rich.console import Console
from rich.panel import Panel
import pynetbox
from infisical_sdk import InfisicalSDKClient
app = typer.Typer()
console = Console()
# ============================================================================
# Configuration and Authentication
# ============================================================================
@dataclass
class NetBoxConfig:
"""NetBox connection configuration."""
url: str
token: str
ssl_verify: bool = True
def get_netbox_client() -> pynetbox.api:
"""
Get authenticated NetBox API client.
Uses Infisical SDK with Universal Auth to securely retrieve API token.
Requires INFISICAL_CLIENT_ID and INFISICAL_CLIENT_SECRET environment variables.
Returns:
pynetbox.api: Authenticated NetBox client
Raises:
ValueError: If token cannot be retrieved or is empty
typer.Exit: On connection or authentication errors (CLI exits)
"""
try:
# Initialize Infisical SDK client
client = InfisicalSDKClient(host="https://app.infisical.com")
# Authenticate using Universal Auth (machine identity)
client_id = os.getenv("INFISICAL_CLIENT_ID")
client_secret = os.getenv("INFISICAL_CLIENT_SECRET")
if not client_id or not client_secret:
console.print(
"[red]INFISICAL_CLIENT_ID and INFISICAL_CLIENT_SECRET environment variables required[/red]")
raise typer.Exit(1)
client.auth.universal_auth.login(
client_id=client_id,
client_secret=client_secret
)
# Get NetBox API token from Infisical
secret = client.secrets.get_secret_by_name(
secret_name="NETBOX_API_TOKEN",
project_id="7b832220-24c0-45bc-a5f1-ce9794a31259",
environment_slug="prod",
secret_path="/matrix"
)
token = secret.secretValue
if not token:
console.print("[red]NETBOX_API_TOKEN is empty in Infisical[/red]")
raise ValueError("NETBOX_API_TOKEN is empty")
config = NetBoxConfig(
url="https://netbox.spaceships.work",
token=token,
ssl_verify=True
)
return pynetbox.api(config.url, token=config.token)
except ValueError:
# ValueError already logged above, re-raise to propagate
raise
except Exception as e:
console.print(f"[red]Failed to connect to NetBox: {e}[/red]")
raise typer.Exit(1)
def validate_dns_name(name: str) -> bool:
"""
Validate DNS naming convention.
Pattern: <service>-<number>[-<purpose>].<domain>
Examples: docker-01.spaceships.work, docker-01-nexus.spaceships.work
"""
pattern = r'^[a-z0-9-]+-\d{2}(-[a-z0-9-]+)?\.[a-z0-9.-]+$'
return bool(re.match(pattern, name.lower()))
def validate_vm_name(name: str) -> bool:
"""
Validate VM name (without domain).
Pattern: <service>-<number> or <service>-<number>-<purpose>
- service: one or more lowercase letters, digits, or hyphens
- number: exactly two digits (00-99)
- purpose (optional): one or more lowercase letters/numbers/hyphens
Example: docker-01, k8s-01-master, or docker-proxy-01
"""
pattern = r'^[a-z0-9-]+\-\d{2}(-[a-z0-9-]+)?$'
return bool(re.match(pattern, name.lower()))
@app.command()
def main(
name: str = typer.Option(...,
help="VM name (e.g., docker-02, k8s-01-master)"),
cluster: str = typer.Option("matrix", help="Cluster name"),
vcpus: int = typer.Option(2, help="Number of vCPUs"),
memory: int = typer.Option(2048, help="Memory in MB"),
disk: int = typer.Option(20, help="Disk size in GB"),
ip: Optional[str] = typer.Option(
None, help="Specific IP address (e.g., 192.168.3.50/24)"),
prefix: str = typer.Option(
"192.168.3.0/24", help="Prefix for auto IP assignment"),
dns_name: Optional[str] = typer.Option(
None, help="Custom DNS name (FQDN)"),
description: Optional[str] = typer.Option(None, help="VM description"),
tags: str = typer.Option("terraform,production-dns",
help="Comma-separated tags"),
dry_run: bool = typer.Option(
False, help="Show what would be created without creating")
):
"""
Create a VM in NetBox with automatic IP assignment and DNS.
This tool follows Matrix cluster conventions and integrates with PowerDNS sync.
"""
# Validate VM name
if not validate_vm_name(name):
console.print(f"[red]Invalid VM name: {name}[/red]")
console.print(
"[yellow]VM name must contain only lowercase letters, numbers, and hyphens[/yellow]")
raise typer.Exit(1)
# Generate DNS name if not provided
if not dns_name:
dns_name = f"{name}.spaceships.work"
# Validate DNS name
if not validate_dns_name(dns_name):
console.print(f"[red]Invalid DNS name: {dns_name}[/red]")
console.print(
"[yellow]DNS name must follow pattern: service-NN[-purpose].domain[/yellow]")
console.print(
"[yellow]Examples: docker-01.spaceships.work or docker-01-nexus.spaceships.work[/yellow]")
raise typer.Exit(1)
# Parse tags
tag_list = [tag.strip() for tag in tags.split(',') if tag.strip()]
# Show configuration
console.print(Panel(
f"[green]VM Name:[/green] {name}\n"
f"[green]Cluster:[/green] {cluster}\n"
f"[green]vCPUs:[/green] {vcpus}\n"
f"[green]Memory:[/green] {memory} MB\n"
f"[green]Disk:[/green] {disk} GB\n"
f"[green]IP:[/green] {ip or f'Auto (from {prefix})'}\n"
f"[green]DNS Name:[/green] {dns_name}\n"
f"[green]Description:[/green] {description or 'N/A'}\n"
f"[green]Tags:[/green] {', '.join(tag_list)}",
title="VM Configuration",
border_style="cyan"
))
if dry_run:
console.print("\n[yellow]Dry run - no changes made[/yellow]")
raise typer.Exit(0)
# Confirm
if not typer.confirm("\nCreate this VM in NetBox?"):
console.print("[yellow]Aborted[/yellow]")
raise typer.Exit(0)
nb = get_netbox_client()
try:
# 1. Get cluster
console.print(f"\n[cyan]1. Looking up cluster '{cluster}'...[/cyan]")
cluster_obj = nb.virtualization.clusters.get(name=cluster)
if not cluster_obj:
console.print(f"[red]Cluster '{cluster}' not found[/red]")
raise typer.Exit(1)
console.print(f"[green]✓ Found cluster: {cluster_obj.name}[/green]")
# 2. Check if VM already exists
console.print(
f"\n[cyan]2. Checking if VM '{name}' already exists...[/cyan]")
existing_vm = nb.virtualization.virtual_machines.get(name=name)
if existing_vm:
console.print(
f"[red]VM '{name}' already exists (ID: {existing_vm.id})[/red]")
raise typer.Exit(1)
console.print("[green]✓ VM name available[/green]")
# 3. Create VM
console.print(f"\n[cyan]3. Creating VM '{name}'...[/cyan]")
vm = nb.virtualization.virtual_machines.create(
name=name,
cluster=cluster_obj.id,
status='active',
vcpus=vcpus,
memory=memory,
disk=disk,
description=description or "VM created via netbox_vm_create.py",
tags=[{"name": tag} for tag in tag_list]
)
console.print(f"[green]✓ Created VM (ID: {vm.id})[/green]")
# 4. Create VM interface
console.print("\n[cyan]4. Creating network interface 'eth0'...[/cyan]")
vm_iface = nb.virtualization.interfaces.create(
virtual_machine=vm.id,
name='eth0',
type='virtual',
enabled=True,
mtu=1500
)
console.print(
f"[green]✓ Created interface (ID: {vm_iface.id})[/green]")
# 5. Assign IP
# Validate IP format if provided
if ip:
try:
ipaddress.ip_interface(ip)
except ValueError:
console.print(f"[red]Invalid IP address format: {ip}[/red]")
console.print(
"[yellow]Expected format: 192.168.3.50/24[/yellow]")
# Rollback
vm_iface.delete()
vm.delete()
raise typer.Exit(1)
try:
if ip:
# Use specific IP
console.print(f"\n[cyan]5. Creating IP address {ip}...[/cyan]")
vm_ip = nb.ipam.ip_addresses.create(
address=ip,
dns_name=dns_name,
status='active',
assigned_object_type='virtualization.vminterface',
assigned_object_id=vm_iface.id,
tags=[{"name": tag} for tag in tag_list]
)
else:
# Auto-assign from prefix
console.print(
f"\n[cyan]5. Getting next available IP from {prefix}...[/cyan]")
prefix_obj = nb.ipam.prefixes.get(prefix=prefix)
if not prefix_obj:
console.print(f"[red]Prefix '{prefix}' not found[/red]")
# Rollback
vm_iface.delete()
vm.delete()
raise typer.Exit(1)
vm_ip = prefix_obj.available_ips.create(
dns_name=dns_name,
status='active',
assigned_object_type='virtualization.vminterface',
assigned_object_id=vm_iface.id,
tags=[{"name": tag} for tag in tag_list]
)
console.print(f"[green]✓ Assigned IP: {vm_ip.address}[/green]")
except Exception as e:
console.print(f"[red]Failed to assign IP: {e}[/red]")
console.print(
"[yellow]Rolling back: deleting interface and VM...[/yellow]")
try:
vm_iface.delete()
vm.delete()
except Exception as rollback_error:
console.print(f"[red]Rollback failed: {rollback_error}[/red]")
raise typer.Exit(1)
# 6. Set as primary IP
console.print(
f"\n[cyan]6. Setting {vm_ip.address} as primary IP...[/cyan]")
vm.primary_ip4 = vm_ip.id
vm.save()
console.print("[green]✓ Set primary IP[/green]")
# Success summary
console.print("\n" + "="*60)
console.print(Panel(
f"[green]VM Name:[/green] {vm.name}\n"
f"[green]ID:[/green] {vm.id}\n"
f"[green]Cluster:[/green] {cluster_obj.name}\n"
f"[green]IP Address:[/green] {vm_ip.address}\n"
f"[green]DNS Name:[/green] {vm_ip.dns_name}\n"
f"[green]vCPUs:[/green] {vm.vcpus}\n"
f"[green]Memory:[/green] {vm.memory} MB\n"
f"[green]Disk:[/green] {vm.disk} GB\n"
f"[green]Tags:[/green] {', '.join(tag_list)}",
title="✓ VM Created Successfully",
border_style="green"
))
console.print(
"\n[yellow]Note:[/yellow] DNS record will be automatically created by NetBox PowerDNS sync plugin")
console.print(
f"[yellow]DNS:[/yellow] {dns_name} → {vm_ip.address.split('/')[0]}")
except pynetbox.RequestError as e:
console.print(f"\n[red]NetBox API Error: {e.error}[/red]")
raise typer.Exit(1)
except Exception as e:
console.print(f"\n[red]Unexpected error: {e}[/red]")
raise typer.Exit(1)
if __name__ == "__main__":
app()

225
skills/netbox-powerdns-integration/tools/validate_dns_naming.py Executable file
View File

@@ -0,0 +1,225 @@
#!/usr/bin/env -S uv run --script --quiet
# /// script
# dependencies = []
# ///
"""
Validate DNS names against Virgo-Core naming convention.
Naming Convention: <service>-<NN>-<purpose>.<domain>
Example: docker-01-nexus.spaceships.work
Usage:
./validate_dns_naming.py docker-01-nexus.spaceships.work
./validate_dns_naming.py --file hostnames.txt
./validate_dns_naming.py --check-format docker-1-nexus.spaceships.work
"""
import argparse
import re
import sys
from typing import Tuple
class DNSNameValidator:
"""Validates DNS names against naming convention."""
# Pattern: <service>-<NN>-<purpose>.<domain>
PATTERN = r'^[a-z0-9-]+-\d{2}-[a-z0-9-]+\.[a-z0-9.-]+$'
# Common service types
KNOWN_SERVICES = {
'docker', 'k8s', 'proxmox', 'storage', 'db', 'network',
'app', 'vip', 'service', 'test', 'dev', 'staging', 'prod'
}
def __init__(self):
self.pattern = re.compile(self.PATTERN)
def validate(self, name: str) -> Tuple[bool, str, dict]:
"""
Validate DNS name.
Returns:
(is_valid, message, details_dict)
"""
# Basic pattern match
if not self.pattern.match(name):
return False, "Name doesn't match pattern: <service>-<NN>-<purpose>.<domain>", {}
# Split into components
parts = name.split('.')
if len(parts) < 2:
return False, "Must include domain", {}
hostname = parts[0]
domain = '.'.join(parts[1:])
# Split hostname
components = hostname.split('-')
if len(components) < 3:
return False, "Hostname must have at least 3 components: <service>-<NN>-<purpose>", {}
service = components[0]
number = components[1]
purpose = '-'.join(components[2:]) # Purpose can have hyphens
# Validate number component (must be 2 digits)
if not number.isdigit() or len(number) != 2:
return False, f"Number component '{number}' must be exactly 2 digits (01-99)", {}
# Additional checks
warnings = []
# Check for known service type
if service not in self.KNOWN_SERVICES:
warnings.append(f"Service '{service}' not in known types (informational)")
# Check for uppercase
if name != name.lower():
return False, "Name must be lowercase only", {}
# Check for invalid characters
if not re.match(r'^[a-z0-9.-]+$', name):
return False, "Name contains invalid characters (only a-z, 0-9, -, . allowed)", {}
# Build details
details = {
'service': service,
'number': number,
'purpose': purpose,
'domain': domain,
'warnings': warnings
}
message = "Valid"
if warnings:
message = f"Valid (with warnings: {', '.join(warnings)})"
return True, message, details
def validate_batch(self, names: list) -> dict:
"""
Validate multiple names.
Returns:
{
'valid': [(name, details), ...],
'invalid': [(name, reason), ...]
}
"""
results = {'valid': [], 'invalid': []}
for name in names:
name = name.strip()
if not name or name.startswith('#'):
continue
is_valid, message, details = self.validate(name)
if is_valid:
results['valid'].append((name, details))
else:
results['invalid'].append((name, message))
return results
def print_validation_result(name: str, is_valid: bool, message: str, details: dict, verbose: bool = False):
"""Print formatted validation result."""
status = "✓" if is_valid else "✗"
print(f"{status} {name}: {message}")
if verbose and is_valid and details:
print(f" Service: {details.get('service')}")
print(f" Number: {details.get('number')}")
print(f" Purpose: {details.get('purpose')}")
print(f" Domain: {details.get('domain')}")
warnings = details.get('warnings', [])
if warnings:
print(f" Warnings:")
for warning in warnings:
print(f" - {warning}")
def main():
parser = argparse.ArgumentParser(
description="Validate DNS names against Virgo-Core naming convention",
epilog="Pattern: <service>-<NN>-<purpose>.<domain>\nExample: docker-01-nexus.spaceships.work"
)
parser.add_argument(
"name",
nargs="?",
help="DNS name to validate"
)
parser.add_argument(
"--file",
help="File containing DNS names (one per line)"
)
parser.add_argument(
"--verbose", "-v",
action="store_true",
help="Show detailed component breakdown"
)
parser.add_argument(
"--check-format",
action="store_true",
help="Only check format, don't suggest corrections"
)
args = parser.parse_args()
validator = DNSNameValidator()
# Batch mode from file
if args.file:
try:
with open(args.file, 'r') as f:
names = f.readlines()
except IOError as e:
print(f"❌ Failed to read file: {e}", file=sys.stderr)
sys.exit(1)
results = validator.validate_batch(names)
print(f"\n📊 Validation Results:")
print(f" Valid: {len(results['valid'])}")
print(f" Invalid: {len(results['invalid'])}")
if results['invalid']:
print(f"\n✗ Invalid Names:")
for name, reason in results['invalid']:
print(f" {name}")
print(f" Reason: {reason}")
if results['valid']:
print(f"\n✓ Valid Names:")
for name, details in results['valid']:
print(f" {name}")
if args.verbose:
print(f" Service: {details['service']}, Number: {details['number']}, Purpose: {details['purpose']}")
sys.exit(0 if len(results['invalid']) == 0 else 1)
# Single name mode
if not args.name:
print("❌ Provide a DNS name or use --file", file=sys.stderr)
parser.print_help()
sys.exit(1)
is_valid, message, details = validator.validate(args.name)
print_validation_result(args.name, is_valid, message, details, args.verbose)
# Provide suggestions for common mistakes
if not is_valid and not args.check_format:
print(f"\n💡 Common Issues:")
print(f" - Use lowercase only: {args.name.lower()}")
print(f" - Use hyphens, not underscores: {args.name.replace('_', '-')}")
print(f" - Number must be 2 digits: docker-1-app → docker-01-app")
print(f" - Pattern: <service>-<NN>-<purpose>.<domain>")
print(f" - Example: docker-01-nexus.spaceships.work")
sys.exit(0 if is_valid else 1)
if __name__ == "__main__":
main()

570
skills/netbox-powerdns-integration/workflows/ansible-dynamic-inventory.md Normal file
View File

@@ -0,0 +1,570 @@
# Ansible Dynamic Inventory from NetBox
## Overview
Use NetBox as a dynamic inventory source for Ansible, eliminating the need for static inventory
files and ensuring your automation always has up-to-date infrastructure data.
## Architecture
```text
┌──────────┐
│ NetBox │ (Source of Truth)
│ IPAM │
└────┬─────┘
│
│ API Query
│
▼
┌────────────────┐
│ nb_inventory │ (Ansible Plugin)
│ Plugin │
└────┬───────────┘
│
│ Generates Dynamic Inventory
│
▼
┌────────────────┐
│ Ansible │ (Uses inventory for playbooks)
│ Playbooks │
└────────────────┘
```
## Prerequisites
### Install NetBox Ansible Collection
```bash
cd ansible
uv run ansible-galaxy collection install netbox.netbox
```
**Or add to requirements.yml:**
```yaml
---
collections:
- name: netbox.netbox
version: ">=3.0.0"
```
```bash
uv run ansible-galaxy collection install -r requirements.yml
```
### NetBox API Token
Create read-only API token in NetBox:
**NetBox UI:** Admin → API Tokens → Add
- User: ansible (create service user)
- Key: Generated automatically
- Write enabled: No (read-only)
**Save token securely:**
```bash
# Option 1: Environment variable
export NETBOX_API_TOKEN="your-token-here"
# Option 2: Ansible Vault
ansible-vault create group_vars/all/vault.yml
# Add: netbox_token: "your-token-here"
```
## Basic Configuration
### Create Inventory File
**File:** `ansible/inventory/netbox.yml`
```yaml
---
plugin: netbox.netbox.nb_inventory
# NetBox API connection
api_endpoint: https://netbox.spaceships.work
token: !vault |
$ANSIBLE_VAULT;1.1;AES256
...
# Validate SSL (set to false for self-signed certs)
validate_certs: true
# Group hosts by these NetBox attributes
group_by:
- device_roles
- tags
- sites
- platforms
# Set ansible_host variable from primary_ip4
compose:
ansible_host: primary_ip4
# Only include active devices/VMs
query_filters:
- status: active
```
### Test Inventory
```bash
# List all hosts
ansible-inventory -i ansible/inventory/netbox.yml --list
# View in YAML format
ansible-inventory -i ansible/inventory/netbox.yml --list --yaml
# View specific host
ansible-inventory -i ansible/inventory/netbox.yml --host docker-01-nexus
# Graph inventory
ansible-inventory -i ansible/inventory/netbox.yml --graph
```
## Advanced Configuration
### Filter by Tags
**Only include hosts with specific tag:**
```yaml
---
plugin: netbox.netbox.nb_inventory
api_endpoint: https://netbox.spaceships.work
token: !vault |
$ANSIBLE_VAULT;...
# Only hosts tagged with "ansible-managed"
query_filters:
- tag: ansible-managed
- status: active
group_by:
- tags
```
### Filter by Device Role
**Only include specific device roles:**
```yaml
query_filters:
- role: docker-host
- role: k8s-node
- status: active
```
### Custom Groups
**Create custom groups based on NetBox data:**
```yaml
---
plugin: netbox.netbox.nb_inventory
api_endpoint: https://netbox.spaceships.work
token: !vault |
$ANSIBLE_VAULT;...
group_by:
- device_roles
- tags
- sites
# Custom group mappings
keyed_groups:
- key: tags
prefix: tag
- key: device_role.name
prefix: role
- key: platform.name
prefix: platform
compose:
ansible_host: primary_ip4
ansible_user: ansible
ansible_become: true
```
### Include Custom Fields
**Use NetBox custom fields in inventory:**
```yaml
---
plugin: netbox.netbox.nb_inventory
api_endpoint: https://netbox.spaceships.work
token: !vault |
$ANSIBLE_VAULT;...
compose:
ansible_host: primary_ip4
# Use custom fields from NetBox
backup_schedule: custom_fields.backup_schedule
monitoring_enabled: custom_fields.monitoring_enabled
application_owner: custom_fields.owner
group_by:
- tags
- custom_fields.environment
```
## Usage Examples
### Example 1: Configure All Docker Hosts
**Inventory:** `ansible/inventory/netbox.yml`
```yaml
---
plugin: netbox.netbox.nb_inventory
api_endpoint: https://netbox.spaceships.work
token: !vault |
$ANSIBLE_VAULT;...
query_filters:
- tag: docker-host
- status: active
group_by:
- tags
compose:
ansible_host: primary_ip4
ansible_user: ansible
ansible_become: true
```
**Playbook:** `ansible/playbooks/configure-docker-hosts.yml`
```yaml
---
- name: Configure Docker hosts from NetBox inventory
hosts: tag_docker_host
become: true
tasks:
- name: Ensure Docker is running
ansible.builtin.systemd:
name: docker
state: started
enabled: true
- name: Update Docker daemon config
ansible.builtin.copy:
dest: /etc/docker/daemon.json
content: |
{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
}
}
notify: Restart Docker
handlers:
- name: Restart Docker
ansible.builtin.systemd:
name: docker
state: restarted
```
**Run playbook:**
```bash
cd ansible
uv run ansible-playbook -i inventory/netbox.yml playbooks/configure-docker-hosts.yml
```
### Example 2: Site-Specific Deployments
**Inventory with site grouping:**
```yaml
---
plugin: netbox.netbox.nb_inventory
api_endpoint: https://netbox.spaceships.work
token: !vault |
$ANSIBLE_VAULT;...
group_by:
- sites
- tags
compose:
ansible_host: primary_ip4
query_filters:
- status: active
```
**Playbook targeting specific site:**
```yaml
---
- name: Update hosts at primary site
hosts: site_homelab # Automatically grouped by site name
become: true
tasks:
- name: Update all packages
ansible.builtin.apt:
upgrade: dist
update_cache: true
when: ansible_os_family == "Debian"
```
### Example 3: Platform-Specific Configuration
**Inventory:**
```yaml
---
plugin: netbox.netbox.nb_inventory
api_endpoint: https://netbox.spaceships.work
token: !vault |
$ANSIBLE_VAULT;...
group_by:
- platforms
compose:
ansible_host: primary_ip4
keyed_groups:
- key: platform.name
prefix: platform
```
**Playbook with platform-specific tasks:**
```yaml
---
- name: Platform-specific configuration
hosts: all
become: true
tasks:
- name: Configure Ubuntu hosts
ansible.builtin.apt:
name: netbox-agent
state: present
when: "'ubuntu' in group_names"
- name: Configure Rocky hosts
ansible.builtin.dnf:
name: netbox-agent
state: present
when: "'rocky' in group_names"
```
## Integration with Secrets Management
### Use with Infisical
**Combine dynamic inventory with Infisical secrets:**
```yaml
---
- name: Deploy app with NetBox inventory and Infisical secrets
hosts: tag_app_server
become: true
vars:
infisical_project_id: "7b832220-24c0-45bc-a5f1-ce9794a31259"
infisical_env: "prod"
infisical_path: "/app-config"
tasks:
- name: Retrieve database password
ansible.builtin.include_tasks: "{{ playbook_dir }}/../tasks/infisical-secret-lookup.yml"
vars:
secret_name: 'DB_PASSWORD'
secret_var_name: 'db_password'
- name: Deploy application config
ansible.builtin.template:
src: app-config.j2
dest: /etc/app/config.yml
owner: app
group: app
mode: '0600'
vars:
db_host: "{{ hostvars[groups['tag_database'][0]]['ansible_host'] }}"
db_password: "{{ db_password }}"
```
## Caching for Performance
### Enable Inventory Caching
**File:** `ansible/ansible.cfg`
```ini
[defaults]
inventory_plugins = /usr/share/ansible/plugins/inventory
[inventory]
enable_plugins = netbox.netbox.nb_inventory
# Enable caching
cache = true
cache_plugin = jsonfile
cache_timeout = 3600 # 1 hour
cache_connection = /tmp/ansible-netbox-cache
```
**Benefits:**
- Faster playbook runs
- Reduced API calls to NetBox
- Works offline (for cache duration)
**Clear cache:**
```bash
rm -rf /tmp/ansible-netbox-cache
```
## Troubleshooting
### Authentication Errors
**Error:** `Failed to query NetBox API`
**Check:**
```bash
# Test API token
curl -H "Authorization: Token $NETBOX_API_TOKEN" \
https://netbox.spaceships.work/api/dcim/devices/ | jq
# Verify token permissions
# Token must have read access to: DCIM, IPAM, Virtualization
```
### SSL Certificate Errors
**Error:** `SSL: CERTIFICATE_VERIFY_FAILED`
**Solutions:**
```yaml
# Option 1: Add CA certificate
validate_certs: true
ssl_ca_cert: /path/to/ca-bundle.crt
# Option 2: Disable for self-signed (dev only!)
validate_certs: false
```
### No Hosts Found
**Error:** Inventory is empty
**Check:**
```bash
# List all devices in NetBox
curl -H "Authorization: Token $NETBOX_API_TOKEN" \
https://netbox.spaceships.work/api/dcim/devices/ | jq '.count'
# Check query filters
# Ensure devices match your filters (status, tags, etc.)
```
**Debug inventory plugin:**
```bash
ansible-inventory -i ansible/inventory/netbox.yml --list -vvv
```
### Primary IP Not Set
**Error:** `ansible_host` is undefined
**Cause:** Devices/VMs in NetBox don't have primary_ip4 set
**Solution:**
```yaml
# Fallback to custom field or use DNS name
compose:
ansible_host: primary_ip4 | default(custom_fields.management_ip) | default(name + '.spaceships.work')
```
## Best Practices
### 1. Use Service Account
Create dedicated NetBox user for Ansible:
```text
Username: ansible-automation
Permissions: Read-only (DCIM, IPAM, Virtualization)
Token: Never expires (or set appropriate expiration)
```
### 2. Tag for Inventory
Tag devices/VMs intended for Ansible management:
```text
Tag: ansible-managed
```
**Filter in inventory:**
```yaml
query_filters:
- tag: ansible-managed
```
### 3. Set Primary IPs
Always set primary_ip4 in NetBox for devices/VMs:
```text
Device → Edit → Primary IPv4
```
### 4. Use Custom Fields
Add custom fields to NetBox for Ansible-specific data:
```text
ansible_user (Text)
ansible_port (Integer)
ansible_python_interpreter (Text)
backup_enabled (Boolean)
```
### 5. Test Before Running
Always test inventory before running playbooks:
```bash
# Verify hosts
ansible-inventory -i inventory/netbox.yml --graph
# Test connectivity
ansible all -i inventory/netbox.yml -m ping
```
### 6. Document in NetBox
Use NetBox description fields to document:
- Ansible playbooks that manage this host
- Special configuration requirements
- Dependencies on other hosts
## Further Reading
- [NetBox Ansible Collection Documentation](https://docs.ansible.com/ansible/latest/collections/netbox/netbox/)
- [Dynamic Inventory Plugin Guide](https://docs.ansible.com/ansible/latest/plugins/inventory.html)
- [NetBox API Documentation](https://demo.netbox.dev/api/docs/)

592
skills/netbox-powerdns-integration/workflows/dns-automation.md Normal file
View File

@@ -0,0 +1,592 @@
# DNS Automation Workflows
## Overview
This guide covers end-to-end workflows for automating DNS record management using NetBox as the
source of truth and PowerDNS as the authoritative DNS server.
## Architecture
```text
┌─────────────┐
│ Terraform │───┐
│ Ansible │ │
│ Manual │ │
└─────────────┘ │
▼
┌──────────┐
│ NetBox │ (Source of Truth)
│ IPAM │
└────┬─────┘
│
│ netbox-powerdns-sync plugin
│
▼
┌──────────┐
│ PowerDNS │ (Authoritative DNS)
│ API │
└────┬─────┘
│
│ Zone files / API
│
▼
┌──────────┐
│ DNS │ (Resolvers query here)
│ Clients │
└──────────┘
```
## Workflow 1: Create VM with Automatic DNS
### Using Terraform
**End-to-end automation:**
```hcl
# 1. Create VM in Proxmox
resource "proxmox_vm_qemu" "docker_host" {
name = "docker-01-nexus"
target_node = "foxtrot"
vmid = 101
clone = "ubuntu-template"
full_clone = true
cores = 4
memory = 8192
network {
bridge = "vmbr0"
model = "virtio"
tag = 30
}
disk {
storage = "local-lvm"
type = "scsi"
size = "50G"
}
# Cloud-init IP configuration
ipconfig0 = "ip=192.168.1.100/24,gw=192.168.1.1"
sshkeys = file("~/.ssh/id_rsa.pub")
}
# 2. Register in NetBox (triggers DNS sync)
resource "netbox_ip_address" "docker_host" {
ip_address = "192.168.1.100/24"
dns_name = "docker-01-nexus.spaceships.work"
status = "active"
description = "Docker host for Nexus container registry"
tags = [
"terraform",
"production-dns", # Triggers auto DNS sync
"docker-host"
]
# Ensure VM is created first
depends_on = [proxmox_vm_qemu.docker_host]
}
# 3. DNS records automatically created by netbox-powerdns-sync plugin:
# A: docker-01-nexus.spaceships.work → 192.168.1.100
# PTR: 100.1.168.192.in-addr.arpa → docker-01-nexus.spaceships.work
# 4. Output for verification
output "vm_fqdn" {
value = netbox_ip_address.docker_host.dns_name
}
output "vm_ip" {
value = split("/", netbox_ip_address.docker_host.ip_address)[0]
}
```
**Apply workflow:**
```bash
cd terraform/netbox-vm/
tofu init
tofu plan
tofu apply
# Verify DNS
dig @192.168.3.1 docker-01-nexus.spaceships.work +short
# Returns: 192.168.1.100
# Verify PTR
dig @192.168.3.1 -x 192.168.1.100 +short
# Returns: docker-01-nexus.spaceships.work
```
### Using Ansible
**Playbook for VM with DNS:**
```yaml
---
- name: Provision VM with automatic DNS
hosts: localhost
gather_facts: false
vars:
vm_name: docker-01-nexus
vm_ip: 192.168.1.100
vm_fqdn: "{{ vm_name }}.spaceships.work"
proxmox_node: foxtrot
tasks:
# 1. Create VM in Proxmox
- name: Clone template to create VM
community.proxmox.proxmox_kvm:
api_host: "{{ proxmox_api_host }}"
api_user: "{{ proxmox_api_user }}"
api_token_id: "{{ proxmox_token_id }}"
api_token_secret: "{{ proxmox_token_secret }}"
node: "{{ proxmox_node }}"
vmid: 101
name: "{{ vm_name }}"
clone: ubuntu-template
full: true
storage: local-lvm
net:
net0: 'virtio,bridge=vmbr0,tag=30'
ipconfig:
ipconfig0: 'ip={{ vm_ip }}/24,gw=192.168.1.1'
cores: 4
memory: 8192
agent: 1
state: present
register: vm_result
- name: Start VM
community.proxmox.proxmox_kvm:
api_host: "{{ proxmox_api_host }}"
api_user: "{{ proxmox_api_user }}"
api_token_id: "{{ proxmox_token_id }}"
api_token_secret: "{{ proxmox_token_secret }}"
node: "{{ proxmox_node }}"
vmid: 101
state: started
# 2. Register in NetBox
- name: Create IP address in NetBox
netbox.netbox.netbox_ip_address:
netbox_url: "{{ netbox_url }}"
netbox_token: "{{ netbox_token }}"
data:
address: "{{ vm_ip }}/24"
dns_name: "{{ vm_fqdn }}"
status: active
description: "Docker host for Nexus container registry"
tags:
- name: production-dns
- name: ansible
- name: docker-host
# 3. Wait for DNS propagation
- name: Wait for DNS record
ansible.builtin.command: dig @192.168.3.1 {{ vm_fqdn }} +short
register: dns_check
until: dns_check.stdout == vm_ip
retries: 10
delay: 5
changed_when: false
# 4. Verify DNS resolution
- name: Verify DNS forward resolution
ansible.builtin.command: dig @192.168.3.1 {{ vm_fqdn }} +short
register: forward_dns
changed_when: false
- name: Verify DNS reverse resolution
ansible.builtin.command: dig @192.168.3.1 -x {{ vm_ip }} +short
register: reverse_dns
changed_when: false
- name: Report DNS status
ansible.builtin.debug:
msg:
- "VM created: {{ vm_name }}"
- "IP: {{ vm_ip }}"
- "FQDN: {{ vm_fqdn }}"
- "Forward DNS: {{ forward_dns.stdout }}"
- "Reverse DNS: {{ reverse_dns.stdout }}"
```
**Run playbook:**
```bash
cd ansible
uv run ansible-playbook playbooks/provision-vm-with-dns.yml
```
## Workflow 2: Bulk IP Address Management
### Reserve IP Range in NetBox
```python
#!/usr/bin/env python3
# /// script
# dependencies = ["pynetbox"]
# ///
import pynetbox
import os
netbox = pynetbox.api(
os.getenv("NETBOX_URL"),
token=os.getenv("NETBOX_TOKEN")
)
# Define IP range for Docker hosts
docker_ips = [
{"ip": "192.168.1.100/24", "dns": "docker-01-nexus.spaceships.work", "desc": "Nexus registry"},
{"ip": "192.168.1.101/24", "dns": "docker-02-gitlab.spaceships.work", "desc": "GitLab CI/CD"},
{"ip": "192.168.1.102/24", "dns": "docker-03-monitoring.spaceships.work", "desc": "Monitoring stack"},
]
for entry in docker_ips:
ip = netbox.ipam.ip_addresses.create(
address=entry["ip"],
dns_name=entry["dns"],
description=entry["desc"],
status="reserved",
tags=[{"name": "production-dns"}, {"name": "docker-host"}]
)
print(f"Created: {ip.dns_name} → {entry['ip']}")
```
**Run script:**
```bash
export NETBOX_URL="https://netbox.spaceships.work"
export NETBOX_TOKEN="your-api-token"
uv run reserve-docker-ips.py
```
### Update Status to Active
When VMs are deployed, update IPs from "reserved" to "active":
```python
#!/usr/bin/env python3
# /// script
# dependencies = ["pynetbox"]
# ///
import pynetbox
import os
import sys
netbox = pynetbox.api(
os.getenv("NETBOX_URL"),
token=os.getenv("NETBOX_TOKEN")
)
fqdn = sys.argv[1] if len(sys.argv) > 1 else "docker-01-nexus.spaceships.work"
# Find IP by DNS name
ips = netbox.ipam.ip_addresses.filter(dns_name=fqdn)
if not ips:
print(f"No IP found for {fqdn}")
sys.exit(1)
ip = ips[0]
ip.status = "active"
ip.save()
print(f"Updated {ip.dns_name}: {ip.address} → active")
```
## Workflow 3: DNS Record Auditing
### Verify NetBox and PowerDNS are in Sync
```python
#!/usr/bin/env python3
# /// script
# dependencies = ["pynetbox", "requests"]
# ///
import pynetbox
import requests
import os
import sys
netbox = pynetbox.api(
os.getenv("NETBOX_URL"),
token=os.getenv("NETBOX_TOKEN")
)
powerdns_url = os.getenv("POWERDNS_URL", "http://192.168.3.1:8081/api/v1")
powerdns_key = os.getenv("POWERDNS_API_KEY")
zone = sys.argv[1] if len(sys.argv) > 1 else "spaceships.work"
# Get NetBox IPs tagged for DNS
netbox_ips = netbox.ipam.ip_addresses.filter(tag="production-dns")
# Get PowerDNS records
headers = {"X-API-Key": powerdns_key}
pdns_resp = requests.get(f"{powerdns_url}/servers/localhost/zones/{zone}", headers=headers)
pdns_zone = pdns_resp.json()
# Extract A records from PowerDNS
pdns_records = {}
for rrset in pdns_zone.get("rrsets", []):
if rrset["type"] == "A":
name = rrset["name"].rstrip(".")
for record in rrset["records"]:
pdns_records[name] = record["content"]
# Compare
print("NetBox → PowerDNS Sync Status\n")
print(f"{'DNS Name':<45} {'NetBox IP':<15} {'PowerDNS IP':<15} {'Status'}")
print("-" * 90)
for nb_ip in netbox_ips:
if not nb_ip.dns_name:
continue
dns_name = nb_ip.dns_name.rstrip(".")
nb_addr = str(nb_ip.address).split("/")[0]
pdns_addr = pdns_records.get(dns_name, "MISSING")
if pdns_addr == nb_addr:
status = "✓ SYNCED"
elif pdns_addr == "MISSING":
status = "✗ NOT IN POWERDNS"
else:
status = f"✗ MISMATCH"
print(f"{dns_name:<45} {nb_addr:<15} {pdns_addr:<15} {status}")
```
**Run audit:**
```bash
export NETBOX_URL="https://netbox.spaceships.work"
export NETBOX_TOKEN="your-netbox-token"
export POWERDNS_URL="http://192.168.3.1:8081/api/v1"
export POWERDNS_API_KEY="your-powerdns-key"
uv run dns-audit.py spaceships.work
```
## Workflow 4: Dynamic Inventory for Ansible
### Use NetBox as Inventory Source
**Create dynamic inventory file:**
```yaml
# ansible/inventory/netbox.yml
plugin: netbox.netbox.nb_inventory
api_endpoint: https://netbox.spaceships.work
token: !vault |
$ANSIBLE_VAULT;1.1;AES256
...
# Group hosts by tags
group_by:
- tags
# Group hosts by device role
compose:
ansible_host: primary_ip4
# Filter to only include VMs with production-dns tag
query_filters:
- tag: production-dns
```
**Test inventory:**
```bash
ansible-inventory -i ansible/inventory/netbox.yml --list --yaml
```
**Use in playbook:**
```yaml
---
- name: Configure all Docker hosts
hosts: tag_docker_host # Automatically grouped by tag
become: true
tasks:
- name: Ensure Docker is running
ansible.builtin.systemd:
name: docker
state: started
enabled: true
- name: Report host info
ansible.builtin.debug:
msg: "Configuring {{ inventory_hostname }} ({{ ansible_host }})"
```
**Run with dynamic inventory:**
```bash
cd ansible
uv run ansible-playbook -i inventory/netbox.yml playbooks/configure-docker-hosts.yml
```
## Workflow 5: Cleanup and Decommission
### Remove VM and DNS Records
**Terraform destroy workflow:**
```bash
cd terraform/netbox-vm/
tofu destroy
# This will:
# 1. Remove NetBox IP address record
# 2. netbox-powerdns-sync plugin removes DNS records
# 3. Proxmox VM is deleted
```
**Ansible decommission playbook:**
```yaml
---
- name: Decommission VM and remove DNS
hosts: localhost
gather_facts: false
vars:
vm_fqdn: docker-01-nexus.spaceships.work
vm_ip: 192.168.1.100
tasks:
- name: Remove IP from NetBox
netbox.netbox.netbox_ip_address:
netbox_url: "{{ netbox_url }}"
netbox_token: "{{ netbox_token }}"
data:
address: "{{ vm_ip }}/24"
state: absent
# DNS records automatically removed by plugin
- name: Verify DNS record removed
ansible.builtin.command: dig @192.168.3.1 {{ vm_fqdn }} +short
register: dns_check
failed_when: dns_check.stdout != ""
changed_when: false
- name: Delete VM from Proxmox
community.proxmox.proxmox_kvm:
api_host: "{{ proxmox_api_host }}"
api_user: "{{ proxmox_api_user }}"
api_token_id: "{{ proxmox_token_id }}"
api_token_secret: "{{ proxmox_token_secret }}"
node: foxtrot
vmid: 101
state: absent
```
## Best Practices
### 1. Always Use Tags
Tag IP addresses for automatic DNS sync:
```hcl
tags = ["terraform", "production-dns", "service-type"]
```
### 2. Reserve Before Deploy
Reserve IPs in NetBox before deploying VMs:
```text
Status: reserved → active (after deployment)
```
### 3. Validate Names
Use naming convention validation before creating:
```bash
./tools/validate_dns_naming.py docker-01-nexus.spaceships.work
```
### 4. Monitor Sync Status
Regular audits to ensure NetBox and PowerDNS are in sync:
```bash
./tools/dns-audit.py spaceships.work
```
### 5. Use Descriptions
Document in NetBox description field:
```text
Description: Docker host for Nexus container registry
Owner: Platform Team
Related: docker-02-gitlab.spaceships.work
```
### 6. Test DNS Resolution
Always verify DNS after creation:
```bash
dig @192.168.3.1 <fqdn> +short
dig @192.168.3.1 -x <ip> +short
```
## Troubleshooting
### DNS Records Not Created
#### Check 1: Tag matching
```bash
# Verify IP has production-dns tag
curl -H "Authorization: Token $NETBOX_TOKEN" \
"$NETBOX_URL/api/ipam/ip-addresses/?address=192.168.1.100" | jq '.results[0].tags'
```
#### Check 2: Plugin configuration
```python
# In NetBox: Plugins → NetBox PowerDNS Sync → Zones
# Verify zone exists and tag rules match
```
#### Check 3: Manual sync
```bash
# In NetBox UI: Plugins → NetBox PowerDNS Sync → Zones → <zone> → Sync Now
```
### DNS Resolution Failures
**Check PowerDNS API:**
```bash
curl -H "X-API-Key: $POWERDNS_API_KEY" \
http://192.168.3.1:8081/api/v1/servers/localhost/zones/spaceships.work
```
**Check DNS server:**
```bash
dig @192.168.3.1 spaceships.work SOA
```
## Further Reading
- [NetBox PowerDNS Sync Plugin](../reference/sync-plugin-reference.md)
- [Terraform NetBox Provider](../reference/terraform-provider-guide.md)
- [DNS Naming Conventions](naming-conventions.md)

418
skills/netbox-powerdns-integration/workflows/naming-conventions.md Normal file
View File

@@ -0,0 +1,418 @@
# DNS Naming Conventions
## Overview
Consistent DNS naming is critical for automation and infrastructure documentation. This document defines the naming
conventions used in the Virgo-Core infrastructure.
## Standard Pattern
**Format:** `<service>-<number>-<purpose>.<domain>`
**Components:**
- `<service>` - Service type (docker, k8s, proxmox, storage, db, etc.)
- `<number>` - Instance number (01, 02, 03, etc.) - always 2 digits
- `<purpose>` - Specific purpose or application name
- `<domain>` - DNS domain (e.g., spaceships.work)
**Regex Pattern:**
```regex
^[a-z0-9-]+-\d{2}-[a-z0-9-]+\.[a-z0-9.-]+$
```
## Service Types
### Container Platforms
**Docker hosts:**
```text
docker-01-nexus.spaceships.work # Nexus container registry
docker-02-gitlab.spaceships.work # GitLab CI/CD
docker-03-monitoring.spaceships.work # Monitoring stack (Prometheus, Grafana)
```
**Kubernetes nodes:**
```text
k8s-01-master.spaceships.work # Control plane node 1
k8s-02-master.spaceships.work # Control plane node 2
k8s-03-master.spaceships.work # Control plane node 3
k8s-04-worker.spaceships.work # Worker node 1
k8s-05-worker.spaceships.work # Worker node 2
```
### Infrastructure
**Proxmox nodes:**
```text
proxmox-foxtrot-mgmt.spaceships.work # Foxtrot management interface
proxmox-foxtrot-ceph.spaceships.work # Foxtrot CEPH public interface
proxmox-golf-mgmt.spaceships.work # Golf management interface
proxmox-hotel-mgmt.spaceships.work # Hotel management interface
```
**Storage systems:**
```text
storage-01-nas.spaceships.work # NAS storage (TrueNAS/FreeNAS)
storage-02-backup.spaceships.work # Backup storage
storage-03-archive.spaceships.work # Long-term archive storage
```
### Databases
```text
db-01-postgres.spaceships.work # PostgreSQL primary
db-02-postgres.spaceships.work # PostgreSQL replica
db-03-mysql.spaceships.work # MySQL/MariaDB
db-04-redis.spaceships.work # Redis cache
```
### Network Services
```text
network-01-pfsense.spaceships.work # pfSense router
network-02-unifi.spaceships.work # UniFi controller
network-03-dns.spaceships.work # DNS server (PowerDNS)
network-04-dhcp.spaceships.work # DHCP server
```
### Application Services
```text
app-01-netbox.spaceships.work # NetBox IPAM
app-02-vault.spaceships.work # HashiCorp Vault
app-03-consul.spaceships.work # HashiCorp Consul
app-04-nomad.spaceships.work # HashiCorp Nomad
```
## Special Cases
### Management Interfaces
For infrastructure with multiple interfaces, include interface purpose:
```text
proxmox-<node>-mgmt.spaceships.work # Management network
proxmox-<node>-ceph.spaceships.work # CEPH public network
proxmox-<node>-backup.spaceships.work # Backup network
```
### Virtual IPs (FHRP/VIPs)
```text
vip-01-k8s-api.spaceships.work # Kubernetes API VIP
vip-02-haproxy.spaceships.work # HAProxy VIP
vip-03-postgres.spaceships.work # PostgreSQL VIP
```
### Service Endpoints
```text
service-01-api.spaceships.work # API endpoint
service-02-web.spaceships.work # Web frontend
service-03-cdn.spaceships.work # CDN endpoint
```
## Rules and Best Practices
### Mandatory Rules
1. **Always lowercase** - No uppercase letters
2. **Hyphens only** - No underscores or other special characters
3. **Two-digit numbers** - Use 01, 02, not 1, 2
4. **Descriptive purpose** - Purpose should clearly indicate function
5. **Valid DNS characters** - Only `a-z`, `0-9`, `-`, `.`
### Recommended Practices
1. **Consistent service names** - Stick to established service types
2. **Logical numbering** - Start at 01, increment sequentially
3. **Purpose specificity** - Be specific but concise (nexus, not nexus-container-registry)
4. **Avoid ambiguity** - Don't use `test-01-prod` or similar confusing names
5. **Document exceptions** - If you must break a rule, document why
## Validation
### Python Validation Script
```python
#!/usr/bin/env python3
# /// script
# dependencies = []
# ///
import re
import sys
PATTERN = r'^[a-z0-9-]+-\d{2}-[a-z0-9-]+\.[a-z0-9.-]+$'
def validate_dns_name(name: str) -> tuple[bool, str]:
"""Validate DNS name against convention."""
if not re.match(PATTERN, name):
return False, "Name doesn't match pattern: <service>-<NN>-<purpose>.<domain>"
parts = name.split('.')
if len(parts) < 2:
return False, "Must include domain"
hostname = parts[0]
components = hostname.split('-')
if len(components) < 3:
return False, "Hostname must have at least 3 components: <service>-<NN>-<purpose>"
# Check number component (should be 2 digits)
number_component = components[1]
if not number_component.isdigit() or len(number_component) != 2:
return False, f"Number component '{number_component}' must be exactly 2 digits (01-99)"
return True, "Valid"
if __name__ == "__main__":
if len(sys.argv) != 2:
print("Usage: validate_dns_naming.py <dns-name>")
sys.exit(1)
name = sys.argv[1]
valid, message = validate_dns_name(name)
if valid:
print(f"✓ {name}: {message}")
sys.exit(0)
else:
print(f"✗ {name}: {message}", file=sys.stderr)
sys.exit(1)
```
**Usage:**
```bash
./tools/validate_dns_naming.py docker-01-nexus.spaceships.work
# ✓ docker-01-nexus.spaceships.work: Valid
./tools/validate_dns_naming.py Docker-1-Nexus.spaceships.work
# ✗ Docker-1-Nexus.spaceships.work: Name doesn't match pattern
```
## NetBox Integration
### Setting DNS Names in NetBox
**Via Web UI:**
IP Addresses → Add → DNS Name field: `docker-01-nexus.spaceships.work`
**Via Terraform:**
```hcl
resource "netbox_ip_address" "docker_nexus" {
ip_address = "192.168.1.100/24"
dns_name = "docker-01-nexus.spaceships.work"
description = "Docker host for Nexus container registry"
tags = ["terraform", "production-dns"]
}
```
**Via Ansible:**
```yaml
- name: Create IP address in NetBox
netbox.netbox.netbox_ip_address:
netbox_url: "{{ netbox_url }}"
netbox_token: "{{ netbox_token }}"
data:
address: "192.168.1.100/24"
dns_name: "docker-01-nexus.spaceships.work"
tags:
- name: production-dns
```
### Tagging for Auto-DNS
Tag IP addresses to trigger automatic DNS record creation:
**Production DNS:**
```text
Tag: production-dns
Zone: spaceships.work
```
**Development DNS:**
```text
Tag: dev-dns
Zone: dev.spaceships.work
```
**Lab/Testing DNS:**
```text
Tag: lab-dns
Zone: lab.spaceships.work
```
## PowerDNS Record Generation
### Automatic Record Creation
When an IP address with correct tags is created in NetBox:
```text
IP: 192.168.1.100/24
DNS Name: docker-01-nexus.spaceships.work
Tag: production-dns
→ A record created: docker-01-nexus.spaceships.work → 192.168.1.100
→ PTR record created: 100.1.168.192.in-addr.arpa → docker-01-nexus.spaceships.work
```
### Sync Verification
```bash
# Verify DNS record exists
dig @192.168.3.1 docker-01-nexus.spaceships.work +short
# Should return: 192.168.1.100
# Verify PTR record
dig @192.168.3.1 -x 192.168.1.100 +short
# Should return: docker-01-nexus.spaceships.work
```
## Common Mistakes
### Wrong Number Format
```text
❌ docker-1-nexus.spaceships.work # Single digit
✓ docker-01-nexus.spaceships.work # Two digits
❌ docker-001-nexus.spaceships.work # Three digits
✓ docker-01-nexus.spaceships.work # Two digits
```
### Wrong Separators
```text
❌ docker_01_nexus.spaceships.work # Underscores
✓ docker-01-nexus.spaceships.work # Hyphens
❌ docker.01.nexus.spaceships.work # Dots in hostname
✓ docker-01-nexus.spaceships.work # Hyphens only
```
### Wrong Case
```text
❌ Docker-01-Nexus.spaceships.work # Mixed case
✓ docker-01-nexus.spaceships.work # Lowercase only
❌ DOCKER-01-NEXUS.SPACESHIPS.WORK # Uppercase
✓ docker-01-nexus.spaceships.work # Lowercase only
```
### Missing Components
```text
❌ docker-nexus.spaceships.work # Missing number
✓ docker-01-nexus.spaceships.work # Complete
❌ 01-nexus.spaceships.work # Missing service
✓ docker-01-nexus.spaceships.work # Complete
❌ docker-01.spaceships.work # Missing purpose
✓ docker-01-nexus.spaceships.work # Complete
```
## Migration Strategy
### From Legacy Names
If you have existing DNS names that don't follow the convention:
1. **Document current names** - Create inventory of legacy names
2. **Create new names** - Following convention
3. **Create CNAME records** - Point legacy names to new names
4. **Update configs gradually** - Migrate services to use new names
5. **Monitor usage** - Track legacy CNAME usage
6. **Deprecate legacy names** - Remove after migration complete
**Example migration:**
```text
Legacy: nexus.spaceships.work
New: docker-01-nexus.spaceships.work
CNAME: nexus.spaceships.work → docker-01-nexus.spaceships.work
After 6 months: Remove CNAME, update all references to use new name
```
## Environment-Specific Domains
### Production
```text
Domain: spaceships.work
Example: docker-01-nexus.spaceships.work
```
### Development
```text
Domain: dev.spaceships.work
Example: docker-01-nexus.dev.spaceships.work
```
### Lab/Testing
```text
Domain: lab.spaceships.work
Example: docker-01-nexus.lab.spaceships.work
```
## Documentation
### In NetBox
Use the **Description** field to document:
- Primary purpose
- Hosted applications
- Related services
- Contact owner
**Example:**
```text
IP: 192.168.1.100/24
DNS Name: docker-01-nexus.spaceships.work
Description: Docker host for Nexus container registry.
Serves Docker and Maven artifacts.
Owner: Platform Team
Related: docker-02-gitlab.spaceships.work
```
### In Infrastructure Code
**Terraform example:**
```hcl
resource "netbox_ip_address" "docker_nexus" {
ip_address = "192.168.1.100/24"
dns_name = "docker-01-nexus.spaceships.work"
description = "Docker host for Nexus container registry"
tags = ["terraform", "production-dns", "docker-host", "nexus"]
}
```
## Further Reading
- [RFC 1035 - Domain Names](https://www.rfc-editor.org/rfc/rfc1035)
- [DNS Best Practices](https://www.ietf.org/rfc/rfc1912.txt)