Building Infrastructure on Deevnet IoTaaS

Stage Artifacts

Mon, 01 Jan 0001 00:00:00 +0000

Stage Artifacts #

The builder node (with internet access) stages artifacts to the artifact server before any recovery is needed.

What Gets Staged #

Artifact	Source	Role/Task
Fedora install tree	rsync from Fedora mirrors	`artifacts` role
Fedora Server ISO	download.fedoraproject.org	`artifacts` role
Proxmox VE ISO	enterprise.proxmox.com	`artifacts` role
SSH public keys	Generated locally	`artifacts` role
Container images	docker.io, etc.	`artifacts` role

Commands #

From builder node with internet:

Seed Inventory

Mon, 01 Jan 0001 00:00:00 +0000

Seed Inventory #

Before any host can be PXE booted, its definition must exist in the Ansible inventory. MAC addresses, IP assignments, DNS records, and DHCP reservations are all driven from host_vars.

Repository: ansible-inventory-deevnet

When This Is Required #

Scenario	Action
Capacity expansion	Add host to `hosts.yml`, create new `host_vars/<hostname>.yml`
Hardware replacement	Update MAC address in existing `host_vars/<hostname>.yml`
Greenfield build	All hosts need both steps

Inventory Structure #

ansible-inventory-deevnet/
└── mobile/
 ├── hosts.yml # Main inventory (hosts and group memberships)
 ├── group_vars/ # Variables by group
 └── host_vars/ # Per-host variables (MAC, IP, DNS, DHCP)
 ├── dv02hyp001p01.yml
 ├── dv02hyp002p02.yml
 └── ...

Adding a New Host (Expansion) #

1. Add to hosts.yml #

Add the hostname to appropriate groups:

Build-Time Secrets

Mon, 01 Jan 0001 00:00:00 +0000

Build-Time Secrets #

Two builds need a live Proxmox API token:

Build	Where	Needs
Fedora VM templates (Packer)	`deevnet-image-factory`: `make proxmox-fedora-pve1` / `-pve2`	the node’s API token
The tenant fabric (Terraform)	`deevnet-tenant-fabric`: `make fabric-init` / `fabric-plan` / `fabric-apply`	the tenant hypervisor’s API token

The rule: a build secret is fetched per run into the build’s own process environment. It is never written to a file and never passed on a command line ( Security Controls, CHG-0026). Until CHG-0026, the image factory rendered the token into build/pve-env/<node>.env. Those files outlived every build, and one of them, under a node’s old name, was still being read by the fabric weeks later.

Vault Operations

Mon, 01 Jan 0001 00:00:00 +0000

Vault Operations #

Ansible Vault protects sensitive variables (passwords, API keys, certificates) stored in the inventory. Each environment has its own set of vault.yml files that must be encrypted at rest and decrypted only while editing.

Repository: ansible-inventory-deevnet

Setup #

After cloning the inventory repository, run the one-time hook setup:

cd ansible-inventory-deevnet
make install-hooks

This runs git config core.hooksPath hooks, pointing Git at the version-controlled hooks/ directory. The hooks stay in sync with the repo automatically — no copying required. This must be run once per clone.

Configure PXE

Mon, 01 Jan 0001 00:00:00 +0000

Configure PXE #

Configure PXE boot authority before provisioning hosts.

Greenfield Build (No Core Router) #

For initial site build or full recovery, the bootstrap node provides DNS/DHCP/TFTP for the management subnet.

cd ~/home/ansible-collection-deevnet.builder
make bootstrap-auth

This:

Discovers the WAN interface from inventory (bootstrap_wan_interface_key)
Enables IP forwarding and masquerading on the WAN interface
Activates dnsmasq for DHCP/DNS/TFTP on the downstream (management) interface
Populates DNS host records and DHCP static reservations from inventory
Swaps the management interface IP from the reserved address to the gateway address

The IP swap is the last step — it drops the SSH connection. All configuration completes first while connectivity is stable. Reconnect at the gateway IP to verify.

Build Network

Mon, 01 Jan 0001 00:00:00 +0000

Build Network #

Configure network infrastructure: Core Router, VLANs, firewall, DHCP, and wireless access points.

Collection: deevnet.net

Components #

Component	Role
Core Router	Firewall, DHCP, DNS, routing
Switch/VLANs	Network segmentation
Wireless AP	SSIDs, guest networks

Prerequisites #

Before the automated build-network procedures begin, the following manual steps must be completed:

Prerequisite	Method	Notes
Core Router	Fresh OPNsense install from USB	Manual installer; no PXE support
Access Switch	Factory reset to default state	Clears any prior VLAN/port config
Wireless AP	Factory reset to default state	Clears any prior SSID/network config

Additionally:

Build Management Plane

Mon, 01 Jan 0001 00:00:00 +0000

Build Management Plane #

Install and configure a Proxmox VE hypervisor, from bare metal to a node that automation can build VMs on. It applies to a new node, and to a rebuild of an existing one. The worked example is the management hypervisor, dv02hyp001p01.

This covers the hypervisors themselves. Putting an OS on the VMs that run on top of them is Build a Management-Plane VM.

Everything below comes from inventory or from code, except three manual steps: booting the ISO, running the first bootstrap script at the console, and creating the API token.

Allocate VM Identity

Mon, 01 Jan 0001 00:00:00 +0000

Allocate VM Identity #

A management-plane VM’s inventory entry exists before its MAC does. For bare metal a MAC is a hardware fact read off a NIC and typed into inventory; for a cloned VM there is no NIC to read until something creates one. So the MAC is derived from the Proxmox VMID and written into inventory by a tool, rather than invented by hand.

Adding a management VM is therefore one decision: allocate a VMID. The MAC, the DHCP reservation and the address all follow from it.

Build a Management-Plane VM

Mon, 01 Jan 0001 00:00:00 +0000

Build a Management-Plane VM #

Two supported ways to put an OS on a management-plane VM. Both produce a host that is reachable at its reserved address, carries a deterministic MAC, and can be rebuilt from its declaration.

	Clone from template	PXE netboot
Source of the OS	Packer-built Proxmox template	Install tree on the artifact server
Driven by	`deevnet.mgmt` `roles/proxmox_vm`	anaconda + `builder-node-<release>.ks`
Firmware	SeaBIOS (the template’s default)	UEFI/OVMF — the kickstart declares an ESP
Typical time	2–5 minutes	20–40 minutes
Use it for	Anything that just needs the current base image	Validating the netboot path, or a host whose disk layout the template cannot produce

Clone is the default. Reach for PXE when the point is the PXE path — proving a new Fedora release installs unattended before pinning any auto-installing host to it — or when the kickstart’s layout (a large /srv, separate /home) is what you actually want.

Verify Site

Mon, 01 Jan 0001 00:00:00 +0000

Verify Site #

Validation after the site is built or rebuilt, once every component is up.

Overview #

Each build phase checks itself: the Ansible roles assert what they deployed, and each change record carries its own verification. This page is the end-to-end pass: a short list of checks that, together, say the site is up.

Run it from the Builder, on the management segment, which reaches every zone. Tenant-facing segments are checked from a real client instead: see From the client segments.

Authority Transition

Mon, 01 Jan 0001 00:00:00 +0000

Authority Transition Runbook #

Procedures for transitioning DNS/DHCP authority between the builder and production network infrastructure.

For the architectural model, see Builder → Authority Transition.

Build context: During a greenfield build, these transitions happen as part of the Building Infrastructure sequence — Configure PXE enters bootstrap-authoritative mode, and Build Network transitions to core-authoritative mode. This page is the standalone reference for both directions.

Overview #

Transition	From	To	When
Promote to production	Builder-authoritative	Router-authoritative	After network infrastructure is configured and validated
Revert to bootstrap	Router-authoritative	Builder-authoritative	Before substrate rebuild or recovery

Both transitions are automated via playbooks in deevnet.builder and deevnet.net. The IP swap is the final step in each playbook and drops the SSH connection — reconnect at the new IP to verify.