PXE Role #

Purpose #

The PXE boot infrastructure enables fully automated, zero-touch provisioning of bare-metal hosts. Hosts boot from the network, receive their OS installation automatically based on their MAC address, and require no human intervention.

Goals:

• Zero-touch — MAC-specific configs eliminate boot menus and manual selection
• UEFI-native — Modern UEFI boot with network-enabled GRUB
• Decoupled services — DHCP (Core Router) and TFTP (bootstrap node) are separate
• Air-gap capable — All boot artifacts served from local infrastructure

Use Cases #

Primary: Bare-Metal Provisioning #

PXE boot is the standard method for provisioning bare-metal hosts:

• Proxmox hypervisors
• Physical workstations and admin nodes
• Network appliances (where supported)

Secondary: VM Testing #

VMs can PXE boot to validate new OS configurations before bare-metal deployment:

• Test kickstart changes without risking physical hardware
• Validate netboot image updates (kernel, initrd)
• Debug boot issues in a controlled environment

Once validated on VMs, the same MAC-specific config works unchanged on bare metal.

UEFI VM clients require a VirtIO RNG device #

A Proxmox VM with bios: ovmf cannot PXE boot without an entropy source. This is a documented Proxmox 8.4 known issue — “PXE boot on VM with OVMF requires VirtIO RNG” ( Roadmap) — and it arrived with pve-edk2-firmware 4.2025.02 in PVE 8.3.5.

EDK II’s fix for CVE-2023-45237 makes the UEFI network stack depend on EFI_RNG_PROTOCOL. With no entropy source the firmware disables network boot entirely, so it never creates a network boot entry and the guest emits zero DHCP packets:

BdsDxe: failed to load Boot0001 "UEFI QEMU QEMU HARDDISK " ... Not Found
BdsDxe: No bootable option or device was found.

Add the device when creating the VM:

rng0: source=/dev/urandom

Two things this failure is not, both verified on 2026-09-04 by rebuilding the VM from scratch each time and capturing on the segment:

• Not the NIC model. virtio and e1000 fail identically without entropy and both work with it. Keep virtio, to match what deevnet.mgmt roles/proxmox_vm uses for every other VM.
• Not stale firmware NVRAM. A brand-new efidisk0 behaves the same.

The alternative entropy source is a CPU exposing RDRAND (cpu: host, or a named model). VirtIO RNG is preferred: it is migration-safe. Note the PVE 8 default kvm64 has no RDRAND, so a VM with neither setting has no entropy at all.

Bare-metal clients are unaffected — real firmware has its own entropy sources.

Architecture #

sequenceDiagram
    participant Client as PXE Client
(VM or bare)
    participant Router as Core Router
(Kea DHCP)
    participant Boot as Bootstrap Node
(TFTP)

    Client->>Router: DHCP Request
    Router-->>Client: IP + next-server + boot-file
    Client->>Boot: TFTP: grubx64.efi
    Client->>Boot: TFTP: grub.cfg-MAC
    Client->>Boot: TFTP: vmlinuz, initrd.img

Note: The Core Router is currently OPNsense but the PXE infrastructure works with any router providing Kea DHCP with PXE options.

DHCP Configuration (Core Router) #

The Core Router’s Kea DHCP provides two critical options for PXE:

Subnet-Level Settings #

Applied to all hosts on the subnet unless overridden:

Subnet: 192.168.10.0/23
Next Server: 192.168.10.95
Boot File Name: grubx64.efi

Per-Host Reservations #

Per-host reservations override subnet settings. Each PXE-bootable host must have:

TFTP Server (Bootstrap Node) #

The bootstrap node runs in.tftpd via systemd socket activation:

Service: tftp.socket / tftp.service
Root: /srv/tftp
Port: 69/udp

Directory Structure #

/srv/tftp/
├── grubx64.efi                    # Network-enabled GRUB (built by grub2-mkimage)
├── grub.cfg                       # Default menu (fallback)
├── grub.cfg-02:DE:20:00:00:CB     # MAC-specific: dv02bld002v01
├── grub.cfg-02:de:20:00:00:cb     # ...and the lower-case variant
├── grub.cfg-02-DE-20-00-00-CB     # ...and both again, hyphen-separated
├── grub.cfg-02-de-20-00-00-cb     #    (firmware differs on which it asks for)
├── grub/
│   ├── grub.cfg                   # Alternate location
│   └── grub.cfg-*                 # MAC-specific configs
├── pxelinux.0                     # BIOS bootloader (legacy)
├── pxelinux.cfg/default           # BIOS menu (legacy)
├── fedora/43/
│   ├── vmlinuz                    # Fedora kernel
│   └── initrd.img                 # Fedora initramfs
└── vyos/
    ├── vmlinuz                    # VyOS kernel
    └── initrd.img                 # VyOS initramfs

Network-Enabled GRUB #

The bootloader is built with grub2-mkimage including network modules:

grub2-mkimage \
  -O x86_64-efi \
  -o /srv/tftp/grubx64.efi \
  -p "(tftp)/grub" \
  -d /usr/lib/grub/x86_64-efi \
  efinet tftp http net normal linux boot configfile \
  part_gpt part_msdos fat ext2 iso9660 \
  gzio all_video gfxterm

MAC-Specific Boot Configs #

Each host has a MAC-specific GRUB config that boots immediately without a menu:

Example: /srv/tftp/grub.cfg-02:DE:20:00:00:CB (dv02bld002v01)

# GRUB2 MAC-specific Boot Configuration
# Managed by Ansible - DO NOT EDIT MANUALLY
# Host: dv02bld002v01
# MAC: 02:de:20:00:00:cb

set default=0
set timeout=0

menuentry "Fedora 44 Server" {
    linux /fedora/44/vmlinuz \
        ip=dhcp \
        rd.neednet=1 \
        inst.repo=http://artifacts.mobile.deevnet.net/fedora/44/mirror inst.stage2=http://artifacts.mobile.deevnet.net/fedora/44/mirror inst.ks=http://artifacts.mobile.deevnet.net/kickstart/builder-node-44.ks
    initrd /fedora/44/initrd.img
}

Key points:

• timeout=0 — No menu, boots immediately
• default=0 — First (only) entry
• Kernel options — Point to artifact server for install media

Adding a New PXE Host #

1. Add DHCP Reservation (Core Router) #

Via Core Router UI or API, create a host reservation:

MAC Address: <new-host-mac>
IP Address: <static-ip>
Hostname: <hostname>
TFTP Server Name: 192.168.10.95
Boot File Name: grubx64.efi

2. Add to Ansible Inventory #

In ansible-inventory-deevnet/mobile/group_vars/bootstrap_nodes.yml:

bootstrap_grub_mac_configs:
  - hostname: new-host
    mac: "aa:bb:cc:dd:ee:ff"
    image_name: "Fedora 43 Server"
    dest_subdir: "fedora/43"
    boot_options: >-
      inst.repo=http://artifacts.mobile.deevnet.net/fedora/43/mirror
      inst.ks=http://artifacts.mobile.deevnet.net/kickstart/builder-node.ks      

3. Apply Bootstrap Role #

cd ~/home/ansible-collection-deevnet.builder
make rebuild
ansible-playbook playbooks/site.yml --limit bootstrap_nodes

4. Reconfigure Kea #

After adding DHCP reservation:

# Via Core Router API
curl -X POST "https://dv02cor002p01/api/kea/service/reconfigure"

Boot Sequence #

1. Power on — Host starts UEFI PXE boot
2. DHCP — Core Router Kea provides IP + next-server (192.168.10.95) + boot-file (grubx64.efi)
3. TFTP grubx64.efi — Host downloads network-enabled GRUB
4. TFTP grub.cfg — GRUB fetches default config
5. TFTP grub.cfg-MAC — GRUB finds MAC-specific config (no menu)
6. TFTP kernel/initrd — GRUB downloads OS boot files
7. HTTP install — Installer fetches packages from artifact server
8. Kickstart — Automated installation completes

Troubleshooting #

Check DHCP Options #

On the PXE boot screen, verify:

• Server IP: Should be 192.168.10.95 (not 192.168.10.1)
• Boot file: Should be grubx64.efi

If wrong, check both subnet AND per-host reservation in Kea.

Check TFTP Logs #

# On bootstrap node
journalctl -u tftp.service -f

Look for:

• RRQ from <ip> filename grubx64.efi — Bootloader request
• Client <ip> finished grubx64.efi — Successful transfer
• RRQ from <ip> filename /grub.cfg-<MAC> — Config lookup

Verify Files Exist #

# On bootstrap node
ls -la /srv/tftp/grubx64.efi
ls -la /srv/tftp/grub.cfg-*

Test TFTP Manually #

tftp 192.168.10.95 -c get grubx64.efi /tmp/test.efi
ls -la /tmp/test.efi  # Should be ~1.2MB

Ansible Configuration #

The PXE infrastructure is managed by the bootstrap role in deevnet.builder:

Summary #

1. DHCP (Core Router Kea) provides next-server and boot-file-name
2. TFTP (bootstrap node) serves GRUB and boot files
3. MAC-specific configs enable zero-touch automated installs
4. Per-host reservations override subnet defaults — update both when adding hosts