Decisions on Deevnet Infrastructure Platform

ADR-0001: Tenant Network Fabric

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0001: Tenant Network Fabric #


Status	Accepted
Date	2026-08-30
Scope	Tenant networking on the tenant hypervisor; relationship to the core router and management hypervisor
Supersedes	The Option A tenant-networking model previously described in `architecture/tenant/networking.md` (core-router-owned per-tenant VLANs)

Context #

We are ready to begin hosting tenants — isolated application workloads — on the tenant hypervisor. A tenant needs a network: an address space, a gateway, routing, isolation from other tenants, and a controlled path to the outside world.

ADR-0002: Tenant Fabric Numbering

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0002: Tenant Fabric Numbering #


Status	Accepted
Date	2026-08-30
Scope	Address and identifier allocation for the tenant fabric and the tenants on it
Depends on	ADR-0001: Tenant Network Fabric

Context #

ADR-0001 made globally-unique numbering its first build requirement: allocate VNIs, VRF identifiers, and tenant subnets as if they already share one fabric, so that gaining a member is additive rather than a renumbering. It did not say what that scheme is. The Tenant Platform roadmap carried it as an open thread.

ADR-0003: Tenant Egress on a Single-Member Fabric

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0003: Tenant Egress on a Single-Member Fabric #


Status	Accepted
Date	2026-09-01
Scope	How tenant workloads reach the perimeter, and why they must reach it rather than route around it
Depends on	ADR-0001: Tenant Network Fabric

Context #

ADR-0001 chose a self-contained EVPN/VXLAN fabric per tenant hypervisor, realized today as a single-member fabric on hv02 and designed to expand by adding members. Seam 1 of that record defines north-south egress: aggregate tenant traffic leaves the fabric on a transit VLAN, and the core router is a perimeter that never learns tenant address space.

ADR-0004: Tenant DNS Publication

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0004: Tenant DNS Publication #


Status	Accepted
Date	2026-09-01
Scope	How a tenant’s own DNS records reach the resolver substrate clients use
Depends on	ADR-0001: Tenant Network Fabric, ADR-0002: Tenant Fabric Numbering
Extended by	ADR-0005: Tenant Zone Apex Ownership — the zone apex, which §4 does not assign
Extended by	ADR-0006: Tenant Code Boundary — where the tenant’s code lives, which §5 does not say
Extended by	ADR-0010: Tenants Consume Platform Services — §5’s line as a rule for every platform service, not only DNS (Proposed)

Context #

Seam 2 of ADR-0001 splits addressing from naming: IPAM belongs to the fabric, while DNS is tenant-owned but published to the substrate zone. With ADR-0003 closing egress, this is the last part of the tenant contract still undecided.

ADR-0005: Tenant Zone Apex Ownership

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0005: Tenant Zone Apex Ownership #


Status	Accepted
Date	2026-09-03
Scope	Who owns the SOA and apex NS records inside a delegated tenant zone
Extends	ADR-0004: Tenant DNS Publication

Context #

ADR-0004 drew a clean line: the substrate creates the zone and issues the TSIG key, and the tenant writes the records inside it. Substrate provides the machine; tenants provide the content.

Building it exposed a category the line does not cover. A zone’s apex — its SOA, and the NS records naming the servers authoritative for it — is made of records, which puts it on the tenant side of that table. But it is created by pdnsutil create-zone, which is a substrate act, and it describes the zone’s own existence rather than anything the tenant runs. It is neither the machine nor the content.

ADR-0006: Tenant Code Boundary

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0006: Tenant Code Boundary #


Status	Accepted
Date	2026-09-03
Scope	Where a tenant’s code lives, and what it needs from the substrate to stand alone
Depends on	ADR-0001: Tenant Network Fabric, ADR-0002: Tenant Fabric Numbering
Extends	ADR-0004: Tenant DNS Publication §5

Context #

ADR-0004 §5 drew a line that has held up well: onboarding may touch substrate; the recurring lifecycle may not. Adding a record, rebuilding, destroying — those are terraform apply and nothing else.

ADR-0007: Terraform State Custody

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0007: Terraform State Custody #


Status	Accepted
Date	2026-09-03
Scope	Where Terraform state lives, who may read it, and what protects it
Depends on	ADR-0006: Tenant Code Boundary
Extended by	ADR-0014: Tenant State Durability — how the store’s contents survive losing their host, once state holds device secrets (Proposed)

Context #

State custody was never decided. architecture/tenant/building.md offered three bullets —

Tenant Terraform state should be:

Stored in version control (for small deployments)

Or in remote backend (for team access)

Never edited manually

— which is a menu, not a decision, and it is contradicted on its own page by a terraform_remote_state example. Both state files have been committed to git on the strength of the first bullet.

ADR-0008: Host Naming and Site Codes

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0008: Host Naming and Site Codes #


Status	Accepted
Date	2026-09-04
Accepted	2026-09-05
Scope	How a host is named, and how a site is identified in that name and in DNS
Related	ADR-0002: Tenant Fabric Numbering — the site octet this record’s site codes sit alongside

Context #

Only the mobile site has hosts. Its names — hv01, core-rt01, provisioner-ph01 — are scoped to their site by inventory boundary alone, and the home site’s shell would reuse every one of them.

ADR-0009: Network Device Configuration Is Inventory-Owned and Controller-Applied

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0009: Network Device Configuration Is Inventory-Owned and Controller-Applied #


Status	Accepted
Date	2026-09-10
Accepted	2026-09-10
Scope	Who owns, and who applies, the configuration of controller-managed network devices — the access switch and the AP — and how automation talks to the Omada controller
Extended by	ADR-0013: Management-Hypervisor Services Run as Containers on Domain VMs — where the controller runs (a container in the network management VM, `nms`), and the order a rebuild uses it in
Related	CHG-0004 — brings the controller current; CHG-0005 — the AP, the first device adopted under this record; CHG-0006 — the switch’s firmware; INC-0001 — the guards any reconciling role now carries

Context #

The mobile site’s network devices are configured three different ways today:

ADR-0010: Tenants Consume Platform Services

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0010: Tenants Consume Platform Services; They Are Not Woven Into the Substrate #


Status	Proposed
Date	2026-09-13
Scope	What makes a substrate-run service safe for a tenant to depend on, and where issuing a tenant something ends and becoming part of it begins
Extends	ADR-0004: Tenant DNS Publication §5, which drew the onboarding-versus-recurring line for DNS only
Extended by	ADR-0012: IoT Platform Services Through a Deevnet API and Terraform Provider — how §3 is met when the backing service can’t confine a tenant itself (Proposed)
Related	ADR-0006: Tenant Code Boundary, ADR-0007: Terraform State Custody, ADR-0011: Edge Devices Are Application-Owned and Platform-Attached

Context #

Three records have drawn the same line, each for one thing, and none of them says it is a rule:

ADR-0011: Edge Devices Are Application-Owned and Platform-Attached

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0011: Edge Devices Are Application-Owned and Platform-Attached #


Status	Accepted
Accepted	2026-09-15, once its four open questions were answered. Questions 1–3 were settled by the operator on 2026-09-15 ( CHG-0008 Step 9’s controller work made the platform side real); question 4 was settled on 2026-09-14.
Date	2026-09-13
Validated	2026-09-14, read-only, before acceptance. See Validation
Scope	Who owns a physical device an application uses, what the platform knows about it, which network it joins, and how it reaches the services it needs
Depends on	ADR-0010: Tenants Consume Platform Services
Related	ADR-0001: Tenant Network Fabric, ADR-0003: Tenant Egress on a Single-Member Fabric, ADR-0009: Network Device Configuration Is Inventory-Owned and Controller-Applied, ADR-0012: IoT Platform Services Through a Deevnet API and Terraform Provider

Context #

Physical microcontrollers have been treated as substrate IoT devices. The treatment conflated concerns that answer to different owners:

ADR-0012: IoT Platform Services Through a Deevnet API and Terraform Provider

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0012: IoT Platform Services Through a Deevnet API and Terraform Provider #


Status	Accepted
Accepted	2026-09-18, at the close of CHG-0013 — when the Wi-Fi half had been built, deployed and proven on the AP, not when it was written. The broker half (§8) and the device registry (§3) are decided but not built.
Date	2026-09-14
Reviewed	2026-09-14. Four of the original open questions were decided in review: the broker (§8), secrets after a rebuild (§4, §5), provider distribution (§7) and credential delivery (§9). The sources are quoted in each section. Revised the same day: the API is provisioning-only, and the broker authenticates from its own auth database (§8). Open questions 2, 5 and 6 were then answered: where the API, its database, the broker’s auth database and the Omada controller run (§7, ADR-0013). Revised 2026-09-16: tenant workloads get broker accounts too (§3), and topic confinement is decided (§10, Open question 4). Revised 2026-09-18 by CHG-0013, which built the Wi-Fi half: a key is per tenant per trust class rather than per device (§3), v1 ships on `DVNTM-IOT` only (§3), the drift report names profiles rather than counting keys (§6), and `DVNTM-IOT` turned out to be a creation rather than a migration (Consequences).
Scope	How a tenant reaches an IoT platform service whose own interface can’t confine it to its scope, and what the substrate builds so it can
Extends	ADR-0010: Tenants Consume Platform Services §1 and §3, which require a scoped service but don’t say how one is built when the backing software can’t scope itself
Answers, in part	ADR-0011: Edge Devices Are Application-Owned and Platform-Attached open questions 1 (scoped registration) and 3 (Wi-Fi keys — answered per tenant per trust class, not per device; §3)
Related	ADR-0004: Tenant DNS Publication, ADR-0006: Tenant Code Boundary, ADR-0007: Terraform State Custody, ADR-0009: Network Device Configuration Is Inventory-Owned and Controller-Applied, ADR-0013: Management-Hypervisor Services Run as Containers on Domain VMs, ADR-0014: Tenant State Durability (§4 and §5 depend on it)

Context #

The rule has no mechanism for IoT #

ADR-0010 set the test for a platform service: does this recurring tenant action need a substrate commit? For IoT, two actions fail it today:

ADR-0013: Management-Hypervisor Services Run as Containers on Domain VMs

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0013: Management-Hypervisor Services Run as Containers on Domain VMs #


Status	Accepted
Accepted	2026-09-16, once CHG-0008 built all six domain VMs and moved the controller, tenant DNS and the tenant state store onto them
Date	2026-09-14
Scope	How services on the management hypervisor are grouped into VMs and named, where the site’s Omada controller officially runs, and the order a rebuild uses them in
Extends	ADR-0009: Network Device Configuration Is Inventory-Owned and Controller-Applied, which decided that the controller is the actuator for switch and AP configuration, but not where it runs
Extended by	ADR-0014: Tenant State Durability — the provisioning VM holds both the state store and the API’s database, so their data needs a data disk and a copy on separate hardware (Proposed)
Related	ADR-0004: Tenant DNS Publication, ADR-0008: Host Naming and Site Codes, ADR-0012: IoT Platform Services Through a Deevnet API and Terraform Provider (its Open questions 2, 5 and 6), CHG-0005

Context #

One VM per function is where things were heading #

The management hypervisor, dv02hyp001p01, runs one VM per service today:

ADR-0014: Tenant State Durability

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0014: Tenant State Is Kept Data, Copied Away From What It Restores #


Status	Proposed
Date	2026-09-16
Scope	How the tenant state store’s contents survive losing the host, disk or hypervisor they live on, and how that copy is kept apart from the platform API’s database, which the state restores
Extends	ADR-0007: Terraform State Custody, which offered the store and recorded its durability as “the weakest part of the decision”; ADR-0013: Management-Hypervisor Services Run as Containers on Domain VMs, whose §1 and §6 put the store and the API’s database in one VM
Related	ADR-0010: Tenants Consume Platform Services §4, ADR-0012: IoT Platform Services Through a Deevnet API and Terraform Provider §4 and §5, Substrate Storage, Limits

Context #

ADR-0007 sized state loss as a rebuild #

When the store was offered, losing state was cheap, and ADR-0007 said exactly how cheap:

ADR-0015: Tenants Are Built Through the Deevnet API

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0015: Tenants Are Built Through the Deevnet API #


Status	Proposed
Date	2026-09-17
Revised	2026-09-17, before review. First written as onboarding only; widened so that everything that builds a tenant (its network, workloads and DNS records) is behind the API and tenants hold no substrate credential (§11–§14). Admission by enrollment token (§10). Secrets handling moved to ADR-0016.
Scope	Who allocates a tenant’s index, who builds a tenant’s network, workloads and names, what a tenant holds, and where the record of tenants lives
Extends	ADR-0012: IoT Platform Services Through a Deevnet API and Terraform Provider, whose API and provider become the way every tenant is created, not only the way it registers devices
Supersedes, in part	ADR-0002: Tenant Fabric Numbering, where it says allocation is recorded in `TENANTS.md` (the numbering itself stands); ADR-0006: Tenant Code Boundary §1 (the module consumed by tag), §2 (the attachment rendered into the repository) and §3 (the reference implementation that cannot be applied). A tenant still lives in its own repository.
Related	ADR-0003: Tenant Egress on a Single-Member Fabric, ADR-0004: Tenant DNS Publication, ADR-0005: Tenant Zone Apex Ownership, ADR-0007: Terraform State Custody, ADR-0010: Tenants Consume Platform Services, ADR-0014: Tenant State Durability, ADR-0016: Substrate Secrets in OpenBao

Context #

Onboarding today is a substrate commit and five automation runs #

A tenant is created in two halves. The substrate half runs once:

ADR-0016: Substrate Secrets in OpenBao

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0016: The Substrate Keeps Its Runtime Secrets in OpenBao #


Status	Proposed
Date	2026-09-17
Scope	Where the credentials substrate services use at runtime are kept, how tenant secrets are protected at rest and in transit, and where the internal certificate authority lives
Extends	ADR-0015: Tenants Are Built Through the Deevnet API, whose API holds credentials for PowerDNS, the core router, the state store and the tenant hypervisor, and stores tenant secrets
Related	ADR-0012: IoT Platform Services Through a Deevnet API and Terraform Provider §9 (credential delivery), ADR-0013: Management-Hypervisor Services Run as Containers on Domain VMs, ADR-0014: Tenant State Durability

Context #

How the API’s credentials are handled in ADR-0015’s first slice #

Credential	Kept in	Reaches the API as	Can do
Operator token, database password	inventory vault	a root-only env file, then container environment	the whole API; its database
PowerDNS HTTP API key	inventory vault; also in `pdns.conf` on the DNS host	the same env file	every zone and key on the server
Core router API key	the router automation user’s vault entry	the same env file	everything that user may do on the router
State-store admin	inventory vault	the same env file	user and policy administration
Proxmox token	inventory vault	the same env file	what ADR-0015 grants it on the tenant hypervisor

And for what is handed to tenants:

ADR-0017: How Tenant Code Reaches a Tenant Workload

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0017: How Tenant Code Reaches a Tenant Workload #


Status	Proposed
Date	2026-09-18
Revised	2026-09-18, twice, before review. First written as a single choice between delivery mechanisms, with tenant-authored cloud-init user-data recommended; that draft argued the wrong layer. Rewritten to separate the tenant-facing contract (§1, settled) from the substrate-side mechanism (§2, open). Then revised again once both halves of the leading mechanism were tested rather than assumed, adding the measured SMBIOS limit and §3, the fetch rule that keeps ADR-0012 provisioning-only.
Scope	How a tenant’s own application code and configuration arrive on a tenant workload, and how they are kept current — not what the code is, and not how the workload itself is built
Extends	ADR-0010: Tenants Consume Platform Services, which says tenants own their code but does not say how it is delivered; ADR-0015: Tenants Are Built Through the Deevnet API, which builds the empty workload this decision fills
Related	ADR-0001: Tenant Network Fabric, ADR-0003: Tenant Egress on a Single-Member Fabric, ADR-0011: Edge Devices Are Application-Owned, ADR-0012: IoT Platform Services Through a Deevnet API and Terraform Provider, ADR-0016: Substrate Secrets in OpenBao, ADR-0018: Operator Access to Tenant Workloads

Context #

The substrate can build a workload but cannot fill one #

ADR-0015 made tenant building complete: a tenant declares deevnet_workload and gets a running Fedora VM with an address, a name, DNS and egress. What arrives is an empty machine. Nothing in the substrate puts an application on it, and nothing says how a tenant should.

ADR-0018: Operator Access to Tenant Workloads

Mon, 01 Jan 0001 00:00:00 +0000

ADR-0018: Operator Access to Tenant Workloads #


Status	Proposed
Date	2026-09-18
Scope	Whether the substrate’s operator networks may reach tenant workloads, and what that does to the “no inbound path” property other records rely on
Supersedes, in part	ADR-0001: Tenant Network Fabric and ADR-0002: Tenant Fabric Numbering, where each says the core router never learns tenant address space. It now learns one aggregate route. Everything else in both records stands, including SNAT at the exit node and per-tenant isolation inside the fabric.
Related	ADR-0003: Tenant Egress on a Single-Member Fabric, ADR-0010: Tenants Consume Platform Services, ADR-0011: Edge Devices Are Application-Owned, ADR-0012: IoT Platform Services Through a Deevnet API and Terraform Provider, ADR-0017: How Tenant Code Reaches a Tenant Workload

Context #

The property, and who relies on it #

Six records state some form of “a tenant has no inbound path”. They do not all mean the same thing, and the difference matters: