Review: Tenant PKI, October 2026 #

Round 3: the owner’s decisions #

The owner’s decisions on 2026-10-25 return the design to ADR-0031’s hierarchy, and every item in the decision register is now answered. The records were rewritten to match. The Summary through What uses OpenBao record rounds 1 and 2 as they stood; the dependency matrix, the migration and validation plans and the register below are round 3’s.

The model, in one picture:

Deevnet Root CA                     offline · the one anchor for people, computers and devices
├── Deevnet Mobile Site CA          offline · the Mobile Factory
│   ├── Deevnet Mobile Substrate CA     the factory's own infrastructure      key: vault, used by Ansible
│   ├── Deevnet Tenant Service CA       tenant servers, at every site         key: vault → imported into OpenBao
│   └── Deevnet Tenant Device CA        tenant devices, at every site         key: vault → imported into OpenBao
└── Deevnet <Site> Site CA          offline · any other site
    └── Deevnet <Site> Substrate CA     that site's own infrastructure        key: that site's vault, used by Ansible

What the owner decided:

• Only the Mobile Factory creates tenants. It is a contractor site: it goes to the devices, or devices are brought to it, for creation, renewal and development. Its long form is the Deevnet IoTaaS Mobile Development Factory.
• Identity outlives location by names, not by CAs. The owner first set the principle aside: being in development on the kit is part of a deployment’s identity. The design that followed keeps identity site-less anyway. A service’s identity is its site-less name, and a device’s is its URI. A site appears only as an extra SAN on a service’s certificate and as a device’s registration, its visa. A move to a site already in the SANs needs no new certificate.
• A site has only its substrate’s CAs; the Mobile Factory’s existing tenant CAs serve every site; devices trust the Deevnet Root (ADR-0039).
• A tenant service is named without a site, <service>.<tenant>.deevnet.net, in its CN and first SAN; its sites are extra SANs; devices dial the site-less names (ADR-0040).
• A take-home Pi is a Deevnet Pi by default; a non-deevnet card carries a starter kit, built from the ceremony tools, for a two-tier PKI of the owner’s own (ADR-0041).
• One OpenBao, sealed by Shamir shares, three with a threshold of two and one held in the vault, opened for a work session and rebuilt from the vault, with no snapshots (ADR-0042).

What the round found in the rewritten design, now requirements in the records:

Summary #

The direction holds, with one premise corrected. A Tenants CA beside the Site CAs fits the root’s pathlen:2. Per-tenant CAs give pinned devices a boundary they enforce. A tenant subtree gives the Tenants CA a bound that does not age. Keeping CA keys sealed except in sessions removes the largest risk of the first draft. The premise corrected: there is no runtime OpenBao. Nothing a deployed tenant runs on uses OpenBao today. The second store is the platform’s provisioning store, and the split is still worth having, for the reasons in ADR-0042.

Three things are not settled by reading and need proofs of concept: whether devices can keep one anchor across sites through SNI routing (ADR-0039 §6); what mbedTLS does with a non-self-signed, renewed anchor; and whether a restored PKI store keeps its issuers’ identities.

Two things must come before any new CA exists: backups of OpenBao that have been restored, and the PKI store’s manual seal.

Assessment #

What is sound #

• The hierarchy fits as built. Root pathlen:2 → Tenants CA pathlen:1 → tenant CAs pathlen:0 → leaves. The Names CA is a pathlen:0 sibling. No re-root.
• Per-tenant CAs give an enforceable device boundary, qualified: a pinned device refuses a CA made with another key, including one made with a stolen Tenants CA. Relying parties that anchor at the root (browsers, computers, tools) are not protected by it (ADR-0039 §4).
• A request does not authorize. The API admits only the operator or the tenant’s own token (ownTenant), and reads a request for its public key alone (ADR-0037 §2).
• A tenant subtree bounds the Tenants CA durably; exclusions could not.
• Runtime already survives a sealed OpenBao. The broker authenticates from its own database, and ADR-0012 says an API outage stops provisioning, never devices. The code confirms it.
• VerneMQ can run a listener per tenant, each with its own certificate and CA file (2.2.0’s listener.ssl.<name>).

Findings from the first round, and where each stands #

Conflict analysis #

“Runtime operation does not require PKI availability” was already true. It does not motivate a “runtime” store; it describes the system as it is. What a second store buys is different: the API starts and provisions unattended while CA keys stay sealed. The owner’s description of the runtime store as holding broker credentials and service-to-service secrets does not match the code. Broker passwords are bcrypt hashes in the broker’s own database, and no service reads OpenBao.

“Self-service onboarding” against “PKI availability is deliberate.” They meet in the PKI session: self-service holds within a session the operator opens, and a tenant created outside one is refused cleanly. At a meetup the operator is present anyway.

“The factory is autonomous” against “recovery must be tested off the kit.” Backups have to live somewhere other than the kit. A copy at the home site is a backup, not a dependency, provided the factory never needs it to operate. ADR-0042 leaves the place open.

“Certificates travel” against “the substrate holds no tenant key” (ADR-0037 §6 in the other direction). Under SNI routing, a site’s broker holds a key for a tenant’s canonical broker name, issued by the tenant’s Service CA. It is the site’s key, for the tenant’s name, and is recorded as a consequence rather than hidden.

“Human-facing names are friendly” against “one authority per namespace.” Vanity names need an issuer outside the tenant subtree. The Names CA (ADR-0040 option A) is another online issuer, broad in scope, whose control is the API’s ownership rule. That is the same position the Tenant Service CA is in today.

“Trust transitions are explicit” against firmware reality. A device must take a new anchor at graduation, and perhaps when its tenant CA is renewed. Many devices, EdS’s included, take an anchor only by cable. The PKI can require the transition; the application decides its cost (ADR-0011).

Revocation is still absent. Leaves last a year. A stolen server key is good until it expires. A stolen device key is refused only by deregistering the device. A stolen CA key is good until the CA expires. Nothing in these records changes that; ADR-0031’s open question remains the most important one.

What uses OpenBao #

OpenBao shares the identity VM with PowerDNS, has its Raft data on the OS disk, no audit device and no snapshot, and three AppRoles with no secret-ID TTL or address binding.

Dependency matrix #

What each record touches, as rewritten in round 3:

Migration plan #

Each phase has a change record, and none starts without the owner’s approval. P3 and P4 share one. As of 2026-10-31: P0 and P1 are complete, P1’s rebuild drill (V12) still to run; P3, P4 and P7 are in progress; P2, P5, P6 and P8 are planned.

P0 comes before P2. Sealing OpenBao before the API can tell a sealed store from a lost secret would damage tenants.

Validation plan #

Each test has an acceptance criterion, and the records cite them by number. V5 and V9 of round 2 are withdrawn with what they tested.

Where each stands, as of 2026-10-31. Passed in part: V2 and V10 (EdS’s home Pi, and on 2026-11-01 its reissued workload console: a name outside deevnet.net refused, and one certificate for both console names; recorded in CHG-0054), V1 (three ESP32s holding only the Root connected, by name, to that Pi), V3 (both mounts serve the ceremony certificates and every tenant read back after the reseal, CHG-0036; the key-identifier check is not recorded), V4 (the reserved labels in the API’s unit tests, not yet against the deployed API) and V13 (the API’s half, CHG-0052). V7, V8 and V11 passed on a Pi of the Pi lab, 2026-11-08 ( CHG-0056). Not run: V6, V12 and V14.

Decision register #

Every item is answered, by the owner, on 2026-10-25.

Principles, as round 3 leaves them (the owner’s): the Mobile Factory alone creates tenants and is autonomous, and builds sites as well as tenants; certificates travel, authorities stay; key export between sites is rejected; identity outlives location by site-less names, with a site only a SAN or a registration; a take-home Pi is a Deevnet Pi by default; recovery is a rebuild from code and the vault. "pki=own the generic default" is withdrawn.