Custody

Custody #

The Root CA’s and every Site CA’s keys exist only as passphrase-encrypted files on two encrypted key drives, kept apart. Both passphrases, the drives’ and the key files’, are kept offline, under the holder’s own control, and never with the key media.

Two key media #

The keys live on two key media, kept apart:

  • Primary key media: in the holder’s secure storage.
  • Backup key media: somewhere else, so one loss (fire, theft, a bag left behind) cannot take both.
  • The paper record: fingerprints and dates of every certificate the ceremonies made. It is how you recognize the right file, and how anyone checks a certificate they were handed.

Neither key media is ever plugged into a networked machine.

Opening a key drive needs only the drive and both passphrases, on any Linux machine with cryptsetup: the pi-pki image on any Pi, or a Fedora live USB. Nothing ties a drive to the Pi or microSD that made it ( Root and Site CA Ceremony).

Yearly check #

Every year, check that both copies still open. On the offline machine ( Root and Site CA Ceremony):

sudo deevnet-pki-media.sh keys open primary          # the drive's passphrase
openssl pkey -in /mnt/keys/deevnet-root-ca.key -noout
openssl pkey -in /mnt/keys/deevnet-mobile-site-ca.key -noout   # and each other site's
sudo deevnet-pki-media.sh keys close
# then the same with: keys open backup

Each openssl pkey asks for the key’s passphrase and prints nothing. Note the date in the paper record. A copy that does not open is replaced from the other at once, by copying the file, never by generating a new key.

Rotation calendar #

Each CA is rotated a year before it expires. At the yearly check, also check the calendar:

CAIts key is held bySignsValid forRotate it
Deevnet Root CAthe operator, offlineSite CAstwenty years19 years after it was made
Site CAthe operator, offlineits site’s issuing CAsten years9 years after it was made
Substrate CAsite automationevery substrate server certificatefive years4 years after it was made
Tenant Device CAthe site’s secret storetenant devices’ client certificatesfive years4 years after it was made
Tenant Service CAthe site’s secret storeserver certificates for tenants’ own servicesfive years4 years after it was made

Rotating a CA means making a new one with the same ceremony:

What a rotation reaches #

A rotation reaches everything below the CA rotated, and nothing above it. A certificate works only while every CA above it is still valid. When a CA expires, everything it signed stops working with it, however long those certificates had left. A rotation therefore has to move everything below the rotated CA onto the new one before the old one expires. The year’s head start is for that.

RotatingMade againReissuedHanded outWho notices
Deevnet Root CAthe root, every site’s Site CA, and every site’s issuing CAsevery certificate at every sitethe new root, to every trust store that holds the old one: substrate hosts, operator computers, tenants’ computers and Pis, device firmwareeveryone: each trust store must take the new root before the old one expires
Site CAthat site’s Site CA and all three of its issuing CAsevery certificate at that sitenothing: the root is unchangedonly that site’s services, as automation reinstalls their chains
Substrate CAthat site’s Substrate CAthat site’s substrate server certificatesnothingno one: the certificates roll over at their yearly renewal
Tenant Device CAthat site’s Tenant Device CAthat site’s tenant device certificatesnothingno one: devices pick up new certificates at their yearly renewal
Tenant Service CAthat site’s Tenant Service CAthat site’s tenant service certificatesnothingno one: each tenant’s certificate moves to the new CA when the tenant next renews it, within the year

A Deevnet Root CA rotation runs the old and new roots side by side.

  1. Make the new root.
  2. Make the Site CAs and issuing CAs under it, site by site.
  3. Install the new root beside the old one in every trust store.
  4. Reissue every certificate under the new chain.
  5. Remove the old root only once nothing uses it.

Treat it as a planned change of its own.

A Site CA rotation reaches only its own site.

  1. Make the new Site CA.
  2. Sign new Substrate, Tenant Device and Tenant Service CAs under it.
  3. Reissue every certificate at the site before the old Site CA expires.

Clients trust the root, not the Site CA, so nothing is handed out. Other sites are untouched.

An issuing CA rotation is absorbed by ordinary renewal. Certificates last a year and are reissued once fewer than 60 days remain ( Certificates standard 4.7). Once automation issues from the new CA, every certificate moves to it within the year, before the old CA expires. The old and new CAs both chain to the same Site CA and root, so clients trust both while they overlap.

These are planned rotations, with the old key still safe. A key that may have been exposed is replaced at once, without waiting for the calendar.

Lost key media #

A lost key media exposes nothing while the passphrases are safe, because the drive is encrypted and so is each key file on it. Copy the surviving media onto a new drive, and note it in the paper record.

Exposed key #

An exposed key is replaced, never reused, and only a re-root ends its trust. A key is exposed when the file and its passphrase may both have been seen, or, for the online issuing CAs, when the system holding it may have been compromised.

Deevnet publishes no revocation list, so a CA certificate stays valid until it expires. Making a new CA stops the site using the old one. It does not stop a thief using it: anything signed with the stolen key still chains to the trusted root. The Site CA’s name constraints bound the damage to Deevnet’s own names and private addresses; a public site cannot be impersonated.

ExposedIt stays valid forDo nowThen decide
Substrate CA (online, in the vault)up to 5 yearssign a new one ( Issuing CA), rotate the vault password, reissue everything (certs.yml)re-root, or accept the risk until it expires
Tenant Device CA (online, in OpenBao)up to 5 yearssign a new one, re-enroll every devicere-root, or accept the risk until it expires
Tenant Service CA (online, in OpenBao)up to 5 yearssign a new one, and have every tenant renew its certificates. A thief can certify any name under deevnet.net, a substrate name included, and every client would accept itre-root, or accept the risk until it expires
Site CA (offline)up to 10 yearsnew Site CA and new issuing CAs ( ceremony, path 2)re-root, or accept the risk until it expires
Root CA (offline)up to 20 yearsre-root—

Re-root when the key may be in someone’s hands who could use it on the site’s networks. Someone in a position to intercept traffic on the site, with a stolen key, can impersonate any Deevnet service to anything that trusts the root. That is worth the re-root: a new root and Site CAs ( ceremony, path 1), new issuing CAs, and the new root handed to every machine, tenant and computer, as in a root rotation.

Accepting the risk is reasonable when the exposure is only possible. A key media misplaced and found again, a passphrase glimpsed, a vault file copied with no sign it was opened: replace the CA, note the decision and the date in the paper record and the incident record, and let the old certificate run out.

Until a re-root, a computer can refuse the exposed CA by hand. Windows (the Untrusted Certificates store) and macOS (Never Trust in Keychain Access) can distrust a single CA certificate. Site automation has no equivalent today.

Record every exposure as an incident.

Both copies lost #

Losing both copies of a key stops new signing at that level. Certificates already issued keep working until they expire, but nothing new can be signed above the issuing CAs. For a Site CA, make a new one. For the root, it is the re-root above.

Page last modified: October 11, 2026