Tenant Removal #

Taking a tenant out of service. The API tears down almost everything it built for the tenant; this page covers what that removes and the leftovers the operator cleans up by hand. Creating a tenant is Tenant Admission.

Who deletes #

Either the tenant or the operator can delete a tenant. Both end in DELETE /v1/tenants/{name}.

make remove-tenant NAME=<name>

Removing tenant trm (index 5):
  workloads:  w1
  then its network, DNS zones, broker accounts, Wi-Fi keys, log tokens and Grafana login,
  and its Terraform state in tf-state/tenants/trm/, every version.
This cannot be undone. Type 'trm' to go ahead: trm
deleted workload w1
deleted tenant trm
state purged: tf-state/tenants/trm/ is empty

It deletes the tenant’s workloads, then the tenant, then its state, and stops at the first step that fails. It reads the operator token from the API container, as make admit does. CONFIRM=<name> answers the question in advance.

The API refuses to delete a tenant (409) while its registry holds any workloads, which is why the target deletes them first. It checks only the registry: a VM made outside the API is not seen, and would also keep the tenant’s VNet from being deleted.

What a delete removes #

The steps run in this order. Each one treats “already gone” as done.

The index is free again as soon as the row is gone. The next tenant created gets the lowest free index.

What a delete leaves #

The state holds every credential the tenant was issued. They are all revoked by the delete, but the state is still the tenant’s data, and any tenant later created under the same name is given the same prefix.

Log lines are addressed by index, not by name. A new tenant is given the freed index straight away, so until the retention runs out its partitions still hold the previous tenant’s lines.

The last three rows are gaps in the delete itself, tracked on the Tenant Removal roadmap.

Removing the tenant’s state #

make remove-tenant ends with this. After a tenant has run its own terraform destroy, run it alone:

make purge-tenant-state NAME=<name>

It refuses while <name> is still a tenant, since that tenant would still be writing there. It removes every version under tf-state/tenants/<name>/, not just the current objects: the bucket is versioned, so a plain delete only adds delete markers, and every earlier state, credentials and all, stays stored underneath them. It runs mc inside the state store’s MinIO container, whose root credentials never leave it, and finishes only when no versions are left.

Remove the state before admitting the name again. Otherwise the new tenant’s make state-backend finds the old state and a lock nobody holds.

When it goes wrong #