// Benutzung · Konfiguration & Rollback
Config revisions, checkpoints & rollback
This is the reference for how the platform versions its own configuration — Locations and Resources and their settings, as configured through the platform itself. Every such change is journalled as an immutable revision; a user can mark a "safe" checkpoint and later roll a location back to it, write-forward, without ever erasing history.
This is deliberately scoped to platform config, not infrastructure state.
Kubernetes rollout history and Proxmox snapshots are a
separate system that wraps each backend's own mechanism — they only exist once
control does, and are documented separately. Revisions and checkpoints live in the same
inventory-svc database, since it is already the authority on this data.
What gets versioned, and what doesn't
Revisions cover user-initiated writes to config entities:
- Location: create, update, delete
- Resource: create, update, delete, credential attach/replace
They deliberately exclude:
- Auto-discovered sub-resources (Proxmox VMs/CTs, k8s nodes/pods). These
re-derive from the live source every poll cycle — their status and timestamps
churn constantly, and "rolling back" a VM the platform merely observed is
meaningless. Versioning them would drown the real config history in noise.
That holds for a user's writes to them too: an edit of a discovered resource
(its own name and tags,
display_nameanduser_tags), or a credential set on or detached from one, applies but writes no revision , as its delete never did. Discovery keeps the own name and tags, but no rollback restores them. Revisions of discovered resources written by older releases stay in the history, and every rollback skips them. - Hardware inventory reports (the Redfish component tree), for the same reason — the agent replaces them wholesale on every walk.
So the history is exactly the set of decisions a human made through the platform, and nothing else.
Rollback — always write-forward
Rolling back to a checkpoint writes the checkpoint's snapshots back as the
current state, and records each change as a new revision with
operation = "rollback". History is never rewritten: no row is ever edited to
say something other than what it said when it was written, so you can always see
that a rollback happened, when, by whom, and what it restored. A rollback of a
bad rollback is just another rollback. (Old rows can be pruned once an admin
opts in — see below — but a surviving row always still means what it meant.)
Rollback is scoped to a location's own settings and its user-managed resources (auto-discovered ones are left alone — they re-derive from the live source). A checkpoint taken while discovered resources had revisions (written by an older release) holds them, and a rollback skips them in either mode: it never restores, recreates or removes a discovered resource, and does not count it in its summary. In older releases a full rollback to such a checkpoint failed while the resource existed, or recreated it as a registered resource once it was gone. It has two modes, chosen per rollback:
| Mode | Existing entity changed since | Entity deleted since | Entity created since |
|---|---|---|---|
| safe (default) | restored to checkpoint values | left absent | left in place |
| full reconcile | restored to checkpoint values | recreated (same id) | removed |
Safe is the least-destructive default: it fixes what drifted but never deletes or resurrects anything. Full reconcile makes the location match the checkpoint exactly — use it when you want the location to be the checkpoint, accepting that resources added since will be removed and ones deleted since will come back. A resource it removes takes the resources discovered under it along, as a delete does; a resource it recreates gets its discovered resources back, as new resources, from the agent's first collection of it (for a Proxmox node, after its credential is re-attached; a Kubernetes cluster without one is collected in-cluster).
"Created since" is proven, not assumed: full reconcile removes a
registered resource the checkpoint does not know only if the resource was
created after the checkpoint was taken (its created_at) and has a revision.
Anything else it leaves, and lists in the summary's kept_ids: a resource with
no revision at all, or one that existed before the checkpoint without being in
it, such as a row from before revision history taken into a checkpoint made
before migration 0038 journalled it.
A location's settings, local_collector included
A rollback restores the location's name, kind, region, network config and
local_collector flag as the checkpoint holds them. local_collector
decides whether the hub's bundled core agent or the location's own agent
collects it, so a rollback that changes it moves where collection runs. A
checkpoint taken before the flag existed (migration 0014) does not record it;
rolling back to one leaves the flag as it is now rather than clearing it.
Changing the flag needs location.edit on all locations, as an edit of it
does (see RBAC & per-location access). A rollback by a caller without that grant
that would change it is refused with 403 before anything is written; one
whose checkpoint holds the value the location has now goes ahead.
Open edit forms go stale
A rollback writes the location's row and the resources it restores in place
as an edit does, and moves their tokens where it changes a value: the
location's updated_at when one of the location's own fields (name, kind,
region, network config, local_collector) differs from the checkpoint, and a
restored resource's edited_at when one of its restored fields does.
Values are compared as the API and the agent read them, as a save compares
them, so a NONE an older row or snapshot holds is the default it stands for
(no tags or extra vantages, not public, method GET, no redirects followed,
probing enabled), not a change. An edit form
opened before a rollback that changed its record is refused with 409 STALE
on save, and reloads, rather than writing the values it read over the
restored ones; a rollback
that puts back what a record already holds moves no token, so no form goes
stale over it (the location is still journalled). A recreated resource gets
a new edited_at as any create does.
A resource's TLS certificate checking is restored
A rollback restores a resource's allow_invalid_certs flag as the checkpoint
holds it, whether it restores the resource in place or recreates it (before, an in-place restore kept the current value). The flag turns off TLS
certificate verification for the resource's agent connection or check, so
a rollback can turn certificate checking back off: if the resource had it
off at the checkpoint and it was turned on since, the rollback turns it off
again. A checkpoint taken before the flag existed (migration 0011) does not
record it; an in-place restore from one leaves the flag as it is now rather
than reading it as off.
A rollback that would turn the checks off for a resource that keeps a
credential needs resource.allow_invalid_certs at the location: without it the whole rollback is refused with 403 before anything is
written, as an edit turning them off would be (see RBAC & per-location access). Detach
the credential first, or have someone holding that permission roll back.
Credentials are never rolled back
Snapshots never contain credential secrets (only whether a credential is attached), so rollback restores config fields only. A resource recreated by a full rollback comes back without its credential and must have one re-attached; an existing resource keeps whatever credential is currently on it. This is intentional — the revision journal is not a secret store.
Vantage points and credentials follow the edit rules
A rollback restores a check's extra vantage points, a resource's kind and address, and keeps an attached credential's host, only as creating or editing the resource would allow. Other restored fields are not re-validated against today's write rules. The whole rollback is planned and checked before anything is written, so a refused rollback changes nothing.
- Extra vantage points (
check_vantages) come back only where the location still exists, only when the checkpoint recorded a check kind (the kind is restored with them), and only on a check that ends up with no credential (a credential is location-scoped). Any other entry is dropped from the restored list, not refused, so a checkpoint stays usable after one of its vantage locations is deleted. - The status-page flag (
check_public) comes back only when the checkpoint recorded a check kind; on any other kind it is restored asfalse, so a checkpoint taken while a resource still held a stale one (before migration0035cleared it) does not put it back. - Every vantage point the rollback adds to a check needs
resource.createat that location, as creating a check with it directly does. "Adds" means the check does not hold it right now: a vantage point the same check already has needs nothing, so a check an admin gave a foreign vantage point does not block rolling the rest of the location back. A check a full rollback recreates adds all of its vantage points. One missing permission refuses the whole rollback with 403. The gateway passes the caller's permitted locations to inventory-svc inx-fleeger-rollback-locations, and drops any copy a client sends. Without that header, as on a call that did not come through the gateway, a rollback that would add a vantage point is refused. An older gateway-svc does not strip a client's copy, so the check holds only while gateway-svc is at least as new as inventory-svc. - A credential never follows an address to another host. A rollback that
would restore an existing resource's address to a different host (another
port or scheme counts, so
httpscannot becomehttp) while a credential is attached is refused with 400, naming the resource. Detach the credential first, as for an edit. - A restored kind and address pass the check a create makes. A
rollback that would change an existing resource's kind or address, or
recreate one, to a pair creating it directly would refuse (a
javascript:URL as anhttp_check's address, say) is refused with 400, naming the resource. A resource whose kind and address the rollback leaves as they are is not checked, so a location holding one stored before this check can still be rolled back.
Who may roll back
A rollback needs the rights its changes need when made directly, at the location rolled back:
| Mode | Needs at the location |
|---|---|
| safe | resource.edit and location.edit (it restores the location's settings and its resources in place) |
| full | those, plus resource.create and resource.delete (it also recreates and removes resources) |
plus resource.create at every vantage point it adds at another location
(above). The gateway reads the mode from the request body and fails closed:
only an explicit "mode": "safe" gets the safe set. A body without a mode,
which inventory-svc runs as safe, still needs the full set, as does one the
gateway cannot read. One missing right refuses the rollback with 403,
naming it. Taking a checkpoint and lifting a rollback's lock stay
location.create.
Retention of history
Config revisions were originally kept forever, and they still are by default.
They are, however, covered by the shared retention policy, which
carries two knobs read by inventory-svc's reaper. An admin who sets
revision_retention_secs expecting it to touch only metrics and dead inventory
is wrong — it prunes this journal too, so it is worth knowing exactly what
survives.
revision_retention_secs— an age ceiling, in seconds. Unset (null) means keep forever, which is both the shipped default and how every policy row written before the retention migration reads back. Nothing is pruned until an admin opts in. Must be positive.revision_min_per_entity— a per-entity floor: the newest N revisions of each(entity_type, entity_id)survive however old they are. Default 20; a configured 0 is clamped up to 1, so an entity can never lose its newest revision — that row is its current config state, and it is what checkpoint creation reads.
A revision is deleted only when both conditions hold: it is older than the ceiling and it falls outside its entity's floor. Age alone would erase the entire history of a rarely-edited resource — the record of how it reached its current state, the only reason the table exists — while a busy entity kept its recent churn. With the floor, "old" only ever removes the surplus.
Checkpoints pin what they point at. Every revision id listed in any
checkpoint.revision_ids is exempt from pruning regardless of age or floor. A
checkpoint is a user-marked rollback target and a rollback resolves its revisions
one by one, so a reaped one would silently vanish from the restore — gutting the
checkpoint instead of failing loudly. A checkpoint you keep keeps its
revisions. The practical rule: if a point in history matters, mark it.
Pruning runs inside the ordinary retention sweep and is bounded per cycle (a cap on entities examined and on deletions per entity), so the first sweep after an admin opts in does not try to clear years of history in one transaction — the leftovers go next cycle. Each sweep that deleted something records the count in the sweep audit log alongside the inventory/metrics counts.
Nothing here rewrites history: pruning removes whole old rows, it never alters a surviving one, and the write path remains append-only.
Rollback — all or nothing
A rollback writes everything it changes in one transaction: the location's settings, each restored, recreated and removed resource with the resources discovered under it, their hardware rows, and a revision for each, rendered from the row as written. It either commits all of it or changes nothing; there is no half-rolled-back location to finish by running it again.
- Lease. Before planning, a rollback takes a lease on its location
(
rollback_lock, migration0037, keyed by the location): 120 s, renewed before each attempt, deleted by the rollback's own commit, and lifted by the rollback on every other way out: a failure, a plan with nothing to write, a panic or its request dropped (lifted in the background then). While it holds, every other write of the location is refused with 409ROLLING_BACKinside that write's own transaction: a location edit, a resource create, edit or resource delete, a credential create, set, detach, rename, rotation or delete, and a second rollback. A discovery report stores nothing and answers without an error, and an agent's per-resourcecollect_statusand hardware stamps are skipped (the next cycle writes them). The reaper and the orphan stamp of a credential do not wait: one committing first only makes the rollback plan again. Without the lease a busy agent made a rollback lose its write conflict on every retry. A lease left by a process that died lapses on its own, and a transaction under a lapsed lease does not commit; the unlock endpoint lifts one by hand. - Location delete wins. Deleting the location does not wait for the
lease; it deletes it with the location. The rollback then finds its lease
gone and answers 404 having written nothing (
rollback_lock::lost), as it does when its transaction finds the location gone. - Stale plans. The rollback plans outside its transaction; the transaction first checks that the rows it planned against (the location's registered resources with their credential, kind, address and vantage points, and the vantage locations) are still as planned, and otherwise rolls back and plans again. A vantage location deleted while the transaction itself runs is not seen (it writes no location row). A rollback that loses this or a write conflict 5 times answers 409 with nothing changed.
- Lost answer. The transaction's revision ids are chosen before it runs.
If its answer is lost (a transport failure; a query the database refused
ran nothing), the rollback looks for the first of them: found, it
succeeded; absent, it plans again. The lost transaction may still have
been in flight, so if the rollback then fails, or finds nothing left to
write, it looks again: found, it succeeded with that attempt's summary;
otherwise, or when a lookup fails, it answers 500 with
outcome_unknown: true. The gateway answers the same when inventory-svc's answer is lost after the request was sent. The history panel then says the outcome is unclear and to check the history, as it does for a 502 or 504 from a proxy in front of the gateway or a request that failed in the browser. Such a rollback's ids are unknown, so the gateway reconciles no metric state for it: if it did apply, the state of a resource it removed ages out through the stale ceiling, and one it recreated shows no state until its delete's tombstone expires (5 minutes). Check the location's history; running the rollback again is safe either way: applied, it finds nothing to write; not applied, it applies now, and the gateway reconciles what it reports then. - A failed rollback's error carries no ids; the gateway acts on a
rollback's
recreated_idsanddeleted_idsonly when it succeeded.