Zum Inhalt springen
Produkte

// Benutzung · Konfiguration & Rollback

Config revisions, checkpoints & rollback

This is the reference for how the platform versions its own configuration — Locations and Resources and their settings, as configured through the platform itself. Every such change is journalled as an immutable revision; a user can mark a "safe" checkpoint and later roll a location back to it, write-forward, without ever erasing history.

This is deliberately scoped to platform config, not infrastructure state. Kubernetes rollout history and Proxmox snapshots are a separate system that wraps each backend's own mechanism — they only exist once control does, and are documented separately. Revisions and checkpoints live in the same inventory-svc database, since it is already the authority on this data.

What gets versioned, and what doesn't

Revisions cover user-initiated writes to config entities:

  • Location: create, update, delete
  • Resource: create, update, delete, credential attach/replace

They deliberately exclude:

  • Auto-discovered sub-resources (Proxmox VMs/CTs, k8s nodes/pods). These re-derive from the live source every poll cycle — their status and timestamps churn constantly, and "rolling back" a VM the platform merely observed is meaningless. Versioning them would drown the real config history in noise. That holds for a user's writes to them too: an edit of a discovered resource (its own name and tags, display_name and user_tags), or a credential set on or detached from one, applies but writes no revision , as its delete never did. Discovery keeps the own name and tags, but no rollback restores them. Revisions of discovered resources written by older releases stay in the history, and every rollback skips them.
  • Hardware inventory reports (the Redfish component tree), for the same reason — the agent replaces them wholesale on every walk.

So the history is exactly the set of decisions a human made through the platform, and nothing else.

Rollback — always write-forward

Rolling back to a checkpoint writes the checkpoint's snapshots back as the current state, and records each change as a new revision with operation = "rollback". History is never rewritten: no row is ever edited to say something other than what it said when it was written, so you can always see that a rollback happened, when, by whom, and what it restored. A rollback of a bad rollback is just another rollback. (Old rows can be pruned once an admin opts in — see below — but a surviving row always still means what it meant.)

Rollback is scoped to a location's own settings and its user-managed resources (auto-discovered ones are left alone — they re-derive from the live source). A checkpoint taken while discovered resources had revisions (written by an older release) holds them, and a rollback skips them in either mode: it never restores, recreates or removes a discovered resource, and does not count it in its summary. In older releases a full rollback to such a checkpoint failed while the resource existed, or recreated it as a registered resource once it was gone. It has two modes, chosen per rollback:

Mode Existing entity changed since Entity deleted since Entity created since
safe (default) restored to checkpoint values left absent left in place
full reconcile restored to checkpoint values recreated (same id) removed

Safe is the least-destructive default: it fixes what drifted but never deletes or resurrects anything. Full reconcile makes the location match the checkpoint exactly — use it when you want the location to be the checkpoint, accepting that resources added since will be removed and ones deleted since will come back. A resource it removes takes the resources discovered under it along, as a delete does; a resource it recreates gets its discovered resources back, as new resources, from the agent's first collection of it (for a Proxmox node, after its credential is re-attached; a Kubernetes cluster without one is collected in-cluster).

"Created since" is proven, not assumed: full reconcile removes a registered resource the checkpoint does not know only if the resource was created after the checkpoint was taken (its created_at) and has a revision. Anything else it leaves, and lists in the summary's kept_ids: a resource with no revision at all, or one that existed before the checkpoint without being in it, such as a row from before revision history taken into a checkpoint made before migration 0038 journalled it.

A location's settings, local_collector included

A rollback restores the location's name, kind, region, network config and local_collector flag as the checkpoint holds them. local_collector decides whether the hub's bundled core agent or the location's own agent collects it, so a rollback that changes it moves where collection runs. A checkpoint taken before the flag existed (migration 0014) does not record it; rolling back to one leaves the flag as it is now rather than clearing it.

Changing the flag needs location.edit on all locations, as an edit of it does (see RBAC & per-location access). A rollback by a caller without that grant that would change it is refused with 403 before anything is written; one whose checkpoint holds the value the location has now goes ahead.

Open edit forms go stale

A rollback writes the location's row and the resources it restores in place as an edit does, and moves their tokens where it changes a value: the location's updated_at when one of the location's own fields (name, kind, region, network config, local_collector) differs from the checkpoint, and a restored resource's edited_at when one of its restored fields does. Values are compared as the API and the agent read them, as a save compares them, so a NONE an older row or snapshot holds is the default it stands for (no tags or extra vantages, not public, method GET, no redirects followed, probing enabled), not a change. An edit form opened before a rollback that changed its record is refused with 409 STALE on save, and reloads, rather than writing the values it read over the restored ones; a rollback that puts back what a record already holds moves no token, so no form goes stale over it (the location is still journalled). A recreated resource gets a new edited_at as any create does.

A resource's TLS certificate checking is restored

A rollback restores a resource's allow_invalid_certs flag as the checkpoint holds it, whether it restores the resource in place or recreates it (before, an in-place restore kept the current value). The flag turns off TLS certificate verification for the resource's agent connection or check, so a rollback can turn certificate checking back off: if the resource had it off at the checkpoint and it was turned on since, the rollback turns it off again. A checkpoint taken before the flag existed (migration 0011) does not record it; an in-place restore from one leaves the flag as it is now rather than reading it as off.

A rollback that would turn the checks off for a resource that keeps a credential needs resource.allow_invalid_certs at the location: without it the whole rollback is refused with 403 before anything is written, as an edit turning them off would be (see RBAC & per-location access). Detach the credential first, or have someone holding that permission roll back.

Credentials are never rolled back

Snapshots never contain credential secrets (only whether a credential is attached), so rollback restores config fields only. A resource recreated by a full rollback comes back without its credential and must have one re-attached; an existing resource keeps whatever credential is currently on it. This is intentional — the revision journal is not a secret store.

Vantage points and credentials follow the edit rules

A rollback restores a check's extra vantage points, a resource's kind and address, and keeps an attached credential's host, only as creating or editing the resource would allow. Other restored fields are not re-validated against today's write rules. The whole rollback is planned and checked before anything is written, so a refused rollback changes nothing.

  • Extra vantage points (check_vantages) come back only where the location still exists, only when the checkpoint recorded a check kind (the kind is restored with them), and only on a check that ends up with no credential (a credential is location-scoped). Any other entry is dropped from the restored list, not refused, so a checkpoint stays usable after one of its vantage locations is deleted.
  • The status-page flag (check_public) comes back only when the checkpoint recorded a check kind; on any other kind it is restored as false, so a checkpoint taken while a resource still held a stale one (before migration 0035 cleared it) does not put it back.
  • Every vantage point the rollback adds to a check needs resource.create at that location, as creating a check with it directly does. "Adds" means the check does not hold it right now: a vantage point the same check already has needs nothing, so a check an admin gave a foreign vantage point does not block rolling the rest of the location back. A check a full rollback recreates adds all of its vantage points. One missing permission refuses the whole rollback with 403. The gateway passes the caller's permitted locations to inventory-svc in x-fleeger-rollback-locations, and drops any copy a client sends. Without that header, as on a call that did not come through the gateway, a rollback that would add a vantage point is refused. An older gateway-svc does not strip a client's copy, so the check holds only while gateway-svc is at least as new as inventory-svc.
  • A credential never follows an address to another host. A rollback that would restore an existing resource's address to a different host (another port or scheme counts, so https cannot become http) while a credential is attached is refused with 400, naming the resource. Detach the credential first, as for an edit.
  • A restored kind and address pass the check a create makes. A rollback that would change an existing resource's kind or address, or recreate one, to a pair creating it directly would refuse (a javascript: URL as an http_check's address, say) is refused with 400, naming the resource. A resource whose kind and address the rollback leaves as they are is not checked, so a location holding one stored before this check can still be rolled back.

Who may roll back

A rollback needs the rights its changes need when made directly, at the location rolled back:

Mode Needs at the location
safe resource.edit and location.edit (it restores the location's settings and its resources in place)
full those, plus resource.create and resource.delete (it also recreates and removes resources)

plus resource.create at every vantage point it adds at another location (above). The gateway reads the mode from the request body and fails closed: only an explicit "mode": "safe" gets the safe set. A body without a mode, which inventory-svc runs as safe, still needs the full set, as does one the gateway cannot read. One missing right refuses the rollback with 403, naming it. Taking a checkpoint and lifting a rollback's lock stay location.create.

Retention of history

Config revisions were originally kept forever, and they still are by default. They are, however, covered by the shared retention policy, which carries two knobs read by inventory-svc's reaper. An admin who sets revision_retention_secs expecting it to touch only metrics and dead inventory is wrong — it prunes this journal too, so it is worth knowing exactly what survives.

  • revision_retention_secs — an age ceiling, in seconds. Unset (null) means keep forever, which is both the shipped default and how every policy row written before the retention migration reads back. Nothing is pruned until an admin opts in. Must be positive.
  • revision_min_per_entity — a per-entity floor: the newest N revisions of each (entity_type, entity_id) survive however old they are. Default 20; a configured 0 is clamped up to 1, so an entity can never lose its newest revision — that row is its current config state, and it is what checkpoint creation reads.

A revision is deleted only when both conditions hold: it is older than the ceiling and it falls outside its entity's floor. Age alone would erase the entire history of a rarely-edited resource — the record of how it reached its current state, the only reason the table exists — while a busy entity kept its recent churn. With the floor, "old" only ever removes the surplus.

Checkpoints pin what they point at. Every revision id listed in any checkpoint.revision_ids is exempt from pruning regardless of age or floor. A checkpoint is a user-marked rollback target and a rollback resolves its revisions one by one, so a reaped one would silently vanish from the restore — gutting the checkpoint instead of failing loudly. A checkpoint you keep keeps its revisions. The practical rule: if a point in history matters, mark it.

Pruning runs inside the ordinary retention sweep and is bounded per cycle (a cap on entities examined and on deletions per entity), so the first sweep after an admin opts in does not try to clear years of history in one transaction — the leftovers go next cycle. Each sweep that deleted something records the count in the sweep audit log alongside the inventory/metrics counts.

Nothing here rewrites history: pruning removes whole old rows, it never alters a surviving one, and the write path remains append-only.

Rollback — all or nothing

A rollback writes everything it changes in one transaction: the location's settings, each restored, recreated and removed resource with the resources discovered under it, their hardware rows, and a revision for each, rendered from the row as written. It either commits all of it or changes nothing; there is no half-rolled-back location to finish by running it again.

  • Lease. Before planning, a rollback takes a lease on its location (rollback_lock, migration 0037, keyed by the location): 120 s, renewed before each attempt, deleted by the rollback's own commit, and lifted by the rollback on every other way out: a failure, a plan with nothing to write, a panic or its request dropped (lifted in the background then). While it holds, every other write of the location is refused with 409 ROLLING_BACK inside that write's own transaction: a location edit, a resource create, edit or resource delete, a credential create, set, detach, rename, rotation or delete, and a second rollback. A discovery report stores nothing and answers without an error, and an agent's per-resource collect_status and hardware stamps are skipped (the next cycle writes them). The reaper and the orphan stamp of a credential do not wait: one committing first only makes the rollback plan again. Without the lease a busy agent made a rollback lose its write conflict on every retry. A lease left by a process that died lapses on its own, and a transaction under a lapsed lease does not commit; the unlock endpoint lifts one by hand.
  • Location delete wins. Deleting the location does not wait for the lease; it deletes it with the location. The rollback then finds its lease gone and answers 404 having written nothing (rollback_lock::lost), as it does when its transaction finds the location gone.
  • Stale plans. The rollback plans outside its transaction; the transaction first checks that the rows it planned against (the location's registered resources with their credential, kind, address and vantage points, and the vantage locations) are still as planned, and otherwise rolls back and plans again. A vantage location deleted while the transaction itself runs is not seen (it writes no location row). A rollback that loses this or a write conflict 5 times answers 409 with nothing changed.
  • Lost answer. The transaction's revision ids are chosen before it runs. If its answer is lost (a transport failure; a query the database refused ran nothing), the rollback looks for the first of them: found, it succeeded; absent, it plans again. The lost transaction may still have been in flight, so if the rollback then fails, or finds nothing left to write, it looks again: found, it succeeded with that attempt's summary; otherwise, or when a lookup fails, it answers 500 with outcome_unknown: true. The gateway answers the same when inventory-svc's answer is lost after the request was sent. The history panel then says the outcome is unclear and to check the history, as it does for a 502 or 504 from a proxy in front of the gateway or a request that failed in the browser. Such a rollback's ids are unknown, so the gateway reconciles no metric state for it: if it did apply, the state of a resource it removed ages out through the stale ceiling, and one it recreated shows no state until its delete's tombstone expires (5 minutes). Check the location's history; running the rollback again is safe either way: applied, it finds nothing to write; not applied, it applies now, and the gateway reconciles what it reports then.
  • A failed rollback's error carries no ids; the gateway acts on a rollback's recreated_ids and deleted_ids only when it succeeded.

// Zusammenarbeit

Bereit, Ihre digitale Infrastruktur zu bauen?

Ob individuelle Automatisierung für Ihr Unternehmen oder ein durchgeplantes Netzwerk für Ihr Zuhause — lassen Sie uns über das Projekt sprechen.