A Fourth Node for Maintenance, Not Capacity → tima.dev

Why this matters

The September audit found the fleet’s load running backwards from its hardware, and one of the consequences was a maintenance problem. Node-B held twelve guests: identity, documentation, home automation, the domain controller, and most of the observability stack. Patching or rebooting it meant an outage for nearly everything. The fix wasn’t more compute. It was a place to move things while a node is down.

The hardware

The node is a Dell OptiPlex 7040 Micro with an Intel i5-6500T (4 cores, 4 threads, 35W). It runs 24GB of DDR4, the 8GB it shipped with plus a spare 16GB module, a 1TB NVMe for boot and VM storage, and a 2TB SATA drive for bulk storage. Single gigabit NIC. It idles around 43C. The name follows the naming convention: the GR-75 medium transport, the unglamorous ship whose job in the films was moving everyone else to safety.

Bringing it into the cluster

Proxmox 9.2.21 went on first, then the cluster join with pvecm add pointed at an existing node. The cluster went to four votes. Then the fleet integration:

A VLAN-aware vmbr0 carrying VLANs 20, 30, 40, and 50.
Tailscale, with SSH working.
A Beszel agent, with the bulk drive added as an extra filesystem.
The bulk drive formatted ext4 and mounted through fstab.

A few things went sideways, as they should be allowed to in a write-up. parted wasn’t installed, so partitioning used fdisk. A bad sed duplicated the VLAN lines in the interfaces file. And the first cluster-join command was the wrong form. Deferred on purpose: the Wazuh agent, vPro/AMT configuration (it needs someone at the machine during boot), and Discord alerts.

What moved

Four guests left Node-B:

Guest	Role	RAM on Node-D
alliance-dc01	Active Directory domain controller	8GB (was running over its 4GB allocation)
Yavin-IV	BookStack, Gitea, NetBox	4GB (was 8GB)
Home Assistant	Home automation, VLAN 30	2GB
Coruscant	Jocasta/Hermes runtime	8GB

Node-B went from twelve guests to eight and from 45.5GB allocated to 23.5GB. Node-D now sits at 22GB of roughly 23GB usable, which is tight. Anything added later, a backup server included, has to be budgeted against that.

The plan changed on contact. Coruscant was originally headed for Node-A, next to its inference backend. Its workloads turned out to be lightweight and not dependent on GPU locality, so it went to Node-D instead. Node-A stays dedicated to the GPU and AI work, and Node-C is left exactly as it was.

Pre-flight caught the usual migration traps. The Proxmox node name is Optiplex7040 with a lowercase p, and the wrong casing fails. Moving between AMD and Intel hosts fails or crashes a guest set to cpu: host, so those guests go offline or switch to x86-64-v3, which every CPU in the cluster supports. Node-D has no fast-lvm storage, so migrations target local-lvm. And the bridge has to carry every VLAN tag the guests use.

What the fourth vote does and doesn’t do

It’s tempting to credit the extra node with more fault tolerance. It doesn’t add any. Quorum is three of four, so the cluster still survives exactly one node failure, the same as with three nodes. An even vote count also means a 2-2 split leaves nobody with quorum. What the fourth node buys is room: capacity to drain a node, work on it, and bring it back with no guest outage beyond the short offline moves.

The lesson

Placement should follow consequence, not whichever node has a free slot. The September audit asked which node would hurt most to lose. This answers a different question: where do guests go while a node is being worked on. The interesting part wasn’t the hardware. It was changing the plan when the facts changed, and leaving the stable parts alone: Wazuh stayed on Node-C because it hasn’t had a problem, and the second DNS resolver is deferred to the Raspberry Pi cluster project instead of being built twice.