Maintaining a node

A node didn’t come back Ready after a reboot/update

What it means: The action stays parked in “rebooting — waiting for the node to come back Ready.” The first boot after a kernel or firmware change can genuinely be slow, which alone can look like a stall. Buddy only marks the action complete, and returns the node to service, once the node’s agent has reconnected and Kubernetes reports it Ready. It never returns a node to service that hasn’t actually come back.

Fix: Give it a minute. If it stays stuck longer than that, confirm the node has power and network, then check its own log:

sudo journalctl -u buddy-agent

If that didn’t work: If the node is reachable but Kubernetes still won’t call it Ready, see A node’s Kubernetes health shows down, with a reason.

The action failed and the node is cordoned

What it means: This is deliberate, not a bug: a failed maintenance action leaves the node cordoned on purpose, so it’s recoverable rather than left silently half-drained. The Maintenance screen shows the actual error.

Fix: Fix whatever the error names, then re-run the same action; it picks up cleanly rather than duplicating work. If the node is actually healthy and you just want it back in rotation, the Maintenance screen’s next run uncordons it for you.

If that didn’t work: If re-running the action fails the same way again, the error text is the specific thing to chase: it names the real cause, not a generic failure.

The drain is stuck

What it means: Moving a node’s workloads elsewhere before maintaining it can be blocked by a pod that genuinely can’t reschedule: most often one bound to node-local storage that lives on this same node (the screen warns you up front when the node holds a pool), or a policy that limits how many copies of an app can be down at once.

Fix: The drain has its own timeout, so it fails cleanly — node left cordoned — rather than hanging forever. Scale down or reschedule whatever’s blocking it, then retry; or accept that the node will only be drained of what genuinely can move.

If that didn’t work: If a pod keeps blocking the drain because it’s pinned to storage on that exact node, moving that app to shared storage first (see Prepare a disk and create a storage) is the only way to let it move with future maintenance.

A halted node won’t come back

What it means: A Raspberry Pi that’s been shut down cannot be woken back up over the network. There’s no remote power-on for it. The action parks itself in an awaiting-power state on purpose.

Fix: Physically restore power to the machine. Once it boots and its agent reconnects, Buddy uncordons it automatically. There’s nothing further to click.

Once you confirm a shutdown, restoring power is the only way to bring the node back. There is no remote power-on.

If that didn’t work: If the node doesn’t reconnect even after power is restored, treat it as A node shows as offline / NotReady from here.

“Update firmware” is missing

What it means: This action only appears at all on a Raspberry Pi.

Fix: Not applicable on other hardware. There’s nothing to fix. On a Pi where you’d expect it and don’t see it, confirm the node is actually recognised as a Raspberry Pi on its Overview screen.

If that didn’t work: After a firmware update, Buddy re-verifies that the node’s boot order survived the update unchanged: NVMe-first if it’s mid-migration to SSD, microSD-first if it’s already retired that card. If the update reverted the boot order, the node is left cordoned rather than returned to service booting from the wrong device. Check the firmware and boot order, then re-run.

The panel went down while maintaining the master

What it means: This is expected. The master runs both the cluster’s control plane and the panel itself, so maintaining it briefly takes both offline, though your workers and their workloads keep running the whole time.

Fix: Nothing to do: the panel reconnects on its own once the master is back, and picks up the action’s current state from the master rather than losing track of it.

If you’re maintaining several nodes in one session, do the master last; otherwise you’ll hit this short outage partway through and have to wait it out before continuing.

If that didn’t work: If the panel doesn’t come back after a reasonable wait, confirm the master itself actually finished rebooting (physical access, or its console) rather than assuming the panel is at fault.

An Activity row for a reboot or shutdown looks wrong

What it means: On an older build, a reboot or shutdown’s entry in Activity could be recorded succeeded a fraction of a second after it was dispatched — before the node could plausibly have gone down — or show failed / “orphaned: buddy-server restarted before this task completed” on a node that plainly rebooted fine. This was a reporting defect, not a failed reboot: the underlying system commands only schedule the shutdown and return almost immediately, so the old code reported success before the node had actually gone anywhere.

Fix: Nothing to fix. This is fixed as of 2026-07-22. The entry now stays running for the whole outage: a reboot’s entry only turns succeeded once the node actually comes back; a shutdown’s entry turns succeeded once it’s confirmed the node stayed down, since staying gone is the whole point of a shutdown.

If that didn’t work: If you’re seeing either symptom on a build from before that date, trust the node’s own actual state — the Nodes list, or journalctl run directly on the node — over that stale entry.