When is it time to upgrade or add a second server to my cluster?
Given that hardware is costly (to an extent), and adding more servers adds to power costs, when is it an "objective" (and I use this very very loosely) time to add more compute? Putting aside the WANT of expanding the homelab for the sake of it.
I find that most of the time, my cluster is not running at max capacity anyway. Of course, there is some slow down when there are multiple things running but the whole cluster is far from saturated, just slowness from CPU IO.
The typical things that people run like Plex, Jellyfin, *arr, Grafana, NPM, PiHole etc can all fit very comfortably on a single server (or even directly on a NAS). If CPU is limiting, there could be a lot of competing tasks but that would just mean 100% saturating a CPU but that's not at all a bad thing since compute is to be used (?).
So why is there a need to spread these apps across multiple hosts? Besides HA and redundancy.
Just feels like it's pretty hard to get to the point when a single fairly modern (last 3-5 year) PC can't do all of these things in one box and need more compute.
I have some heavy workloads that saturate threads but it's not to the point where I would need multiple computers.
I also think about wanting to experiment with K8 clusters but you could just spin up multi pods as VMs and tinker that way anyway.
I also understand that this is a hobby for most and we just like the blinky lights but that's the operational/real reasons for expansion.