There have been multiple accounts created with the sole purpose of posting advertisement posts or replies containing unsolicited advertising.

Accounts which solely post advertisements, or persistently post them may be terminated.

VMs or containers?

I’m thinking about starting a self hosting setup, and my first thought was to install k8s (k3s probably) and containerise everything.

But I see most people on here seem to recommend virtualizing everything with proxmox.

What are the benefits of using VMs/proxmox over containers/k8s?

Or really I’m more interested in the reverse, are there reasons not to just run everything with k8s as the base layer? Since it’s more relevant to my actual job, I’d lean towards ramping up on k8s unless there’s a compelling reason not to.

stefan ,

Why not do both? I run proxmox on my physical hardware, then have guest VMs within proxmox that run k8s.

Advantages of proxmox:

  • Proxmox makes it easy to spin up VMs for non self host purposes (say I want to play with NixOS)
  • Proxmox snapshots make migrations and configuration changes a bit safer (I recently messed up a postgres 15 migration and was able to roll back in a button press)

You can then just run docker images through Proxmox, but I like k8s (specifically k3s) because:

Advantages of k8s:

  • Certmanager means your HTTP services automatically get assigned TLS certs essentially for free (once you’ve set up cert manager for the first time, anyway)
  • I find k8s’ YML-based configuration easier to track and manage. I can spin my containers up fresh just from my config, without worrying about stray environment settings I might not have backed up.
  • k8s makes it easy for me to reason about which services are exposed internally to each other, and which are exposed on the host outside of my k8s cluster.
  • k8s services get persistent DNS and IPs within the cluster, so configuring nodes to talk to each other is very easy.

And yeah, this way I get to learn two technologies rather than one 😁

ChojinDSL ,
@ChojinDSL@discuss.tchncs.de avatar

Containers, unless you have a specific need for a VM.

With a VM you have to reserve resources exclusively. If you give a VM 2gb of ram, then that’s 2gb of ram that you can’t use for other things, even if the guest OS is using less.

With Containers, you only need as many resources as the process inside the container requires at the time.

Midou , (edited )

If it's relevant to your actual job, learning to use k8s will benefit you more. Generally i'd prefer to keep the bare metal OS as clean as possible to avoid breaking anything during upgrades and such, and keep the containers and normal running apps on separate VMs that can communicate with eachothers, k8s is mostly good if you got a lot of servers and want to manage them all at once through a single "orchestrator". But for self hosting stuff in your home it's kinda overkill. But it still can be used to manage things up. So imo go for k8s since it can be used in homeservers, it's just that it's kinda like using a nuclear bomb to kill a wasp.

csm10495 ,
@csm10495@sh.itjust.works avatar

Unless you have multiple systems, I don’t think k8s will yield much benefit over plain docker.

Hizeh ,

I think it depends on your scale. If homelab stuff docker is awesome IMO.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • random
  • [email protected]
  • lifeLocal
  • goranko
  • All magazines
    •