It would be if it wasn’t for NVIDIA, as usual. On Intel/AMD, you assign the seats, the displays light up and you’re good to go, pretty much works out of the box, especially on Wayland.
But for NVIDIA yeah maybe a VM is less pain since NVIDIA works well with VFIO.