using proxmox without the web gui is possible but it is expected to be accessed via webgui or term/ssh and the VMs on there accessed via ssh,rdp,vnc,novnc,etc.
Due to it fundamentally being an advanced qemu/kvm/virtio frontend you can use virtmanager to access the VM from a desktop. In this instance try something like i3, awesomewm, bspwm or even xfce (it can be very light weight).