Single GPU with scripts that run before and after the VM is active to unload the GPU driver modules from the kernel.
I think this was my starting point and I had to do just a few small tweaks to get it right for my setup - i.e. unload and reload the precise set of kernel modules that block GPU passthrough on my machine.
At this point from a user experience p.o.v it’s not much different to dual booting, just with a different boot sequence. The main advantage though is that I can have the Windows OS on a small virtual harddrive for ease of backup/clone/restore and have game installs on a dedicated NVME that doesn’t need backing up