I haven’t done it, but as long as Steam itself is isolated – as I expect flatpak Steam is – anything it launches will be too, and you can add arbitrary binaries. AFAIK, that works with Windows binaries in Proton.
Referring to your response to dillekant, I’m not sure how much Steam buys you in terms of security, though, unless you’re buying from Valve. The flatpak might provide some isolation by virtue of being flatpak (though I dunno how many permissions the Steam flatpak is granted…I assume that at bare minimum, it has to grant games access to stuff like your microphone to let VoIP chat work).
Steam, itself as of today, doesn’t provide isolation, at all.
Adding a non-Steam game to Steam lets you launch from Steam, which might be convenient. Maybe use Proton, which has a few compatibility patches.
If I wanted to run an untrusted Windows binary game today on my Linux box, if it needs 3d acceleration, I don’t have a great answer. If it doesn’t, then running it in a Windows VM with qemu is probably what I’d do – I keep a “throwaway” VM for exactly that. It has read access to a shared directory, and write access to a “dropbox” directory. I wouldn’t bring Steam into the picture at all. I don’t want it near my Steam credentials (Steam credentials have been a target of malware in the part) or a big software package like Steam that may-or-may-not have been well-hardened.
It does get network access to my internal network – I haven’t set up an outbound firewall on the bridge, so a hostile binary could get whatever unauthenticated access it could get from my LAN. And it could use my Internet connection, maybe participate in a DDoS of someone or such. But it doesn’t otherwise have access to the system. It isn’t per-app isolation, but if the VM vanished today, it wouldn’t be a problem – there’s nothing sensitive on it. It doesn’t know my name. It can’t talk to my hardware, outside of what’s virtualized. It doesn’t have access to my data. There are no credentials that enter that VM. Unless qemu itself has security holes, software in the thing is limited to the VM.
I have used firejail to sandbox some Linux-native apps, but while it’s a neat hack, and has a lot of handy tools to isolate software, I have a hard time recommending it as a general solution for untrusted binaries. I don’t know how viable it is to use with WINE, which it sounds like is what you want. It has a lot of “default insecure” behavior, where you need to blacklist a program for access to a resource, rather than whitelisting it. From a security standpoint I’d much rather have something more like Android, where firejail starts a new app with no permissions, warns me if it’s trying to use a resource (network, graphical environment certain directories) and asks me if I want to whitelist that access. It requires some technical and security familiarity to use. I think the most-useful thing I’ve used it for is that it mostly can isolate Ren’Py games, cut network access, disk write access, and a number of games (though not all; arbitrary Python libraries can be bundled) can work with a reasonably-generic restrictive firejail renpy profile. It just requires too much fiddling and knowledge to be a general solution for all users, and “default insecure” is trouble, IMHO.
I do wish that there was some kind of reliable, no-fiddling, lighter-weight per-game isolation available for both Windows binaries and Linux binaries out-of-box. Like, that Joe User can use and I could recommend.
I did see something the other day when reading about an unrelated Proxmox issue, talking about Nvidia apparently having some kind of GPU virtualization support. And searching, it looks like AMD has some kind of “multiuser GPU” thing that they’re billing. I don’t know how hardened either’s drivers are, but running VMs with 3d games may have become more practical since last I looked.
EDIT: Hmm, yeah, sounds like QEMU does have some kind of GPU virtualization these days:
Need native performance, but multiple guests per card: Like with PCI passthrough, but using mediated devices to shard a card on the host into multiple devices, then passing those:
You can read more about vGPU at kraxel and Ubuntu GPU mdev evaluation. The sharding of the cards is driver-specific and therefore will differ per manufacturer – Intel, Nvidia, or AMD.
I haven’t looked into that before, though. Dunno what, if any, issues there are.
EDIT2: Okay, I am sorry. I am apparently about four years out of date on Steam. Steam didn’t have any form of isolation, but apparently in late 2020, they added Pressure Vessel, some form of lxc-based per-game containerization.
I don’t know what it isolates, though. I may need to poke more at that. Pretty sure that it doesn’t block network access, and I dunno what state the container gets access to.