Trial and error, lots of reading ProtonDB, wikis, etc. I only just recently got a decent handle on how to properly use wine prefixes to get mods and things working.
In general, use Steam when you can, then use Heroic for non Steam games. Lutris is very powerful and super useful for games that aren’t installed from a larger distributor, ie from a CD or direct from the devs, but I find the UI can be a bit spartan. Steam and Heroic have fewer features but are way more user friendly.
Good luck. It can definitely be frustrating but remember that you have access to tons of resources and an excellent community if you encounter issues.