Here’s the thing: some distros are aimed at gamers and have some useful gaming stuff in their default setup. But at the end of the day, if you’re comfortable installing stuff on your own, you can game on any Linux distro. It frankly doesn’t matter.
So, yes, if you want to game on Debian because you can avoid bloat, by all means do it.