There is almost certainly a binary version of gcc in Gentoo. I ran Gentoo for 20 odd years and also generally insisted on compiling everything. I recall gcc going from v3 to 4. My laptop ran for over a week on a glass table with a prop to keep the fan vent unobstructed.
I probably should have learned back then that I didn’t really understand exactly how the toolchain worked and how to get from ebuilds to binary code really works. I’m a sysadmin and not a programmer.
With hindsight, I suggest that you pick your fights with care. Use the bin versions of entire packages where available and enjoy the flexibility of USE when it will make a difference.
gcc is not the biggest lump you will compile but it does take a while. It was rather slower 20 years ago.