You sure do get benefits, at least in the form of being able to select your compile-time options.
Now, does this always result in a tangible performance difference? Absolutely not. It’s more about being able to make decisions like “I don’t want <insert_favorite_software> to be compiled with <insert_least_favorite_options> and I want it all handled by my package manager.”
How much of a benefit this would be to you or anyone else is worth exploring — at least in my opinion — and a large reason why Gentoo has its own unique role in the larger Linux world.