Usually the parts that would need some optimization, like, say, using the CPU features properly, are things that are shared with a bunch of other people, and so good support for them makes it into the mainline kernel and from there to everybody.
If you think you have an unusual or super new system, you can look for a way to run newer or specially-optimized-for-your-hardware kernel builds. Or you can always get into compiling the kernel or other system components with the right optimizations or go-faster stripes for exactly your machine: Gentoo people like to do that.