Is there anything else going on in the background? Is something else using up your drives or CPU/GPU time?
I’ve been exclusively on Linux for a while, and I know shader caching happens, but I’ve never noticed it happening. AFAIK it should be very quick and seamless for the most part.
Are you perhaps using the Flatpak version of Steam?