There are still a number of clock sync issues with the Zen4 chips. I’ve had issues on 6.4/5/6 with similar issues that I’ve been able to somewhat mitigate by getting my amd_pstate settings to stop competing with other power tuning tools. Turn off EVERYTHING you have running dealing with cpufreq management, and just let the kernel amd_pstate do it’s thing. See if that helps.