I remember having this issue, albeit not as workflow-disrupting as you’ve had it so I never bothered actively searching for a solution. I can confirm however that there is definitely a solution; at some point in my migrations to different distros, the problem stopped showing up, and right now I actually can’t reproduce the problem - labels are displayed correctly for me, both on my existing drives and new partitions that I create.
After some digging (and a sleep-deprived me smiting my boot partition during my investigation and having to fix that, LMAO - I never thought it’d happen, but Guix saved me from a reinstall), I found that LUKS1 has some kind of limitation that makes Dolphin not display the label as expected according to this and my own testing, which may be your problem if your drives are still using LUKS1. The Arch Wiki has a section on how to convert to LUKS2 here.
At the very least, if this is not the solution for you, I can say with certainty that it is fixable!