The banks own scammy calls don’t help this. I had one a few years ago where they called me about a charge and their system asked for way too much information. I hung up and called the number on the card and they seemed confused when I explained how scammy it was after they confirmed the call was legitimate.
It seems like the best plan for any time your bank calls you is to hang up and call the number on the back of your card.