Yeah this is pretty similar to my experience. My wife supports upgrades because she knows they make a difference to me and she can actually recognize it often, but it’s clear she’d be pretty indifferent if she was making audio decisions just for her.
That said, we’ve spent about 2 years with a nice Yamaha power amp, Elac floorstanders, and SVS sub, full setup around $5k, and she really appreciates it for our focused listening now. Passive listening might as well be out of phone speakers for her, but when we put a record on over Sunday coffee, she always remarks how grateful she is that we invested in the setup.