From the other side, for a lot of women don’t just hear that people do that (though there are many firsthand accounts), they experience it. Even if you don’t mean anything by it, they’ve likely been the victim of enough purposefully suggestive comments that they’re sensitive to it. It’s not really your fault, but it’s not theirs, either.
Out of curiosity, do you have some examples of misconstrued phrases?