I agree about not using it just to disagree with an opinion, but I do think the ability to downvote is very important. It just needs to be clear that it’s supposed to be used to reduce the impact of stuff this is either harmful or just distracts from the conversation.
If I’m in a thread talking about what the best flavor of milkshake is, I will absolutely upvote someone claiming that chocolate is the best even though they are “objectively” wrong. They are however engaging with the conversation. On the other hand, someone who comes in saying that they hate milkshakes and prefer lemonade, while they’re not exactly wrong in having that opinion, it would be worthy of a downvote because they’re in the wrong place for that comment.
And then there’s the bots/people that if they lost the ability to ever talk again, the world would be a better place. Never feel sorry for downvoting them.