WTF, the activity button is also on comments. So you can literally see a full list of users who upvoted or downvoted you.
I’m guessing it was done for transparency, but it just seems ripe for abuse. People can take things like downvotes way too seriously. I’ve seen unhinged users on Reddit go harassing a commenter who disagreed with them.