I can imagine a hacky way to anonymise voting would be to have a pool of fake user accounts on your instance. When someone on the instance clicks to up/downvote, a random fake account is used to make the vote instead. This would then kind of work like a vote tumbler and keep the voting anonymous but still work with activity pub.
Maybe activitypub is actually a bit crap and we should all be using something better like nostr though?