I have no expertise in how ActivityPub works, but my guess is it works something like this:
You make a post on instance A, you can view it on instance B. People all over the fediverse, instances C through Z are voting on your post.
The venn diagram of other instances with which instance A and instance B federate overlap but are not exactly the same. There are instances federated with instance A not visible to instance B, there are instances federated with instance B not visible to instance A.
The vote count for a post is counted by instance A and B based on which instances with which they federate. That means instance A is counting votes from instances that B cannot see, and instance B is counting votes from instances that A cannot see.
Where the Venn diagrams overlap, both instances will agree on the vote count. But the vote counts coming from instances that they don’t both federate with will cause a difference in vote count.