Your instance doesn’t pull the upvotes from other instances. That would not be scalable. How would it know when to pull again, to see new upvotes? When would it stop pulling periodically? Never? And you’d have to do this for every single post and comment everywhere.
No, instead ActivityPub uses a push mechanism here. So any new activity is pushed out to the ones that are deemed relevant to know about the activity. Any other instances are unaware.