Assuming they’re generally on the same instance ::cough EH cough:: others can just defederate if they want. Those that are harassing and spreading hate speech on mixed servers can be blocked, banned from communities, or the instance in question if egregious enough.