Lemmy has a feature/setting called “Private instance” that I think could be used to achieve this, but I think that got broken at some point because it got tied to turning federation off… not sure what the current state is but may be worth looking into.