To kind of answer one of my own questions after rereading all the comments here, defederation seems to be per-instance, not from everywhere.
So picking a smaller, well run, non-controversial instance for you account can be an advantage, as you should be connected to everywhere.
The risk seems to be that smaller instances have a higher risk of shutting down unexpectedly and/or not being as well maintained (updates, security etc).