As pretty as that would be it has its problems. For this to work you need an authentication service. If you use that service for everything then your accounts are linked to this auth provider and you get a problem if that service goes down. You create a single point of failure.
That being said matrix and mastodon support oauth providers. So it is possible to have unified authentication provider for these two. Lemmy doesn’t support this yet.