ActivityPub does use cryptographic keys for Actors (“users” in this case) - so even in theory if you were to destroy your instance and then set it up on the same domain and recreate the user, things would be quite broken still… But unfortunately it still does rely on the domain name itself, so I agree.
I think the problem is, without the domain name, there is no way for you to lookup who @russjr08 would be, or where to send data to them. The domain effectively acts as a mailing address (a well suited analogy considering that ActivityPub also uses inboxes/outboxes) so that Instance A always knows that User B can be found on Instance B.
I doubt its an impossible challenge to solve, but probably quite a difficult one I’m sure.