Currently, the biggest challenge will be storage. Any images or videos uploaded directly to Lemmy (as opposed to linking a 3rd-party site) takes up space on every instance that federates that post. For large instances, this can get overwhelming.
Aside from that, the server specs to host Lemmy, even a large/popular instance, are remarkably low. It can easily be covered by just a handful of users, and not even particularly generous users.