I think you're right; it will probably never have particularly wide reach, but it will (and to some extent does have) deep appeal.
What I mean is that people who are attracted to a platform like Lemmy are the kind of people who are likelier to have those niche passions and knowledge on those topics. And they are the kind of people who are also likelier to participate in communities around those things. No, not everyone, and yes there are still communities with a broader appeal and less depth, but I think my point is clear enough. It's just kind of intrinsic to how the platform works and how it is positioned in the broader internet space.