That profile is very much the early adopters of any new platform or technology. If you described the early users of the internet as a whole, it would be very similar.
I don’t think anything particular is keeping other users away, it’s early. We need evangelist, I suspect that most normies don’t even know the fediverse exists, let alone are considering using it.
We just need to continue to grow the reach of the fediverse, don’t give up if it seems a bit bare and give everyone else a reason to join us