Part of it is that people also moved on from Lemmy too. Lemmy is nice, but there also isn’t very much by way of activity on it, which feeds back into itself. No activity means there’s nothing to draw people into it, and not enough to keep them around when they are there.
One of the communities and (non-world) instances I frequented is all but dead these days.