I originally did, but on further reading I found that dunbar’s number isn’t strictly proven, though it does feel about right.
Also, you would get super tiny towns and the community wouldn’t be diverse enough to support multiple interest groups. For example, assuming a small niche knitting community in a village of 150 would have maybe 3 members who would already know everything about each other, whereas in a town of 5000, there’d be a higher chance of getting at least a mixed bag of people who only know each other through the knitting group.