By “regular” batteries I assume you mean alkaline ones like your standard AA and AAA. Lithium Ion batteries aren’t really more “powerful”, more likely the circuit was simply designed for constant 5V because that is cheap and easy nowadays. Small lithium batteries like the one used here and a simple charging circuit and port are dirt cheap when purchased in bulk.
Not to mention there’s a size difference both for the battery itself and mounting area for it. For something like a AA battery there needs to be a user accessible battery compartment to replace them. It’s cheaper and easier to just not do that.