There are cameras and physical buttons that you might want to use on the phone, and the fold might make it nicer to hold in one orientation. Also, the limitation exists because splitting the apps across the shorter side can make very awkward layouts, at least on small phone screens - no such problem with a square. I see no reason not to have the option of both layouts no matter its physical orientation.
I’ve never used a foldable though, so take my opinion with a grain of salt.