Yes, theoretically it’s less durable compared to a regular smartphone, but practically, you’d probably replace the device well before it comes to it’s breaking point.
Samsung rates the Fold 4 to 200k folds (and 400k for the Flip 5). I’ve been using my Fold 4 for coming up to an year now, and I’ve observed that I don’t open/close more than 20 times a day. That comes to a total life of 27 years. Even if I say open/close 40 times a day, and say, Samsung lied and the real life is 100k folds, that still works out to 6.8 years, which is well past the time I’d normally hold on to a smartphone. So rest assured, folding the thing isn’t really an issue.