It can cause some damage, in that the tissues inside the breast can lengthen and lose their firmness. It doesn’t damage function in any way but it could be considered premature aging. There’s that photo series of the white woman with the African tribeswomen and they’re comparing their breasts,
because the African women were so interested in how her breasts were a different shape than theirs. If you’ve seen pictures of people from cultures who don’t wear clothing that supports breasts, you can see the difference in shape that constantly fighting against gravity makes.