It’s the reverse for me. New clothes feel uncomfortable, sometimes. Then, the more I wear it, they keep getting comfortable.
Maybe, look up tye ingredients of your detergent? Also, consider using treated-soft water for washing clothes. Hard water (water with minerals) can damage the fabric.