Those modern high speed USB controllers are not free. They used up available PCIe lanes. The more you add the less PCIe lanes available in the motherboard.
If you have a lot of low speed USB peripherals, just buy some large USB 2.0 hubs so you can reserve the high speed ports for high speed applications such as external disks.