I mean, the question is what’s fair both to trans women and cis women. Competing against competitors with an advantage and being excluded are both unfair. Absolutely eliminating advantage isn’t the standard that minimizes unfairness, it’s a balancing act between competing interests.
I’m not sure sports have found exactly the perfect balance, and it may vary a bit by sport, but it doesn’t seem to be wildly off in favor of trans women.