You are completely ignoring the externalities of competing at the Olympic level. Trans people are going through so many societal, cultural, interpersonal, and internal conflicts that focusing on training at an Olympic level is going to be more difficult. You can be the most physically gifted athlete in the world, but if your head is not in the right space to train 12 hours a day while still going about your normal life then you aren’t going to be able to compete at that level. Hell, look at how far Tiger Woods’s performances fell after his public disgrace. Yes, some of that was drugs and some was from an injury, but a lot of sports is mental.
Also, you are ignoring that while the Olympic committee might allow it, do you think most countries in the world are so open about trans people?
Also also, do you remember the controversy around the East German women’s team from the 70s? How everyone suspected some were men? There might have been trans people winning medals, just maybe not openly.