Yeah you’re absolutely right that it does create a tradeoff. My experience has just been that I’d usually consider it a worthwhile tradeoff. In general, the number of people who have to deal with setting meetings is lower than the number of people who attend meetings, especially when you take into account multinational companies.
And when you’re attending a meeting, you only care about knowing what time it has been scheduled for already. It’s in scheduling that you have to work out when is going to be best for your audience, and I’m of the opinion that the distinction between “what time is this in my time zone and their time zone?” and “where does this time sit in relation to their working day?” is net neutral. With one aspect being a strict positive and the other being a net neutral (in my opinion), I think it still wins out and becomes worthwhile.