I personally would prefer if we all used UTC. My working hours would be 23:00 to 07:00. A Brits working hours would be 09:00 to 17:00, and a New Yorker would work 13:00 to 21:00.
But this does have its own drawbacks. Personally I just think those drawbacks, in the sorts of real-world time-related conversations I’ve had, are less than the drawbacks of dealing with varying time zones.
But yeah, the biggest factor is daylight saving time. Doing away with it is the number one option places that use it should take, regardless of whether one advocates for abolishing time zones or not.