Again, how is remembering whatever the New York offset is from your own work hours any different than remembering their time zone? If you have a remote coworker in a different time zone do you not already think things like “they’re not at their desk until 10 so I can’t schedule anything with them before that”?
The inconvenience you’re describing already exists and doesn’t change, you’re just used to the current specifics.