Maybe it’s different in the US and other cultures, but as an atheist I’ve never seen the phrase as a very religious thing. I say “merry Christmas” and “happy holidays” indistinctly and I’ve never seen anyone offended by the use of either, independtly of their faith (or lack thereof).
I say “merry Christmas” on the actual Christmas day though.