as an expert^1^ I can affirm that singular usage of they/them is absolutely acceptable in polite conversation with people who may be non-binary or with people whom you don’t wish to assume what gender they identify as.
1 - Why am I claiming I am an expert? I’m nonbinary and trans myself; and I moderated /r/genderqueer for a long time; so I do see trends.