but would it be okay to ask, “where did your ancestors come from,”
I’d suggest it would be best if someone’s racial background wasn’t made to be an important part of the conversation at all.
At least not unless it happens to have some relevance like in relation to places they have personally experienced or languages they speak or something like that.
Where a person’s grandparents came from isn’t (or shouldn’t be) a big deal compared to most other things about that person.