That’s a good question! Xenophobia has an evolutionary basis and exists in other species as well (e.g. chimpanzees). Encountering another group of humans (or chimps) might be dangerous. Being cautious is beneficial for survival.
However, when you add racism and are actively hostile to another group - is it still a Phobia? And if not or if it is not fear-motivated, what do you call it?