In that one respect, they are accidentally right. Everyone has implicit racial biases whether they realize it or not.
To claim one is 100% not racist is basically akin to “I don’t even see color.” It is denial, often self-denial. This is just sociology.
The best way to minimize this reality is to be self-aware of one’s own biases and make conscious effort not to act on those biases.
It is more damaging to pretend you have no implicit racial bias whatsoever and therefore nothing you can do could possibly have racially based motivations. Proud racists use this line of thinkng to attack critical race theory, a valid sociological concept.