There are people alive today whose grandparents were born into slavery. Given that slavery did not actually end with the civil war, and Jim Crow, and mass incarceration, and the current dismantling of civil rights era laws, there are hundreds of millions of people alive today who are still directly suffering the aftermath.
So no, it is nowhere out of living memory and I am astonished that there are two people in one place so ignorant that they are willing to argue that it is.