That is true only to some extent. Frances Wright, who admittedly lived later than Washington (1795-1852), was one of the most vocal public abolitionists in the USA to the extent of my knowledge. Specifically, she was a feminist and abolitionist. Both she and Jefferson were Epicureans and knew the sources well, but she drew other, more ethical, conclusions, and supported the fight for abolition.
It is important to keep in mind that she was living later than Jefferson, and thus had access to different sources than he did. However, her example demonstrates that it was not impossible, even back then, to recognize that owning slaves was wrong and unethical. While I agree that it was typical for the elites to do it regardless, I want to emphasize that the sources to recognize that slavery was wrong were already there. Many people simply chose to ignore it.
Thus my stance is that it definitely was a sign of the times that it was widespread, I think the defining feature of the time was that people chose to ignore ethical conclusions. It isn’t just a sign of the time that people kept slaves - it was sign of the time that people chose to keep slaves even though they could’ve recognized that it was wrong and unethical.
I hope my point is understandeable. Just adding my two cents :)