Don’t correct that. He’s wrong. It’s way more than 46, most multi-term presidents defeated several different contenders.
Edit: I got 63 white men and 1 white woman, not counting pre 12th amendment elections, not counting minor candidates who didn’t win any states electors. There’s a lot more if you include minor candidates, but then one of them would be Cynthia McKinney who is a black woman.