I think the people who want Trump removed from the ballot most are establishment Republicans. Not because of the fascism, but because of the populism and repeated lost elections. They won’t challenge him in public because they might get primaried (or assassinated).