Libertarians are morons in their own right but the core philosophy of libertarianism (in the political compass sense) is the opposite of authoritarianism, which is where fascism stems from. A group that doesn’t want centralized government can’t also want a centralized government that enforces sociopolitical ideology.
The more you throw around the word “fascist” to mean “anything right of me that I don’t like” the more the word loses meaning and becomes like the “communist” rallying cry of the boomers.