Any censorship always boils down to there being somebody who defines what “Far”-anything is.
Judging by the political speech of the “mainstream” in lots of countries (you know the ones: members of parties which have alternated in power, as duopoly with one other “mainstream” party, for more than half a century, with corruption cases blowing up left and right or just a subverted Judiciary and Press), “far” is anybody who says “we should have a true Democracy were all votes count the same, Political decision making is transparent and the Judiciary and Press Pillars of Democracy are Independent and work”.
I frankly don’t really know if or were a line on speech should be drawn, but I am absolutelly certain the whole thing will be subverted to silence way more than merely nazis. I mean, all it takes is to look at what the “terrorist” label and even anti-terror legislation are used for nowadays (for example members of the Greenparty in the UK were under surveilance approved through anti-terror legislation).
All this to say that your take on this seems dangerously simplistic in light of the history of abuse when it comes to limiting speech.