Him dying of an illness or simply old age would remove a good portion of the followers ire (not 100% because I’m sure there would be an interesting portion thinking the cause is fake and someone killed him; and of course some nutjob would try to capitalize it). The problem of a potentially worse successor remains.
The solution is by no means simple and, to do it well, you need a long term plan that includes not only defeating him but also a shock plan for anyone trying to copy him until they return to being a moderate right party.
The solution must include too opening the elections to a more complex system with more than 2 parties and actual alternatives that make people able of voting for something that’s not just “the lesser evil”.