The solution to the paradox of tolerance is looking at it as a social contract. If one person tries to use your tolerance to violate your, or others rights, they are no longer protected by the agreement and do not get to benefit from it.
I wish I could find the article about it, but I can’t seem to find it now. Sorry.