Obviously, you can’t stop people from freely associating (see the First Amendment).
But that’s not what’s important. What’s important is that the current two major political parties have been given a whole bunch of special privileges that make them quasi-official parts of the government, and all those need to be permanently stripped away.