When you state something that is in line with the statements of terrible people it is prudent to denounce those people or otherwise say you are unaffiliated with their overall platform.
Yes, people shouldn’t assume that you are saying more than you are, but it is still prudent to understand that people will make those assumptions. It shows self awareness.
Normally, I would agree with you. But conservatives have been getting more fascist and have been making more calls for violence over the past decade. It’s a good idea to separate yourself from them if you don’t agree with those points.