According to Wikipedia (I know, but it is a protected article), antifa is a loose organization of autonomous groups that use both non-violent and violent means. Based on that last part alone, I would say that is perfectly reasonable to NOT identify as antifa even if one generally agrees with their agenda.
As for vilifying the opposition, that does seem like just the thing the right wing would do to avoid taking responsibility for their own stupidity.