There’s honestly more impactfull things we could collectively do like decarbonizing electricity and transportation, building walkable cities, etc. You definitely don’t have to go vegan to be an environmentalist, and I’m not even sure it’s the highest impact personal change you could do.
There’s some particularly bad vegan foods for the environment, like air freighted berries. But those foods aren’t exclusively vegan, and aren’t hard to avoid. The foods you’d actually consume a lot more of as a vegan like beans and nuts are way better for the environment than beef or chicken.