I think we are saying the same thing. Even without zoning, now all that would change is chemical plants and dumps popping up around working class communities. Wealthy people will snap up rezoned land close to their houses. The housing crisis goes way deeper than just zoning laws, to the point they are just a tool of many.
The tldr version: zoning was initially used to keep minorities out of areas and was changed into it’s current form once the majority of money had already pooled in to too few hands.