Do you have a source that those people leaving are the ones that didn’t vote? I would guess that people who consider a measure as drastic as leaving your home country, would vote above average.
Furthermore, I would expect that mainly well-educated people are leaving the US, simply because it’s much easier to get a work permit elsewhere. And high education typically also correlates with higher participation in elections.
I didn’t have much time right now to a deeper research on these stand points. So if I’m wrong, I’d highly appreciate any sources.