Those people are population, potential labor and soldiers.
Logistically, you are right, Russia is down a population size of about 2 million from the start of the war. Worse is that those are mostly going to be trade skill and factory workers.
That being said, I think Putin is seeing those immigrants as more of an unstable element. In that those migrants might give your population ideas... like separation, leaving too, rebellion, or resistance. Which for a repressed Russia, would be a like lighting a match on a powder keg.
As much as Russia needs the extra manpower, I think Russia can't afford the potential of an insurrection.