The phone is trying this on it’s own but you are preferring not to use it, instead preferring 5g networks.
Honestly you should probably just let the phone do its thing. Various bands have various capabilities. 5g is very fast, goes through concrete but doesn’t travel far (even via air). LTE has more bands in lower frequencies than 5G does IIRC, and lower bands travel greater distances and serve more customers. Phones automatically try to use the 5g signal, falling back on 4g, then 3g, 2g… the reason the feature you enabled exists is when there are stronger 4g signals that are getting picked up and there is still a 5g signal within range. In your case I doubt the 5g signal is in range.
Btw, your pixel should be using wifi calling at home, which uses the internet and gets as good of a signal as your home wifi does.