They have to be listening all the time if you have voice activation. The mic always needs to be open so it knows when you say “hey siri” or “hey google”. How would it know you said that if it didnt already listen to every word. The question is if that stays local on the device.