My opinion here is. Sure keep the valid stuff provided the user agreed to it. Have an opt-out though where data is analysed for whatever purpose then deleted. I don't know why they cannot keep data for a day, run analysis and delete on a rolling basis. The benefit of having old data to run improved analysis on is negligible when you're getting as much new data daily as they do.
But, regardless the excerpts it sends when it thinks you might have said the wake word which turn out to be false should be deleted. Do any short analysis for the why right away and delete. Because it really wasn't for the phone/personal assistant.