This is true, but also it’s implied in technobabble that replicators operate on a lower “molecular” resolution whereas transporters operate on a quantum scale. I rationalize this as a space saving measure; when you’re transporting living organisms, you need perfect precision, and thus a full pattern buffer worth of resolution. This is clearly expensive to store, so much so that it decays over time unless you do something tricky.
Replicators use a lower resolution scan, as you can just reassemble protein molecules into the right shape most of the time. Eddington complains about this issue. (The non-canon TNG technical manual mentions tanks full of protein sludge used for replicators.) Now, is this actually detectable by a human palate? Eh, maybe.
I imagine if you were to beam a plate of non-replicated food though, the full flavor profile wouldn’t be lost. It’s specifically the low resolution of the replicator tech.