Milk and fruit are both only produced for a limited time.
By each individual plant, sure. But for diverse farming, you can easily get a permanent rotation of fruits going.
You’re also completely ignoring the most importat fact - that milk is produced to feed newborns and fruit is produced to attract (and by extension feed) literally whichever species is around.