If Youtube's algorithm doesn't know you from Adam it can't recommend things that might be relevant to your interests
Having watch history off doesn't mean having no history at all. I have my watch history filled with only the videos I enjoyed (mostly videos with fit girls) before I turned it off. Youtube only recommends me stuffs based on those (and my likes/subscriptions obviously). I can watch whatever random videos I found while pooping and never have to worry it might screw up yt recommendations.