Talking to people and examining writing will usually drop references to a couple of other places to explore, or to unanswered questions that are worth looking into. Even if they seem minor, these almost inevitably lead to putting together pieces of the larger story, regardless of which pieces you start with. I don’t specifically remember what whistling guy talks about, but it sounds like that’s the only potential lead you’ve found so far. It’s certainly possible to make progress without ever talking to him, via all kinds of things that can be independently stumbled on, but if you haven’t found anything else I bet revisiting his dialogue will give you an idea on where to search next.
(Okay, I checked the wiki and can confirm that, while Esker is not the richest source of new options in the game, his dialogue does include instructions that lead to new threads for you to pull on)