It sounds like you’re hanging out with the wrong kind of people if they are asking that question to judge you. I find most people ask that question as its a baseline question on getting to know someone, so hobbies would be a perfectly acceptable response