I’m NOT a native English speaker by far, but I am a translator and a linguist (according to my BA degree, at least) and have spent 2.5 years teaching people English (with great results, too). Hopefully you’ll find my experience relevant to yours and comforting.
My native language is Russian (thanks for the alphabet) and I’ve never been outside the country. I learned English mostly playing Space Station 13, which is an online role-playing game (if you let it be) where you have to read a lot (some of it even outside the game, i.e. guides and forums to get a good grasp of some mechanics) and write a lot (to communicate with other players); I mostly opted for translating single words or, sometimes, short phrases that I couldn’t understand, and did it both ways, i.e. to English and to Russian. On top of that, I spent a lot of time talking to people on the Internet, both on forums of various kind and through private/direct messages.
So I read a lot, admittedly, even if it wasn’t fiction. I also watched a lot of YouTube and other media, but I can’t say it did much for my vocabulary.
As expected, I saw more progress when I was not that good with the language, because I had to face a lot of the frequent words, phrases, and constructions. At some point I just stopped encountering much new.
But that was until I picked up fiction in English. I can’t remember how often I had to look a word up, but I can definitely say that I had to do that more with Lovecraft than Rowling, for example, and never with some others. Hell, sometimes I have to look things up when reading Russian writers.
So, first of all, this is completely normal, especially if it happens with words that you recognize as non-essential for the plot or your understanding of the events. Second, it very much depends on the book and the author.
If I had to guess, you probably experience that with verbs or adjectives, as these usually serve as the means of introducing some variety or profoundness into one’s texts. I would say that, if you can help it, ignore them until they really prevent you from truly grasping the idea of what’s going on or what kind of emotions it’s supposed to invoke. You, however, mention that you are able to infer the meaning of many of the words you look up from context - and I, being a bit of a scientist, say that this is probably the best (if not the only) way to actually learn a word.
I’m saying this because language is a very abstract phenomenon, and so is everything that it is comprised of. In English, for example, we have two separate words for a desk and a table, while both are translated as “Стол” (Stol) to Russian, because the distinction between the two never necessitated a single-world solution to my ancestors, apparently. Whatever people put in dictionaries is an approximation of what multitudes of people that speak the given language think of when they hear or use the word, and that’s very different between languages.
So, I think that the best a person who understands the language as much as you do should just try and consciously resist the urge to look up meanings, for it seems to me that the context can already give you more than enough idea. As a result, I believe you should enjoy the act of reading more and develop a more natural, intuitive understanding of the words you may encounter.
But most importantly, don’t forget to praise yourself on the already-massive achievement you’ve made, and thank yourself for the gift of being able to comprehend the vast amount of literature and other thoughts and ideas that the speakers of the English language have produced across centuries. Good job, good luck!