Happens to me too. I notice sometimes a post will say 4 comments, but I only see 1. If I refresh the page the other 3 comments will appear. I found if I refresh the post first, then follow the link in my inbox, everything works as expected.
My uneducated guess is that you get a notification someone replied to you but when you follow the link it takes you to the unrefreshed post with missing comments.