Maybe - certainly generations always assume anything that younger people do is somehow worse than what they did, and the digital landscape is a part of that. When writing slates became accessible, the old guard complained it was ‘lazy’ because they didn’t have to remember it anymore. Any music popular among teenagers (especially teenage girls) is mocked as foolish, cringe, etc.
But I suspect like most hobbies, it’s mostly the following that determine our assumptions:
history of the media and its primary audience (digital mediums are mostly embraced by youth; video games initially marketed to young children)
accessibility; scarcity associated with prestige (eg: vital labour jobs are not considered ‘real jobs’ if they don’t require a degree)
the kind of people we visibly see enjoying it (we mostly see children, teenagers, and directionless adults as gaming hobbyists)
You’re right, reading is not somehow more or less moral than video games. Many modern games have powerful narrative structure that is more impactful for being an interactive medium. Spec Ops: The Line embraces the players actions as the fundamentals of its message. Gamers are hugely diverse; more than half the US population actually plays games at this point, and platforms are rapidly approaching an almost even gender split. (Women may choose to play less or different games, and hide their identity online, but they still own ~40% of consoles.)
Games as a medium is also extremely broad. I don’t think you could compare games to ‘watching anime’ for example, so much as ‘the concept of watching moving pictures’, because they can range from puzzles on your phone, to narrative epics, to grand strategies, to interactive narratives.
So a better comparison for video games isn’t ‘reading books’ so much as reading in general, and are you reading Reddit, the news, fiction, or classic lit? What does your choice of reading mean?
So for your suggested hobby of ‘reading books’, one might assume any (or all) of the following:
they are intelligent and introspective (or pretentious),
they are educated (or think they’re better than you),
they are patient and deliberate (or boring),
they’d be interesting to discuss ideas with (or irrelevant blatherers).
Assuming everybody who reads is ‘smart’ is as much an assumption as assuming everybody who games is ‘lazy’, and the assumptions you make about the hobby are really assumptions you make about the typical person who chooses it. It may not be a guarantee, but its a common enough pattern.
TLDR: Ultimately? I think books have inflated status because it’s seen as a hobby for thinkers; people picture you reading Agatha Christie (but you could be reading Chuck Tingle, or comic books). Games have deflated status because it’s seen as a hobby for people who consume mindlessly - the people who know what games are capable of are the ones playing them, too.