People who listen to records also generally are listening in a better listening environment. If you can get a CD mastered for a great listening environment and listen to in a great listening environment it would be better than a record could ever be - but you can’t get a CD mastered like that and even if you could most people are not listening in a great environment and so the CD will sound worse than one mastered as they are.
You are right that the main difference is that vinyl records are mastered for use in a specific environment designed for the best audio experience and CDs are generally mastered for a wide variety of listening environments that include terrible acoustics. But any well mastered CD will sound better on the same level of hardware compared to vinyl from a technical perspective and the supposed superiority of vinyl tends to come from the imperfections analogue playback, including noise from dust, is really a preference thing.
Not to mention a ton of modern record players are digital, so vinyl is just one physical step between digital mastering and digital playback.