One major downside of hybrid working really is that if you are having a meeting where even a single person is not there, then the entire meeting may as well be a video call. If you are on a video call, then why do you need to be in the office for it?
At my job we work with physical objects, so being in office is a requirement at least part of the time, but if I’m just going to be in meetings for most of the day, there is no way I’m going into the office just to sit on video calls all day.