Yes, but the costs of those things are mostly fixed. If, say, 20% of the workforce goes into the office because they enjoy working there, then you pay the full cost of cleaning, lights, toilet paper, paper cups, and heating and AC for the entire building, even though it’s not at capacity.
Source: My company is hybrid, but a handful of people decide to go in every day, including three people from my team.