Unfortunately, leadership positions tend to be mostly interactions with others by forming working relationships and establishing trust so they see skilled workers who are avoiding work by chatting about non-work stuff to be productive team building even when done to excess. So they consider that not being lazy even if it is when done to excess for that particular position.
Can't talk about sports teams and 'team build' in the same way when working at home in their eyes.