It’s not all about the calories. It’s true butter is dense in calories, but it’s dense in calories your body will convert the bulk of into useful things it needs.
Sugar OTOH, also adds a lot of calories, but your body will only use a small portion of it and convert the rest to fat for later use.
For nearly all of human evolution, sugar was a rarity and your body treated it as such, the wide access we have to sugar nowadays has only been a thing for a tiny blip of time as far as evolution is concerned