I think it has to do with the fact that we are 98° because we create that heat as a by product of living, and there has to be a way for that created heat to be released and moderated.
The hotter it is outside, the harder it is to keep your body cool and prevent overheating. When it's cold, your body has to make more heat energy to keep itself from freezing or slowing down so much you can't operate.