The problem is atmosphere and the water vapor in it. Drop the temp too much and the water condenses out of the air on to the electronics. That can cause ice which causes extreme mechanical stress as it form and expands. Also you can get high thermal gradient across components. All of this causes mechanical stress and things crack.
Really the limits are more about packaging / jointing material science rather that the electronics. For the electronics themselves, the limit is absolute zero when the electrons stop moving.