Yes, as you heat something up to “red hot”, the glow shifts from infrared to being partly in visible red frequencies. This is why a blacksmith can use the color of a piece of hot iron to tell how hot it is.
(This isn’t the only way hot things make light, though — for instance, flames can glow with odd colors like green or blue due to specific chemicals burning.)