If you’re only watching on 1 TV, I don’t think there’s any reason to keep them a separate 4k library. And if your server can handle transcoding easily, there’s still not much reason.
If you have an often-used second (or third, etc) TV with lower resolution and your server doesn’t handle transcoding well, then it’s probably worth keeping them separate.
I’ve also started to disagree with the guide about file size. I don’t think I can tell the difference, and I’m not trying to preserve media for the future. So long as the video has the features I want, I think just about any file size is fine.