Lots of good answers here, but one I haven’t seen is that some people have different value systems. They would be the ones that say “yes, human rights would be nice, but at what cost?”
Typically, as everyone here has pointed out, they value their own well being and comfort. “We can’t end child slave labor because then a KitKat would cost $20.” They might cite economic priorities, national or personal security, religious beliefs, or civic pride (see: China).