True, a system whose only imperative is personal upside maximization with no consideration for anybody else is morally agnostic if you put aside the moral dimensions of having no consideration for anybody else whilst acting for personal upside maximization.
Same as murder being morally agnostic if you put aside the moral dimensions of killing another human being.
Consider the possibility that you’re confusing familiarity, common use in your environment and even normalization of something with it actually being devoid of a moral component: just because people around you got used to act in some way without questioning such way of acting doesn’t mean it’s morally agnostic: after all, slavery too used to be normal.