A hacker group in Seattle (GHI) years ago attempted to build secure systems on top of compromised hardware. Although different levels of security could be achieved, the overall outcome was No. You cannot build a fully secured system on top of compromised hardware.
A VPS for this exercise counts as ‘compromised’ hardware.