You hate it precisely because of how you described it. It’s a game where you often end up paying a price, where the reveal is more often a punishment, not a reward. You’ve sensibly learned to dread it.
His half of the game is that always gets is to see your reaction in the moment. If it’s something bad, he gets a big reaction out of you. It’s always a payout for him. Not for you.