You’re right, your contract is with the business. If you don’t want a separate transaction for your server, then just be honest about it.
I do assume you are being honest, and telling your server at the beginning of the meal that you don’t tip, right? Surely you’re not waiting until after they’ve given you the customary service to withhold the customary payment, right? That would certainly be shameful indeed, and undercut your desire for them to receive thriving pay for honest work.