No, you generally put the money out with the bin on the morning it’s picked up (and it’s more like $20 to $50 lol). My neighborhood is quite safe but even here an envelope filled with cash wouldn’t last a week.
Nothing would happen if they didn’t get a tip. There’s a huge turnover in that job and none of the workers ever last more than a couple of months, so even if they had a grudge against you they’d be gone in no time.