You can reset your voicemail, which on some carriers will result in callers getting the, “This voicemail box has not been set up yet,” outgoing message (which will not let them leave a message). From the quick search I did, it looks like it’s a pretty carrier-specific process, though, so you’ll need to search “<carrier name> reset voicemail” to get anywhere.