You said you were out sick for 4 weeks? Do they know why? Maybe you could spin that, say “sorry, I got some scarring in my throat and talking can make it worse, so I really need to only talk about work or I’ll be up all night in pain”. Or something like that.
Or you could try “sorry, I like working with you, and I know I’ve talked in the past, but I need to admit that I have Asperger’s/autism, and trying to make small talk is very stressful for me, so I hope you’re not offended but I feel like I’m going to burn out unless I make some changes like sticking to only work-oriented discussions from now on. Thank you for your understanding.”
Bonus points if you type that up and hand it to people, like you can’t even tell it to then directly.
I know both my answers are “lie in a way that makes you look abnormal” which you may feel like isn’t something you should have to do. Which is true. But you want to minimize them thinking you’re judging them, and they KNOW you are able to talk, so the best way I see is to make them think it’s a problem you have, not about them.