“my guy” is a popular gender-neutral set phrase that’s widely used to show exasperation or disbelief. Clapping back with a diminutive adjective almost exclusively for women just makes you sound sexist. This isn’t everyone getting offended, it’s just you being odd.