It's mostly this, I would say. But in general there's a valid context to use male/female and another valid context to use man/woman or girl/boy or lady/gentleman.
Most people are not going to hold someone speaking English as a second language to task over it. But if you're speaking natively, there's no real excuse not to know when it is right to use the correct term.