The way I understand the word, it’s more of a “fling”, or a short-term relation, as opposed to something long-lasting or structured. Which, for example, can be fully okay and include no shadiness if it’s communicated with the partner or partners.
I guess words change meaning over time, and since the default has been monogamy for so long, the word is still stigmatized and associated with cheating since it means something short?
As an example, single people can often be heard to have had vacation affairs, which wouldn’t imply cheating since they are single. But the word is used in that context too.