I think the way you applied it showed some nuance, as you talked about it in non-absolute terms. However, when applied broadly it can be harmful.
If we always assume that homophobes and transphobes are queer and trans, then we assume that queer and trans folx are the ones hurting themselves. It’s a small jump from there to say that closted lgbtq people are the source of lgbtq people’s trauma, which is both harmful and false.