“I hadn’t realised that you were hurt/you felt that way. And I am truly sorry”
This both acknowledges that you acknowledge how they feel and also takes responsibility for whatever thing that caused hurt. What you were doing originally puts the blame on them for feeling that way, kinda in a passive aggressive way.