Not only does this phenomenon have a name (Fundamental Attribution Error), OP’s situation is the example case given on the wikipedia page:
In other words, observers tend to overattribute the behaviors of others to their personality (e.g., he is late because he’s selfish) and underattribute them to the situation or context (e.g., he is late because he got stuck in traffic).