Service workers put up with shit 10x worse than this every day. Odds are, he was probably just having an off-moment or you distracted him while he was doing a different task (neither of which are your fault). Doesn’t sound like he was judging you, or like you should he judging him for his reaction