I lived in Japan as an American for 6 years. There were definitely times I went to enter a place and had the employees politely chuckle while handwaving/headshaking “no”. Sometimes I figured out what my error was, but there are many where I’m not sure why I was ushered out the door.