I supposed it depends on the country, but as far as I know in most of Europe you can just enter a coffee shop or the local equivalent and ask for a glass of tap water.
Mind you, even though I bought a metal water bottle years ago and almost never buy bottled water nowadays, as you say sometimes it happens that one needs, though its rare and it’s highly unlikely I would be going to a supermarket to buy water.