Just learn how to navigate properly with a map. You know where you are (hopefully) and you look up where you’re going. Do your own routing, it’s not hard and once you’ve been doing it for awhile you’ll have instinctive routes you use to link different areas. No location permissions required.
If you use navigation all the time you can become like my ex-wife. Lost without it, in her own city. Exercise your brain by finding your way around with landmarks and signage, I have never used navigation apps because I know my way across three provinces, and can navigate the major cities just based on my instinctive mental maps of them.
Edit: I don’t mean a paper map. But being able to use your phone’s map software to find a route without it dictating every turn to you is a valuable skill that is apparently getting lost. What are you going to do if your phone dies, will you be able to go anywhere?