In your younger years, you have a lot more to look forward to that’s tied to your age: first decade, teen years, driving, getting a job, voting, making your own decisions, second decade, and drinking age. After that, the only things that are still age locked are at 25 when you can rent a car and your insurance goes down, the other decades, and legal age requirements for reaching certain political offices. There just isn’t much else to look forward to at a specific age, so it’s not as important for your brain to remember.
Misremembering people’s names is not related to the age. A chunk of the population is really great at remembering names and another chunk remembers faces better. I’m more of a face person, I will most likely recognize you, but might not remember your name.