Thanks for offering your perspective! It’s important that we keep in mind that not everybody who studies computer science becomes a professional programmer, and you’ve offered us good food for thought.
For what it’s worth, pointers are fundamental for Von Neumann machines, which are very common in the computing world; your current machine and the machine serving this page are both Von Neumann. In such machines, memory doesn’t just store data, but also instructions; the machine has an instruction pointer, which is a pointer referencing the currently-executing instruction in memory. So, if one wants to understand how a computer jumps from one instruction to another, then one must somewhat understand pointers.