A reverse proxy will achieve what you’re looking for, yes. I do precisely what you’re describing. I use “local DNS” on my pihole, with CNAMEs for each service pointing to my server IP address. I’m running Caddy on the server, specifically this (because my services are running in Docker containers): github.com/lucaslorentz/caddy-docker-proxy
As far as a “dashboard” app, you have options there as well. I’m using Heimdall currently, but Homarr also looks pretty nice.