Focus on DNS for the host machine and it’s port mappings, not the individual containers.
If you’re instead asking “How can I easily map a DNS name to service and port?”, then you want a reverse proxy on your host machine, like nginx (simplest) or Traefik (more complex, but geared towards service discovery and containers).
In the latter scenario you setup a named virtual host for each service that maps back to the service port exposed for your containers. Example: a request for jellyfin.localdomain.com points to the host machine, nginx answers the request and maps the host name in the request, then proxies your session to the container.
It’s copy and paste for the most part once you get the first one going unless you’re dealing with streaming.
If you’re running a flexible platform on your router like OpenWRT, you could also do some port forwarding as a means to achieve the same thing.