I have a similar setup except I use pfSense as my router and pihole for DNS, but I’m sure you can get the same results with your setup. I’m running HAProxy for my reverse proxy and configs for each of my docker containers so any traffic on 443 or 80 gets sent to the container IP on whatever unique port it uses. I then have DNS entries for each URL I want to access the container by, with all of those entries just pointing to HAProxy. Works like a charm.
I have HAProxy running on the pihole itself but there’s no reason you couldn’t just run that in it’s own container. pfSense also let’s you install an HAProxy package to handle it on the router itself. I don’t know if opensense supports packages like that though.
You can even get fancy and do SSL offloading to access everything over HTTPS.