I don’t have a guide at hand but I basically just set a wildcard DNS rewrite on AdGuard Home so that *.my.domain redirects to the IP running my reverse proxy. Since AdGuard is set up at the router level, everything goes through it so the proxy handles everything from there.
I can share specifics after I get home in a few hours if you need a hand, feel free to reach out.