The main advantage of using Linux for your homelab is that you will get a lot more support because most folks here run their homelabs on Linux, BSD, or a combination of both. I run my home lab on a combination of both. Windows Server will probably end up really frustrating you and adding additional needless expense.
For DNS, you can look into a couple of different solutions:
PiHole
Run just Unbound
Run BIND
I personally just run Unbound as it works well for me. Currently, I am thinking about moving to a split-brain DNS and do away with fake .lan internal TLD that I am using, especially now that I have gotten Let’s Encrypt’s DNS-01 challenge to work properly.