What OS are you planning on running? I personally use FreeNAS(TrueNAS) and largely love it. There’s a steep initial learning curve, but it’s not too high.
I run it in a VM inside of esxi so I don’t need a lot of it’s more advanced features. But I do have a jail with deluge in it to handle my torrents. Deluge isn’t up to the task though so I may migrate to a separate VM with something else, or just make a new jail with a different client.