I started in 2012, and it wasn’t that difficult. I’d say I do about 30mins of maintenance every other month. It took me a while to work out the config originally, but I wrote a guide afterwards which was really popular for other people doing the same thing (it’s quite out of date now but the principles are the same).
Started out using a raspberry pi (which was also hosting a website at the time) but when I moved house to somewhere with a worse internet connection I migrated to a VPS, so there is a cost but it’s not enormous, maybe £20/month.
Don’t even bother if you can’t use a static IP, because all your email will be bounced if your PTR record for the IP (reverse DNS record) doesn’t match your domain name.
It got a bit more complicated when people started adding extra layers of spam protection like SPF, DKIM and DMARC, but those are mostly set and forget.
Overall, I’d say it’s worth it but only because I find it quite interesting/fun.