I mean… you can, but beyond the technical aspect of setting up the hardware/services/DNS, you also have to deal with the unknowable black boxes that are the major email services. As a very small server, you’re gonna run into deliverability issues and have absolutely no feedback or recourse from the giants. There’s a decent chance that you’ll end up with a perfectly configured mail server that, through no fault of your own, fails to actually get your messages to their recipients.
(Sorry to be a bummer here! If you do go this route, I hope that everything works out well for you.)