A really nice budget option is an old Lenovo or HP mini PC. These days they make thin client style machines that are absolutely tiny, use about as much power as a small laptop, and still have decent spec.
Storage wise, there’s room to fit a 2.5" drive inside, and newer ones have NVME slots. You can buy them real cheap from a refurb supplier as businesses are offloading them all the time.
In the same vein, a HP, Lenovo or Dell small form factor tower PC will up your power consumption a little, but give you room for a couple of 3.5" drives as well as an SSD. That’s enough to look at putting in a 12TB mirrored RAID for some serious storage. You’ve also got low profile PCI slots, so you can fit a GPU for faster re-encoding in Jellyfin.