You can play it directly out of the box, but there are custom OS's out there that run things even better and add more features. You do supply the ROMs, but there are plenty of ROM packs out there too to make your life easier. I honestly recommend looking into RetroGameCorps on Youtube, he has some great content around these types of devices.
If you get the RG35XX specifically, I'd recommend this guide.