iOS is probably the worst choice if you want to sideload apps. I would recommend sticking with Android and run your Samsung into the ground (maybe flash a custom ROM?) before getting something like a Fairphone.
Besides, what do you want to be “passionate” about in mobile phones? To me, they’re just tools for browsing the web, playing games, and staying in contact with friends and family while on the go; anything extra is superficial.