What do you consider a larger phone? I have an iPhone 14 Pro, it is slightly smaller (more just a different form factor) than my previous device, the Moto Z 4. I have a case, but have always had a case all the way back to my LG that was the size of an iPod. I do not use any other holding device.
If I do not feel comfortable holding and using it in the environment I am at I put it into my pocket for the time being. When we walk our dogs I usually never take it out because one of my dogs is notorious at yanking the leash if we stop for too long (my sister and I trade plant and other nature photos a lot). A lot of times we walk on a dirt path, I am more comfortable with it out there than I am when we walk the dogs in town and we are on an asphalt path.