I’m not denying that ace spectrum people have relationships and settle down as well. But OP is asking about the normative (read allosexual) experience and explicitly mentions physical attraction.
The vast majority of relationships will involve physical attraction and sex. It’s highly unusual for that to not be the case for allosexual people. That’s not a value judgement—if a minority of allo people find something else works for them, then that’s great. But if OP is asking if this is normal, then no it’s not. Even ‘less attractive’ people, as OP put it, find people they’re attracted to enough to enjoy a lifetime of intimacy and sex with.
Overcoming a lack of physical attraction is a pretty big barrier and I can’t see most people overcoming that barrier just to ‘settle down.’ Not being your physical ideal is one thing, most of us settle down with people who don’t look like models or actors, but finding someone physically unattractive is a tough sell in most cases.