There is a lot of room between attraction and repulsion though. A not ideal looking person who is so good in bed, loves you and you get along with? Maybe they are neutral to you looks-wise but hot for reasons other than looks.
Someone whose looks repel you? No, that’s never going to work.