Normally I’d agree with everything you said, but the fact that they’re struggling with the impeachment process tells me they do need something more than just saying they don’t like him.
I paid attention to the legal side of things during Trump’s impeachments, so I’m quite sure “high crimes and misdemeanors” is open-ended and not limited to violating laws passed by Congress. It more or less means they can just make up a crime to fit the circumstances.
Why then are they struggling to find an accusation to charge him with? If you have another explanation, I’d love to hear it.