If it is the landlord’s primary home then they should not be lumped in. Renting out a room to help pay mortgage on the home you live in is not the problem. It’s the second homes, the third, fourth, tenth, hundredth homes where it is an issues, and I do think we can lump all of those together. They are using our limited housing supply as a portfolio piece, inserting themselves as profit-driven middle-men and making it less attainable for lower income families.
Entities that buy and own homes purely for “investment” at any scale are the problem. For-profit housing should not exist at any level. Want to own a second home and rent it out to cover the costs? Sure, but require that it be a non-profit.