Password managers holds the key to all my other accounts, where as a random poorly secured site do not. Of course I will have less trust in a password host, a compromised host means I also lose my banking and work account, but if a hacker got my free-manga.net password, well they can enjoy my shitty isekai collection for all I care.
The biggest security issue was always shared password leads to poorly secured site compromising highly secured sites, and thats why unique passwords are important. You might be thinking the change-one-letter password is similar to sharing password, but that is just not how hash works.