You’re misunderstanding. Totp apps require authentication to use them, be it a password or bio-authentication. SMS does not, it just requires the phone number.
You can get the phone number through any number of ways, but it can be done remotely meaning no one ever interacts with you or your phone. Through various methods, they have your phone number transferred to a different phone, and then have the SMS sent directly to them.
Totp apps (typically) have a backup system in place. 1password as an example, uses their servers to host the data. But you can also back that up. The chances of someone gaining unauthorized access to your Totp account comes down to your security, and which service is chosen. 1password again as an example, is fully encrypted, they can’t see your passwords, if you forget your security token, the only solution is to wipe the entire password store and start again.
The difference in security is mountainous. It’s the difference between a single family home, and a bank vault.