And with the hex/Allen, it’s the small contact points as well as the smaller volume of material that needs to be deformed or removed before slippage can occur, as well as the angle of force on the contact point.
With a hex, the contact point and direction are such that the tool is effectively trying to scrape off material at an angle, and if/when it succeeds even a little bit, it’s now much more prone to fail.
With a Torx, the contact area might still be small, but it’s being applied to the lobe in a more perpendicular direction, so rather than a scraping failure, it’s more of a force that is pushing directly against steel instead of scraping. Not that it can’t fail, but the route to failure is significantly less likely.