Another thing not mentioned yet is maintenance overhead. These distros operate around the clock, all over the world, with talent from the likes of RH and co. There are far fewer people (who run your mirrors) who know how to maintain a torrent tracker (or similar), and on top of that, I haven’t really seen any good BitTorrent caching methods. Support would need to be added to your package manager of choice.
It also comes down to most client having asymmetric bandwidth, and that most users do not have every package installed and therefore can only distribute a very small amount of the total distro. Those users probably don’t want to be constantly uploading, either. I also can’t imagine torrents are too fun to work with when it comes to distributing constantly changing package manager metadata, too.