Because they can’t see what you’re downloading via direct download especially if it’s over HTTPS. On the other hand, it’s trivial to see everyone’s IP if they download the same torrent you are, then they just send letters to ISPs owning those IPs who then pass it on to you.