If you have a signature you can also sign the contents, so you wouldn’t need to download the content. But AFAIK ActivityPub has no mechanism for including signatures in objects as it is right now. There’s only HTTP signatures, which aren’t on the object itself.