So if your comment hasn’t been sent out out to other instances, they don’t have it.
What’s stopping malicious actors to create an account on the same instance as you and follow you (or your RSS feed) exclusively to pull your data?
Remember “information wants to be free”? That adage works both ways. If people want (or need) real privacy, they need to be equipped with tools that actually guarantee that their communication is only accessible to those intended to. The “ActivityPub” Fediverse is not it. They will be better off by using private Matrix (or XMPP rooms) with actual end-to-end encryption.