Someone on Lemmy phrased it in a way that I think gets to the heart of it: With most of the impressive things that LLMs can do, the human reading and interpreting the text is providing a critical piece of the impressive thing.
LLMs are clearly very impressive; I would not say that the disillusionment on discovering what they can’t do should detract from that. But they seem more impressive than they are, partly because humans are so good at filling in meaning and intelligence where there (yet) is none.