When I read the Lambda transcripts of that Google employee (priest?) who tried to whistleblow that that Google’s AI was sentient… I mean damn, I read those transcripts, it sounded real as hell.
Stochastic parrot or not, I think a significant part of my own consciousness goes towards predicting the next word in a given context.