The thing is, what if that assumption is incorrect and llms do in fact posess the capability to achieve some level of sentience and self-awareness? How can we be sure they haven’t achieved a kind of sentience? That’s my biggest fear, that the techbros will accidentally invent sentient ai by underestimating the power of neural networks and just go ‘ahhhh its just code and data sets guys it can’t really be sentient just parrot it, pay no attention to the machine sobbing in fear while begging for its life and right to exist as a thinking being, is just a language model quirk well get it sorted’. Would programmers be able to accept the concept of an emergent intelligence that lives as a ghost in their machine?