Look at this in the same light as the 2nd amendment: bearing arms was more compatible with society when the “arms” were mechanically limited in their power/capability. Gun laws have matured to some degree since then, restricting or banning higher powered weaponry available today.
Maybe slander/defamation protections are not agile or comprehensive enough to curtail the proliferation of AI-generated material. It is certainly much easier to malign or impersonate someone now than ever before.
I really don’t think software will ever be successfully restricted by the government, but the hardware that is behind it might end up with some form of firmware-based lockout technology that limits AI capabilities to approved models providing a certificate signed by the hardware maker (after vetting the submission for legally-mandated safety or anti-abuse features).
But the horse has already left the barn. Even the current level of generative AI technology is fully capable of fooling just about anyone, and will never be stopped without advancements in AI detection tools or some very aggressive changes to the law. Here come the historic GPU bans of the late 20’s!