This is obviously not intentional on the part of the LLM purveyors. They probably are kicking themselves at the nightmare PR storm in dealing with this, and how it could effectively lead to more actual legislation controlling them. If there’s one thing congressscritters love, it’s writing bad fucking laws to “protect the children.”
However, this paints a clear picture at how few guardrails are on these LLM’s at all, and how they do need legislation to keep them from doing this kind of dumb “move fast, break shit” ethos which almost always results in situations like these where they’re going “oopsies!” about the horrible shit they did to make this happen.
It should be clear that the profit motive and desire to “be the first” out the door with the technology has driven the people making these to make thoughtless short-term decisions of using straight pirated media for their model or not actually checking your data-set and ending up with your image generator being trained on child-porn.
So yeah, we need legislation to wrangle these fuckers into some semblance of doing due diligence in fucking anything they do, and it would be helpful if that legislation wasn’t spurred by dumb fucking “protect the children” horseshit that will result in a bad bill that hurts everybody, including children, like usual.
Also, personal opinion, but as someone who has messed with AI image generation, just based on many weird prompt outputs I’ve gotten, I kind of always suspected they were being trained on abusive material. If you couldn’t see it, honestly, you weren’t paying attention, but once again that’s personal opinion.