Yeah, I’ve gotten a couple ‘omg U dumb, ur wrong’ type responses when i mention this. However, it’s not my idea or something - this has been widely discussed.
What will happen to GPT-{n} once LLMs contribute much of the language found online? We find that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear.