Also policing training would be completely unenforcable
That’s where laws would come in. Obviously it would have civil law, not criminal law, but making sure it would be enforceable would have to be part of such laws. For example, forcing model makers to disclose their training dataset in one way or another.