Taking other people’s creative works to create your own productive work is allowed if you are making a fair use. There’s a very good argument that use such as training a model on a work would be a fair use under the current test; being a transformative use, that replicates practically no actual part of the original piece in the finished work, that (arguably) does not serve as a replacement for that specific piece in the market.
Fair use is the cornerstone of remix art, of fan art, of huge swathes of musical genres. What we are witnessing is the birth of a new technique based on remixing and unfortunately this time around people are convinced that fighting on the side of big copyright is somehow the good thing for artists.