iirc they had to remove them after the Fellowship came out due to environmental impact concerns, rebuilt them for Return of the King’s ending, demolished them again, and then rebuilt them for the Hobbit, finally in a place that was accessible enough to maintain as a community but remote enough to match the film’s landscape.