It’s a hugely grey area but as far as the courts are concerned if it’s on the internet and it’s not behind a paywall or password then it’s publicly available information.
I could write a script to just visit loads of web pages and scrape the text contents of those pages and drop them into a big huge text file essentially that’s exactly what they did.
If those web pages are human accessible for free then I can’t see how they could be considered anything other than public domain information in which case you explicitly don’t need to ask the permission.