• Maybe a noob question, but what's stopping a bot having a user login so it can trawl the rest of the site? Surely that would only take a human somewhere to log it in with a legit device and just let the bot free to trawl the hidden parts of the forum.

  • It's possible, but not trivial in an automated way if the task of the crawlers is to crawl millions of websites as quickly as possible.
    It's not worth it unless a human wants it to specifically target certain websites (which is unlikely in this situation).

About

Avatar for abr @abr started