Maybe a noob question, but what's stopping a bot having a user login so it can trawl the rest of the site? Surely that would only take a human somewhere to log it in with a legit device and just let the bot free to trawl the hidden parts of the forum.
It's possible, but not trivial in an automated way if the task of the crawlers is to crawl millions of websites as quickly as possible.
It's not worth it unless a human wants it to specifically target certain websites (which is unlikely in this situation).
Maybe a noob question, but what's stopping a bot having a user login so it can trawl the rest of the site? Surely that would only take a human somewhere to log it in with a legit device and just let the bot free to trawl the hidden parts of the forum.