I’ve got a site that’s exceeded its bandwidth for the month for the first time since it was uploaded three or four years back. No new content of any weight has been added.
IIUC, the access logs indicate that something called MJ12bot has visited the site - wait for it - 355,000 times in the last few weeks. Seems to be something to do with Mozilla?
Any light that can be shined, please do…
MJ12bot adheres to the robots.txt standard. If you want the bot to prevent website from being crawled then add the following text to your robots.txt:
User-agent: MJ12bot
Disallow: /
From here: https://mj12bot.com/
Cheers Steve. I’ll add. I’d be interested if anyone can take a guess as to why it’s visited so many bloody times!
I would send them an invoice and a legal letter. It’s ridiculous.
it’s just one of Majestic’s bots. Nothing malicious. They are a rather respected SEO company, and have bots constantly crawling the internet. This one in particular is apparently building a new database of backlinks, meaning they have followed other websites links back to you…
OK that’s fine, but 355,000 times in a few weeks?
Wow! I’ve never heard of that! I know years ago malicious scraper bots were “disguising” themselves as legitimate bots, and this might be happening here. You can contact majestic through the mj12bot.com site as that sort of crawling should not be happening.
Ignore. Pls delete