MJ12bot: What it is and Why it Shows Up in Every Server Log
Key takeaways
|
MJ12bot is the crawler people notice first. Not because it's the biggest, but because it's been around longer than almost any other commercial crawler and it never seems to leave. Pull an access log from a site that's existed for a decade and MJ12bot is in there somewhere, month after month, page after page.
That persistence is by design. Understanding why tells you whether to block it.
What is MJ12bot?
MJ12bot is the web crawler operated by Majestic, a backlink analysis company. It builds the link index behind Majestic's Trust Flow and Citation Flow metrics and its Site Explorer product. The name comes from Majestic-12, the distributed search project Majestic grew out of. It identifies as MJ12bot in the user agent string. |
Majestic occupies the same market as Ahrefs, Semrush, and Moz. It sells subscriptions to marketers who want to see link graphs, and MJ12bot is how it maintains the map. Majestic's index is one of the oldest and, by its own count, one of the largest by total link count, which is partly why the crawler is so relentless. It's been accumulating for more than twenty years.
Trust Flow and Citation Flow are Majestic's answers to Domain Authority. Citation Flow measures link quantity. Trust Flow measures link quality by proximity to a seed set of trusted sites. Neither is a Google metric, and Google doesn't use them, but they show up in agency reports and link-building proposals constantly.
Why is MJ12bot in so many server logs?
Two reasons. Majestic's crawl is distributed across volunteer machines running the Majestic-12 software, so requests arrive from thousands of IPs rather than a single verified range. And Majestic's index strategy emphasizes breadth and history, so MJ12bot recrawls pages other crawlers have long since deprioritized. Together that makes it unusually visible and unusually hard to filter by IP. |
The distributed model is the unusual part. Most commercial crawlers, AhrefsBot included, run from infrastructure the operator controls and publish IP ranges you can verify. MJ12bot runs on participants' hardware. Majestic documents this openly, and it means you can't verify a request by IP the way you can with AhrefsBot or Googlebot. You have to trust the user agent string.
That also means IP-based blocking doesn't work. If you want to stop MJ12bot, robots.txt is the tool, and a user-agent WAF rule is the fallback.
Does MJ12bot respect robots.txt?
Yes. Majestic documents that MJ12bot honors robots.txt Disallow and Crawl-delay directives. Because the crawl is distributed, individual nodes may take longer to pick up a new robots.txt than a centralized crawler would, so expect a lag of days rather than hours before requests stop entirely. |
In practice MJ12bot does stop. It just stops unevenly. If you add a Disallow and still see a few requests a week later, that's the distributed design, not defiance. Give it two weeks before you escalate to a WAF rule.
This is nothing like Bytespider, which won't stop no matter how long you wait. If you're already maintaining a robots.txt for the allow side, the robots.txt guide for AI crawlers shows how the groups fit together, and the server log analysis guide is where you'll confirm the requests actually tapered off.
Should you block MJ12bot?
Yes, unless you or your agency use Majestic. Trust Flow and Citation Flow are computed from inbound links discovered on the linking sites, so blocking MJ12bot from your own domain doesn't lower your scores. It just stops Majestic from indexing your pages and outbound links, which benefits Majestic's customers, not you. |
Same test as every SEO-data crawler in the bad bots hub. Documented, yes. Obeys robots.txt, yes. Returns value to you? Only if you're a Majestic customer.
Majestic is the least common of the four major link tools among the B2B marketing teams I work with. Most run Semrush or Ahrefs, occasionally Moz. So for most of my clients, MJ12bot's crawl is pure cost. It's on my block list for mqlmagnet.com and it's the crawler I'd block first if I could only block one, purely on request volume.
If your agency reports Trust Flow to you, ask them before you block. They may be using Majestic's own crawl of your site for the page-level detail.
How do you block MJ12bot?
Add User-agent: MJ12bot followed by Disallow: / as a named block in robots.txt, separate from your User-agent: * group. Publish and verify. Allow up to two weeks for the distributed crawl to fully stop. If requests persist beyond that, add a user-agent match rule in Cloudflare WAF. IP blocking is not effective for MJ12bot. |
User-agent: MJ12bot
Disallow: /
Stack it with DotBot, PetalBot, and the rest if you're blocking several. The Wix walkthrough has the paste-ready version. The Cloudflare guide covers the WAF fallback if you need it.
Does MJ12bot affect AI visibility?
No. MJ12bot feeds Majestic's link index only. It has no relationship with Google, Bing, OpenAI, Anthropic, or Perplexity. Blocking it has no effect on rankings, AI Overviews, or generative engine citations. |
MJ12bot doesn't touch the Engine Optimization Matrix. The Authority lever in the matrix is about real link equity and entity signals, not about which third-party tool measures them. Blocking a measurement crawler changes the measurement, not the thing measured.
The crawlers that do move the matrix are in the AI crawlers overview, and the clearest contrast with MJ12bot is GPTBot: similar request volume, opposite value. If you're reviewing your logs and want help sorting the persistent-but-harmless from the persistent-and-costly, book 30 minutes.
About the author
Harold Bell is Founder and CEO of MQL Magnet, a B2B content marketing agency serving enterprise technology brands. He's the creator of the Engine Optimization Matrix (EOM), author of Partner Over Product, and a Forbes Communications Council member.
Frequently asked questions
What is MJ12bot?
MJ12bot is the web crawler operated by Majestic, a backlink intelligence company. It builds the index behind Trust Flow, Citation Flow, and Majestic Site Explorer.
Who runs MJ12bot?
Majestic, based in the UK. The crawl itself is distributed across volunteer machines running Majestic-12 software, so requests come from many IPs.
Does MJ12bot obey robots.txt?
Yes. Majestic documents support for Disallow and Crawl-delay. Because the crawl is distributed, it can take days for all nodes to pick up a change.
Why can't I block MJ12bot by IP?
Because it runs on distributed volunteer hardware, not a fixed IP range. Use robots.txt or a user-agent WAF rule instead.
Should I block MJ12bot?
Yes, unless you or your agency use Majestic. It's one of the highest-volume, lowest-value crawlers for most B2B sites.
Will blocking MJ12bot lower my Trust Flow?
No. Trust Flow and Citation Flow are computed from links pointing to your site, discovered on the linking pages. Blocking MJ12bot from your own domain doesn't change that.
How do I block MJ12bot?
Add User-agent: MJ12bot and Disallow: / as a separate block in robots.txt. Allow up to two weeks for requests to stop.
Does MJ12bot affect Google rankings?
No. It has no connection to Google, Bing, or any AI engine.
Is MJ12bot malicious?
No. It's a documented, robots.txt-compliant commercial crawler. It's low-value for most sites, not malicious.
How is MJ12bot different from AhrefsBot?
AhrefsBot runs from Ahrefs-controlled infrastructure with published IP ranges. MJ12bot runs on distributed volunteer machines. Both honor robots.txt, but only AhrefsBot can be verified by IP.
