What is DotBot and what Moz does with your crawl data
Key takeaways
|
DotBot has a strange reputation. It's the crawler behind Moz's link index, which means it's indirectly responsible for Domain Authority, a number that marketing teams quote in board decks and agencies put in proposals. And yet it's one of the most commonly blocked crawlers on the web.
Both things are true for the same reason. Domain Authority is a Moz product, and DotBot works for Moz, not for you.
What is DotBot?
DotBot is the web crawler operated by Moz to build its link index. It discovers pages and records the links between them, feeding Moz Link Explorer and the Domain Authority, Page Authority, and Spam Score metrics. It identifies itself as DotBot in the user agent string and links to Moz's crawler documentation. |
Moz sells SEO software, and its best-known asset is Domain Authority, a 1 to 100 score predicting how well a domain will rank based on its link profile. DA isn't a Google metric and Google doesn't use it, but it became the industry's shorthand for site strength years ago and never left.
DotBot is how Moz keeps that score current. It crawls constantly, follows links, and reports back what connects to what. Moz's index is smaller than Ahrefs' or Semrush's, and DotBot's crawl frequency on any given site tends to be lower, but it's still a steady presence in most logs.
What is the difference between DotBot and Rogerbot?
DotBot crawls the whole web to build Moz's link index and visits your site whether you're a Moz customer or not. Rogerbot is Moz Pro's site audit crawler and only crawls a site when its owner has set up a campaign for it. Blocking DotBot affects the link index. Blocking Rogerbot breaks your own Moz Pro audits. |
Moz runs two crawlers and people conflate them constantly. Rogerbot is the one you asked for. DotBot is the one that showed up on its own. If you're a Moz Pro customer, keep Rogerbot allowed or your site crawl reports come back empty. DotBot is a separate decision.
Semrush has the same split. SemrushBot crawls for the backlink index, and SemrushBot-SI runs Site Audit for project owners. Ahrefs consolidates into one AhrefsBot. Knowing which crawler does which job is the difference between a clean block list and one that quietly breaks your own tools.
Does DotBot respect robots.txt?
Yes. Moz documents that DotBot honors robots.txt Disallow directives and the Crawl-delay directive. A named Disallow block is enough to stop it. No network-edge enforcement is needed. |
DotBot is well behaved in my experience. It stops when told and it throttles when asked. That puts it in the same category as AhrefsBot, SemrushBot, and PetalBot, and a very different category from Bytespider.
Should you block DotBot?
Block DotBot if nobody on your team or at your agency uses Moz Link Explorer. If you do use Moz, keep it allowed so your own domain's data stays complete. Blocking DotBot does not lower your Domain Authority, because DA is calculated from inbound links discovered on the linking sites, not from crawling yours. |
The Domain Authority point deserves emphasis because it's the most common reason people hesitate. Your DA is a function of who links to you. Moz finds those links by crawling the sites that contain them. Blocking DotBot from your own domain stops Moz from seeing your outbound links and page content. It doesn't stop Moz from seeing that TechCrunch linked to you.
So the real question is the one I ask about every SEO crawler in the bad bots hub. Who's the customer? If you run content gap analysis or backlink strategy work in Moz, keep it. If your stack is Semrush and Ahrefs, DotBot's crawl benefits Moz customers who aren't you, and you can block it without losing a thing.
I block it on mqlmagnet.com. I don't use Moz, and Moz's index is the smallest of the three major link databases, so the marginal competitive intelligence it provides to anyone else is limited anyway.
How do you block DotBot?
Add User-agent: DotBot followed by Disallow: / as a named block in robots.txt, separate from your User-agent: * group. Publish, verify at yourdomain.com/robots.txt, and check logs after a week. To throttle instead, use Crawl-delay: 10. |
User-agent: DotBot
Disallow: /
Or stack it with the rest of the SEO-data crawlers in a single block, one user agent per line. The exact Wix steps are in the Wix bot blocking walkthrough. If you also run Cloudflare, you don't need a WAF rule for DotBot. robots.txt handles it.
Does DotBot affect AI visibility?
No. DotBot feeds Moz's link index and nothing else. It has no relationship with Google, Bing, OpenAI, Anthropic, or Perplexity. Blocking or allowing it has zero effect on search rankings, AI Overviews, or citations in any generative engine. |
DotBot sits outside the Engine Optimization Matrix entirely. It's a measurement crawler. The crawlers that determine whether you get cited are the ones in the AI crawlers overview, and the one that most often gets confused with a harmless SEO bot when it isn't is GPTBot.
One related caution. Domain Authority is a proxy for link equity, and link equity matters for the Authority lever in every row of the matrix. But DA is Moz's estimate of that, not the thing itself. Blocking DotBot doesn't change your actual entity authority. It just means Moz knows slightly less about your site. If your agency reports DA to you quarterly, tell them before you block, so they're not surprised when the page-level data goes thin.
Want a second opinion on your block list? Book 30 minutes.
About the author
Harold Bell is Founder and CEO of MQL Magnet, a B2B content marketing agency serving enterprise technology brands. He's the creator of the Engine Optimization Matrix (EOM), author of Partner Over Product, and a Forbes Communications Council member.
Frequently asked questions
What is DotBot?
DotBot is Moz's web crawler. It builds the link index behind Moz Link Explorer, Domain Authority, Page Authority, and Spam Score.
What is the difference between DotBot and Rogerbot?
DotBot crawls the open web for Moz's link index regardless of whether you're a customer. Rogerbot is the Moz Pro site audit crawler and only crawls sites with an active Moz Pro campaign.
Does DotBot obey robots.txt?
Yes. Moz documents that DotBot honors Disallow and Crawl-delay directives.
Will blocking DotBot lower my Domain Authority?
No. Domain Authority is computed from links pointing to your site, which Moz discovers by crawling the linking sites. Blocking DotBot from your own domain doesn't affect that.
Should I block DotBot?
Only if nobody on your team or at your agency uses Moz. If you do, blocking it makes your own domain's data incomplete in Link Explorer.
How do I block DotBot?
Add User-agent: DotBot and Disallow: / as a separate block in robots.txt, below your default rules.
Does DotBot affect Google rankings?
No. DotBot has no connection to Google, Bing, or any AI engine.
Is DotBot a bad bot?
Not in the security sense. It's a documented, robots.txt-compliant commercial crawler. Its value to you depends on whether you use Moz.
Can I slow DotBot down instead of blocking it?
Yes. Moz supports Crawl-delay. Add Crawl-delay: 10 under the DotBot user agent.
Does Google use Domain Authority?
No. Domain Authority is a Moz metric that predicts ranking potential from link data. Google has repeatedly stated it doesn't use DA or any third-party authority score.
