AI Crawler Checker
Enter your domain. We check which AI crawlers your site allows and whether your business details can be read by a machine at all.
Free, no sign-up, no email needed for this check.
The part you control
Your robots.txt either allows or blocks GPTBot, ClaudeBot, PerplexityBot and Google-Extended. A surprising number of sites block them without knowing, usually because a plugin or a template did it.
Being honest about what is not proven
There is no published evidence from Google, OpenAI, Anthropic or Perplexity that any particular markup makes an assistant more likely to cite you. Google states plainly that no special AI markup is required. Anybody selling you a guaranteed method for this is guessing.
What structured data does do
It removes ambiguity. Your address, hours and services written as prose have to be interpreted. The same facts in structured data are simply read. That is a real difference, and it is a smaller claim than the one usually made for it.
Common questions
Should I allow AI crawlers?
It is a trade. Allowing them is how you can be quoted; blocking them keeps your content out of training and answers. There is no universally right answer, but it should be a decision rather than an accident.
Does an llms.txt file help?
No provider has documented reading it. It costs nothing to publish, but nobody should promise you a result from it.
Is blocking AI crawlers bad for Google rankings?
Google-Extended controls AI training and AI answers, not normal Search indexing. Blocking it does not remove you from ordinary search results.