AI crawler checker
Find what stops AI reading your site — and get your full report with the exact fixes.
ChatGPT
Claude
Gemini
PerplexityInstant preview · Full report free by email
Includes all AI platforms and all five readiness categories.
canairead.me
AI crawler checker · Full AI readiness report included
Example AI readiness report
canairead.me · 92/100
What we check
Every major AI product sends its own crawler, with its own name and its own purpose — training, live search, or on-demand fetching. Blocking the wrong one, or assuming a blanket "disallow all" rule covers AI, is the single most common reason sites disappear from AI answers.
- GPTBot and OAI-SearchBot (OpenAI / ChatGPT)
- ClaudeBot and Claude-SearchBot (Anthropic / Claude)
- Google-Extended and Googlebot (Gemini / Google)
- PerplexityBot (Perplexity)
Common problems
Wildcard disallow rules
A rule meant for one bot accidentally matches several AI crawlers with similar names.
Confusing training and citation bots
Blocking GPTBot to opt out of training also has no effect on ChatGPT's live search citations — that's OAI-SearchBot.
How to fix them
Name each crawler explicitly
Write a dedicated User-agent rule for each bot you want to allow or block, rather than relying on a catch-all.
Decide training vs. citation separately
You can opt out of model training while still remaining eligible for live citations — they're controlled independently.
Frequently asked
Will blocking GPTBot remove me from ChatGPT entirely?
No. It stops your content being used for model training. Live citations in ChatGPT search are controlled by OAI-SearchBot, which you can allow separately.
Where do I see the full list of AI crawlers?
See our AI crawler directory for the complete, regularly updated list with purpose and source documentation for each.