Skip to main content
Platform · AI Search

​
Definition

ClaudeBot is Anthropic’s web crawler primarily used to gather training data for its AI models, which contributes to the content available for Claude AI responses. Like GPTBot, it is identifiable via its user-agent string in server logs. Content must be accessible to ClaudeBot to be available for citation in Claude’s responses. Each major AI platform has its own crawler with distinct user-agent strings. Brands monitoring server logs for AI crawler activity can track which platforms are indexing their content and identify accessibility issues specific to each bot. ClaudeBot accessibility is the technical prerequisite for Claude citation.

AI Crawler

GPTBot

PerplexityBot

Robots.txt

Indexability

​
Relevant Plate Lunch Collective Services

AI SEO