ClaudeBot

Anthropic crawler that collects public web data for Claude model training.

AI crawler, used for model training and/or live retrieval.

Nov 2023
first seen in our archive: 5
distinct User-Agents: 5+
requests seen: AI
crawler

Official: support.anthropic.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler·Vendor: Anthropic

Parse it

Send the User-Agent to the API, device, OS, browser, and bot detection come back in one response. ClaudeBot is identified by name when the string matches.

curl -s "https://api.parseapi.com/useragent" \
  -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Safari/605.1.15" \
  -H "X-Api-Key: $PARSEAPI_KEY"

Create a free key Useragent API