Claude-User
Fetches made when a person asks Claude to open your link
What Claude-User does
Claude-User is the fetch made when a person asks Claude to open a specific link. Like OpenAI's ChatGPT-User it is one request serving one human request, not a crawl.
Anthropic's position here is more explicit than OpenAI's. Where OpenAI notes that robots.txt may not apply to user-initiated fetches, Anthropic states that its bots honour robots.txt directives across its agents, Claude-User included.
Blocking it means a reader who pastes your link into Claude gets an error rather than your page, which is a bad outcome for a visitor who was actively trying to read you.
How to block Claude-User
Add this to the robots.txt at the root of your domain, alongside what is already there rather than replacing it.
robots.txt
User-agent: Claude-User
Disallow: /What that costs you: Someone who pastes your URL into Claude gets an error instead of your page.
To allow it instead, write nothing. A crawler with no group of its own follows your User-agent: * rules, and giving it a named group with Allow: / detaches it from those rules entirely — including any path you disallow there.
Find it in your logs
Anthropic names the token but does not publish a full user-agent string for it, so match on the token itself.
To find every visit it has made, and the addresses it came from:
Terminal
grep -F 'Claude-User' /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rnCheck that it is really Claude-User
Harder than it should be. Anthropic documents the token and states that its crawlers honour robots.txt, but publishes no list of the addresses they crawl from — unlike OpenAI, Perplexity, Google and Bing. Reverse DNS and request patterns are what is left. Treat an unverifiable user agent as unverified, because anyone can send any string.
Before you block it by IP
Anthropic warns against this one specifically. Blocking its IP addresses may not work correctly or persistently guarantee an opt-out, because it also stops them reading the robots.txt that carries your opt-out. With this vendor an edge block can defeat the thing it was meant to enforce.
Slow it down instead of shutting it out
If the problem is how hard Claude-User is hitting you rather than whether it should be here at all, Anthropic supports the non-standard Crawl-delay extension. Most vendors ignore it, so this is worth knowing where it exists: it keeps you visible and takes the load off.
robots.txt
User-agent: Claude-User
Crawl-delay: 1What Anthropic says about it
It commits to robots.txt
Anthropic states that its bots respect do-not-crawl signals by honouring industry standard directives in robots.txt, and applies that to all three of its agents including the user-triggered one. That is a firmer commitment than OpenAI or Perplexity make about user-initiated fetches.
Read from https://support.anthropic.com/en/articles/8896518 on 14 September 2026.
Should you block Claude-User?
Usually leave it alone. It serves a reader who has actively asked for your page.
Anthropic's commitment covers this agent explicitly, which is worth contrasting with the other two vendors: OpenAI notes robots.txt may not apply to user-initiated fetches, and Perplexity says outright that its equivalent generally ignores them. Here a Disallow is likely to be honoured.
Questions
- Does Claude-User respect robots.txt?
- Anthropic says its crawlers honour robots.txt directives, and names Claude-User among its agents. That is a firmer commitment than the equivalent vendors make about user-initiated fetching.
The user agent and IP figures on this page were read from Anthropic’s own URLs on 14 September 2026. Blocking figures come from our survey of 170 well-known sites across 10 categories on 14 September 2026, 148 of which returned a readable robots.txt.
Is Claude-User allowed on your site right now?
The checker reads your live robots.txt and reports every token in this directory, with the exact directive that decided each one.