OAI-SearchBot
Whether ChatGPT can surface and cite your pages
What OAI-SearchBot does
OAI-SearchBot is the crawler behind ChatGPT's search. When ChatGPT answers a question by citing a page and linking to it, this is the agent that fetched the page. It is not used to train models: OpenAI separates that job into GPTBot, and the separation is the whole point of publishing two tokens.
This makes OAI-SearchBot the single most consequential line in most robots.txt files, and the one most often wrong. A site that disallows it has removed itself from ChatGPT's answers, which is the opposite of what almost everyone who edits that file is trying to do.
It is also the clearest case of a block that was never decided. Blocking a training crawler is a position somebody argued for. Blocking the crawler that sends you readers is, in nearly every case we have looked at, a snippet that was pasted whole.
OAI-SearchBot is not GPTBot
Both are OpenAI, both appear in the same documentation page, and the two names are one letter apart in most people's memory. One decides whether you can be quoted; the other decides whether you are used for training.
What GPTBot does insteadHow to block OAI-SearchBot
Add this to the robots.txt at the root of your domain, alongside what is already there rather than replacing it.
robots.txt
User-agent: OAI-SearchBot
Disallow: /What that costs you: ChatGPT search cannot cite you. This is the row that matters most, and the one most often blocked by accident.
To allow it instead, write nothing. A crawler with no group of its own follows your User-agent: * rules, and giving it a named group with Allow: / detaches it from those rules entirely — including any path you disallow there.
Find it in your logs
The short token is what robots.txt matches. This is what actually arrives in an access log, copied from OpenAI’s documentation. Version numbers move, so match on the token rather than the whole string.
User-agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbotTo find every visit it has made, and the addresses it came from:
Terminal
grep -F 'OAI-SearchBot' /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rnCheck that it is really OAI-SearchBot
A user agent is a string the visitor chooses, so on its own it proves nothing — anyone can send yours. OpenAI publishes the addresses OAI-SearchBot fetches from: 39 prefixes when we read the file, last changed on 2 January 2026. Take an address out of your logs and check it against that list.
Terminal
curl -s https://openai.com/searchbot.json | python3 -c "
import sys, json, ipaddress
addr = ipaddress.ip_address('203.0.113.10') # the address from your logs
nets = json.load(sys.stdin)['prefixes']
print(any(addr in ipaddress.ip_network(n.get('ipv4Prefix') or n['ipv6Prefix']) for n in nets))
"False means something else is wearing the name, and it is under no obligation to respect anything in your robots.txt. That is the traffic worth blocking at the edge, rather than the crawler that was asking politely.
What OpenAI says about it
Changes take about a day
OpenAI states it can take roughly 24 hours from a robots.txt change for their systems to adjust. If you fix a block and nothing happens immediately, that is why. Check again tomorrow before changing anything else.
It labels its own robots.txt fetches
When OpenAI fetches robots.txt rather than a page, it may add a robots.txt marker to the user agent, giving … compatible; OAI-SearchBot/1.4; robots.txt; +https://openai.com/searchbot. That lets you tell a policy read from a content fetch in logs that do not record paths.
Allow the IP ranges as well
OpenAI's own recommendation is not only to allow the token in robots.txt but to permit requests from its published IP ranges. A robots.txt that says yes and a firewall that says no is the most common reason a site is allowed on paper and absent in practice.
Read from https://platform.openai.com/docs/bots on 14 September 2026.
Should you block OAI-SearchBot?
Almost certainly not. This is the crawler that lets ChatGPT quote you and send the reader back, and no training exposure is attached to it: that lives in GPTBot, which you can block separately and for free.
The one real argument for blocking it is that you sell access to your archive and would rather an assistant did not summarise it away. That is a business argument about substitution, not a privacy one, and it is the argument large publishers are actually having.
If you are blocking it because you read that AI crawlers should be blocked, you have almost certainly done the wrong thing. That advice is about training, and this is not the training crawler.
Questions
- Does blocking OAI-SearchBot stop OpenAI training on my content?
- No. Training is governed by GPTBot. Blocking OAI-SearchBot stops ChatGPT citing you and leaves the training question exactly where it was.
- Will allowing it get me cited?
- It makes it possible. Access is a precondition, not a cause: a crawler that cannot read you certainly cannot quote you, and one that can will quote you only if the page answers something clearly.
The user agent and IP figures on this page were read from OpenAI’s own URLs on 14 September 2026. Blocking figures come from our survey of 170 well-known sites across 10 categories on 14 September 2026, 148 of which returned a readable robots.txt.
Is OAI-SearchBot allowed on your site right now?
The checker reads your live robots.txt and reports every token in this directory, with the exact directive that decided each one.