OAI-SearchBot builds the index behind ChatGPT search — the results and links ChatGPT shows when it searches the web for an answer. Builds the index an AI search product answers from; blocking it removes you from those answers.
OAI-SearchBot builds the index behind ChatGPT search — the results and links ChatGPT shows when it searches the web for an answer. OpenAI documents that it is not used for training.
Search index: builds the index an AI search product answers from; blocking it removes you from those answers.
User-agent string:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbotrobots.txt token: OAI-SearchBot. Documented: honours robots.txt.
OpenAI publishes its IP ranges as a feed at https://openai.com/searchbot.json. A request claiming to be OAI-SearchBot from an address outside those ranges is an impostor.
AI bot verifier → checks an IP and user-agent against the published ranges and reverse DNS, free.
Blocking OAI-SearchBot removes your pages from what ChatGPT search can surface and link to. If you want to be found and cited by ChatGPT, this is the one to allow.
https://platform.openai.com/docs/bots
Check your site: the AI Crawler Access Check reads your robots.txt for OAI-SearchBot and the other AI crawlers and fetches your homepage with each one's real user-agent — the CDN challenge that robots.txt cannot show.
Blocking OAI-SearchBot removes your pages from what ChatGPT search can surface and link to. If you want to be found and cited by ChatGPT, this is the one to allow.
Add a group for it: User-agent: OAI-SearchBot on one line and Allow: / on the next. If a User-agent: * group has a Disallow, the named group takes precedence for OAI-SearchBot.
User-agent: OAI-SearchBot then Disallow: /. OpenAI documents that OAI-SearchBot honours robots.txt.
OpenAI publishes its IP ranges as a feed at https://openai.com/searchbot.json. A request claiming to be OAI-SearchBot from an address outside those ranges is an impostor.
The free crawler check reads your robots.txt per bot and fetches your homepage with each one's real user-agent.