← AI crawler directory
AI answer
OAI-SearchBot
OpenAIHonours robots.txt
Builds the search index ChatGPT uses to cite live sources in answers. Blocking this removes you from ChatGPT's citable index. For most brands this is the single most costly block on the list.
User-agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot
How to verify it
OpenAI publishes IP ranges at openai.com/searchbot.json.
What blocking costs
Blocking this removes you from ChatGPT's citable index. For most brands this is the single most costly block on the list.
robots.txt rule
# Block OAI-SearchBot
User-agent: OAI-SearchBot
Disallow: /
# Allow everything else
User-agent: *
Allow: /Robots.txt is enough
OpenAI documents OAI-SearchBot as honouring robots.txt. Verify with your logs after deploying — a spoofed agent will keep coming.
Generate a full robots.txt
Pick which AI crawlers to allow or block across all 42 agents and copy the file out.
Open the generator →Other OpenAI agents