← All free tools
AI crawler robots.txt generator
This generator writes a robots.txt with an explicit rule per AI crawler. The important distinction it enforces: blocking a training crawler like GPTBot keeps you in ChatGPT answers, while blocking an answer indexer like OAI-SearchBot removes you from them entirely.
Block these crawlers
Your robots.txt
# Generated with nostimates.com/tools/robots-txt-generator
User-agent: GPTBot
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: *
Allow: /Runs entirely in your browser
Nothing you type here is sent to us or stored anywhere. Copy the output and go.
- Does blocking GPTBot remove me from ChatGPT?
- No. GPTBot collects training data. ChatGPT's live citations come from OAI-SearchBot, which is a separate agent with a separate robots.txt token.
- Can I block AI training but stay in Google AI Overviews?
- Partly. Disallowing Google-Extended opts you out of Gemini training while keeping Googlebot indexing intact. AI Overviews are built from the Googlebot index, so you cannot appear there while blocking Googlebot.