← AI crawler directory
AI training
Google-Extended
GoogleHonours robots.txt
A robots.txt control token governing whether your content trains Gemini and grounds Vertex AI, without affecting Search. Disallowing it keeps you in Google Search and AI Overviews while opting out of Gemini training. It is the only surgical control Google offers.
User-agent string
(control token only — not a distinct user agent string)
How to verify it
Not applicable — it is a directive, not a fetching agent.
What blocking costs
Disallowing it keeps you in Google Search and AI Overviews while opting out of Gemini training. It is the only surgical control Google offers.
robots.txt rule
# Block Google-Extended
User-agent: Google-Extended
Disallow: /
# Allow everything else
User-agent: *
Allow: /Robots.txt is enough
Google documents Google-Extended as honouring robots.txt. Verify with your logs after deploying — a spoofed agent will keep coming.
Generate a full robots.txt
Pick which AI crawlers to allow or block across all 42 agents and copy the file out.
Open the generator →Other Google agents