Crawls content for Cohere model training and retrieval products. It is operated by Cohere.

Does cohere-ai respect robots.txt?

Yes. Cohere states that cohere-ai honours robots.txt directives, so a Disallow rule for the cohere-ai token is sufficient.

AI training

cohere-ai

CohereHonours robots.txt

Crawls content for Cohere model training and retrieval products. Enterprise RAG exposure. Low consumer impact.

cohere-ai

How to verify it

Cohere support documentation lists source ranges.

What blocking costs

Enterprise RAG exposure. Low consumer impact.

# Block cohere-ai
User-agent: cohere-ai
Disallow: /

# Allow everything else
User-agent: *
Allow: /

Robots.txt is enough

Cohere documents cohere-ai as honouring robots.txt. Verify with your logs after deploying — a spoofed agent will keep coming.

Generate a full robots.txt

Pick which AI crawlers to allow or block across all 42 agents and copy the file out.

Other Cohere agents