← AI crawler directory
Search
Baiduspider
BaiduHonours robots.txt
Baidu's primary crawler, feeding both classic results and ERNIE AI answers. Blocking removes you from the highest-volume Chinese AI answer surface.
User-agent string
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
How to verify it
Reverse DNS to baidu.com or baidu.jp.
What blocking costs
Blocking removes you from the highest-volume Chinese AI answer surface.
robots.txt rule
# Block Baiduspider
User-agent: Baiduspider
Disallow: /
# Allow everything else
User-agent: *
Allow: /Robots.txt is enough
Baidu documents Baiduspider as honouring robots.txt. Verify with your logs after deploying — a spoofed agent will keep coming.
Generate a full robots.txt
Pick which AI crawlers to allow or block across all 42 agents and copy the file out.
Open the generator →