Nostimates
← AI crawler directory

Social preview

facebookexternalhit

MetaIgnores robots.txt

Renders Open Graph link previews when a URL is shared. Blocking breaks your link previews on Facebook, Instagram and WhatsApp.

User-agent string

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

How to verify it

Meta's published IP ranges.

What blocking costs

Blocking breaks your link previews on Facebook, Instagram and WhatsApp.

robots.txt rule

# Block facebookexternalhit
User-agent: facebookexternalhit
Disallow: /

# Allow everything else
User-agent: *
Allow: /

Robots.txt is not enough

facebookexternalhit is documented as not governed by robots.txt. Enforce at the CDN or web server, and verify the request genuinely originates from Meta before blocking by string alone.

Generate a full robots.txt

Pick which AI crawlers to allow or block across all 42 agents and copy the file out.

Open the generator →