What is meta-externalagent?

Collects public web data to train Meta AI models. It is operated by Meta.

Does meta-externalagent respect robots.txt?

Yes. Meta states that meta-externalagent honours robots.txt directives, so a Disallow rule for the meta-externalagent token is sufficient.

AI training

MetaHonours robots.txt

Collects public web data to train Meta AI models. Training-only. Distinct from facebookexternalhit, which renders link previews.

meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)

How to verify it

Meta's published crawler IP ranges.

What blocking costs

Training-only. Distinct from facebookexternalhit, which renders link previews.

# Block meta-externalagent
User-agent: meta-externalagent
Disallow: /

# Allow everything else
User-agent: *
Allow: /

Robots.txt is enough

Meta documents meta-externalagent as honouring robots.txt. Verify with your logs after deploying — a spoofed agent will keep coming.

Generate a full robots.txt

Pick which AI crawlers to allow or block across all 42 agents and copy the file out.

Other Meta agents