Billions of people never open a search engine to ask about a product or a local business. They ask inside the apps they already use. Meta AI, the assistant built into WhatsApp, Instagram, Facebook and Messenger in many markets and also available as a standalone app, answers those questions right inside a chat or a search bar. If someone types "is [your brand] any good?" into WhatsApp, Meta AI may be the one replying.
Meta has published less about how Meta AI chooses sources than Google or Microsoft have. But it has documented one important piece: a dedicated web crawler, Meta-WebIndexer, that exists to improve Meta AI search results, and which Meta says helps it cite and link to your content. It has also signed content deals with news publishers and runs several other crawlers with different purposes.
This guide explains what is known about where Meta AI gets web answers as of October 2026, how the different Meta crawlers affect your visibility, and the practical steps a business can take to be represented accurately across Meta's apps.
Key Takeaways
- Meta documents a crawler called Meta-WebIndexer that navigates the web to improve Meta AI search result quality; allowing it helps Meta cite and link to your content.
- Meta-ExternalAgent is a separate crawler used for purposes such as training foundation AI models; you can block it without blocking Meta-WebIndexer.
- Meta-ExternalFetcher fetches links at a user's request and, per Meta, may bypass robots.txt.
- Meta has licensed news content, starting with a multi-year Reuters deal announced in October 2024, so trusted publishers can shape news answers.
- Accurate public information on your website, your Facebook and Instagram profiles and third-party sites gives Meta AI better material to work with.
Where Meta AI gets web answers
Meta AI combines a large language model with retrieval from the web for current or factual questions. Meta has not published a detailed description of its search pipeline, but its web crawler documentation is explicit about one source: its own web index, built by Meta-WebIndexer.
“Allowing Meta-WebIndexer in your robots.txt file helps us cite and link to your content in Meta AI's responses.”
— Meta for Developers, Meta Web Crawlers
News is another documented input. Third-party trackers such as LLM Pulse report that Meta announced a multi-year deal with Reuters on 25 October 2024, allowing Meta AI to draw on Reuters reporting for news questions with links back to the original stories, and that Meta has since worked with other publishers. For brands, the implication is that coverage in reputable news outlets is likely to carry weight in news-style answers about you.
Meta's crawlers and what each one controls
Meta runs several crawlers, and blocking the wrong one can quietly remove you from Meta AI answers or break link previews. This table summarises Meta's documentation as of October 2026.
| User agent | Purpose (per Meta) | Respects robots.txt? | Recommendation for most brands |
|---|---|---|---|
meta-webindexer | Improves Meta AI search result quality; helps Meta cite and link to your content | Controlled by robots.txt | Allow |
meta-externalagent | Use cases such as training foundation AI models or improving products by indexing content directly | Can be blocked with a disallow rule | Your choice; blocking does not block meta-webindexer |
meta-externalfetcher | Fetches individual links at a user's request for agentic AI features | Meta says it may bypass robots.txt | Allow; manage via server rules if needed |
facebookexternalhit | Builds link previews (title, description, image) when your URL is shared | May bypass robots.txt for security or integrity checks | Allow, or shared links look broken |
meta-externalads | Advertising and business-related products | No specific behaviour documented | Allow if you advertise with Meta |
These crawlers are not minor. Cloudflare's August 2025 analysis found that Meta-ExternalAgent's share of AI and search crawler traffic on its network rose from 0.9% in July 2024 to 7.5% in July 2025. Cloudflare also noted that Meta, like OpenAI and Anthropic, runs separate bots for different purposes, which is exactly why targeted rules work.
A sensible default for a business that wants visibility but prefers not to contribute to model training looks like this:
User-agent: meta-webindexer
Allow: /
User-agent: meta-externalagent
Disallow: /Meta says changes to robots.txt can take up to 24 hours to take effect. Check your server logs afterwards; our guide to AI bots in server logs explains how, and verifying AI crawlers covers spotting fake user agents. For the wider landscape of AI user agents, see AI crawlers: block or allow.
How Meta AI shows up across Meta's apps
Meta AI appears in different places, and the type of question differs by app. In WhatsApp and Messenger it lives in chats, where people ask direct questions and compare options. In Instagram and Facebook it can appear in search and alongside content, where questions are often about creators, products and trends. Because people may ask about your brand in any of these places, the content Meta can see on its own platforms (your Page, profile, posts and comments) matters alongside your website.
| Where the question is asked | Typical customer question | What you control |
|---|---|---|
| WhatsApp or Messenger chat | "Which [service] near me is open on Sunday?" | Website facts, Facebook Page hours and contact details |
| Instagram search | "Is [product] worth it?" | Profile bio, posts, captions, creator collaborations |
| "Reviews of [business]" | Page reviews and recommendations, replies to comments | |
| Standalone Meta AI app or web | "Compare [brand] and [competitor]" | Comparison pages, specs, pricing context, third-party reviews |
These are planning examples rather than a documented map of how Meta AI routes each query, but they help you decide which assets need attention first.
Meta is also building AI tools for businesses themselves. In September 2026 it launched Muse for Small Business, an AI agent that can work with Facebook and Instagram business accounts and many third-party tools, with Meta stating that nothing publishes, sends or spends without the owner's approval. That is a productivity tool rather than a visibility channel, but it signals how central AI is becoming across Meta's business products.
Steps to be represented accurately in Meta AI
- Check your robots.txt. Make sure
meta-webindexerandfacebookexternalhitare not blocked by a broad rule. - Publish a clear facts page. State what you do, where, for whom, prices or price ranges where possible, and key policies. See our guide to a brand facts page.
- Complete your Facebook Page and Instagram profile. Category, description, contact details, hours and website link should match your site.
- Set good link previews. Use Open Graph tags so shared links display correctly in chats and feeds.
- Earn trusted coverage. News and industry articles are likely inputs for news-style answers.
- Encourage genuine discussion. Reviews, community posts and forum threads beyond Meta's apps also shape AI answers. See community mentions beyond Reddit.
- Test monthly. Ask Meta AI in WhatsApp, Instagram and Facebook the questions customers ask, record answers and cited links, and compare with competitors using AI share of voice methods.
Our guides to Instagram SEO and WhatsApp marketing cover the platform-side work in more depth.
Common mistakes
- Blocking all Meta bots at once. A blanket block can remove you from Meta AI citations and break link previews.
- Assuming meta-externalagent is the search crawler. Meta documents meta-webindexer as the one tied to Meta AI search results.
- Neglecting your Facebook Page. Outdated hours or descriptions on Meta's own platform undermine accuracy.
- Expecting instant changes. Robots.txt updates can take up to 24 hours, and indexes take longer to refresh.
- Measuring only clicks. Many Meta AI answers are read in chat without a click. Track mentions and accuracy, not just referrals.
For comparison with another social-first assistant, see how Grok uses X posts.
Related Guides
- Apple Intelligence and Siri: What Changes for Brand Discovery
- Alexa+ and Local Businesses: Getting Recommended by Amazon's Assistant
- Amazon Rufus (Now Alexa for Shopping): Optimizing Listings for Amazon's AI Assistant
Frequently Asked Questions
Which Meta crawler should I allow for Meta AI visibility?
Meta-WebIndexer. Meta says it improves Meta AI search result quality and that allowing it helps Meta cite and link to your content.
Can I block Meta from training on my content but still appear in Meta AI?
Meta documents separate crawlers: you can disallow meta-externalagent, which Meta lists for uses such as training foundation AI models, while allowing meta-webindexer.
Does Meta AI use news sources?
Yes. Meta announced a multi-year deal with Reuters in October 2024 so Meta AI can answer news questions with Reuters reporting, and has reportedly worked with other publishers since.
Does my Facebook or Instagram profile affect Meta AI answers?
Meta has not published how it weights its own platforms, but keeping profile details accurate and consistent with your website reduces the risk of conflicting information.
How can I check what Meta AI says about my business?
Ask Meta AI the same set of customer questions in WhatsApp, Instagram and Facebook each month and record the answers and cited links.
Conclusion
Meta AI reaches customers where they already chat and scroll. Allow the crawler Meta uses for AI search, keep your website and Meta profiles accurate, and build credible coverage elsewhere on the web, and you give Meta AI the right material to describe you. If you want help coordinating your social profiles and AI visibility, our social media management service can take it on. Request a free quote to start.
