An AI search audit answers one practical question: what is stopping AI systems from finding, understanding, trusting and recommending your business — and what should you fix first? It borrows from a classic SEO audit, but adds checks that a traditional audit usually skips: which AI crawlers can reach you, how consistently your brand facts appear across the web, which third-party sources AI tools cite in your category, and whether you can measure any of it.
This guide gives you a six-part audit process you can run in-house over a week or two, plus a scoring sheet so you can compare results over time and across competitors. It is deliberately tool-agnostic: everything can be done with a spreadsheet, your server logs, Search Console, Bing Webmaster Tools and the AI assistants themselves.
If you have never run a standard technical audit, start with our SEO audit guide — most AI search issues sit on top of the same foundations.
Key Takeaways
- Audit six areas: access, technical readiness, entity data, content, off-site mentions and measurement.
- Separate search crawlers (which affect AI search visibility) from training crawlers (which affect model training) before changing robots.txt.
- Entity consistency — the same facts about your business everywhere — is often the cheapest big win.
- Score each check 0–2 so you can prioritise fixes and re-audit quarterly.
- End with a prioritised action list, not a long report nobody acts on.
Step 1: Access — can AI systems reach your content?
Start with robots.txt, then check CDN, firewall and bot-management settings, which often block bots regardless of robots.txt. Each platform documents its own user agents:
| Platform | Search / retrieval agent | Training agent or token | Note |
|---|---|---|---|
| OpenAI | OAI-SearchBot; ChatGPT-User (user-initiated) | GPTBot | Blocking OAI-SearchBot removes you from ChatGPT search answers |
| Googlebot | Google-Extended (robots.txt token) | Google-Extended does not affect Search inclusion or ranking | |
| Anthropic | Claude-SearchBot; Claude-User | ClaudeBot | All honour robots.txt, per Anthropic |
| Perplexity | PerplexityBot; Perplexity-User | — (PerplexityBot is not used for foundation-model training, per Perplexity) | Perplexity-User generally ignores robots.txt as it is user-requested |
| Microsoft | Bingbot | — | Bing Webmaster Tools reports citations in Copilot |
“Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers.”
— OpenAI, Overview of OpenAI crawlers
“Google-Extended does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search.”
— Google, Google's common crawlers
Then confirm what actually happens: look for these user agents in your server logs and verify they are genuine. Our guides to AI bots in server logs and verifying AI crawlers explain how. For the policy decision itself, see AI crawlers: block or allow?
Step 2: Technical readiness
- Key pages are indexed in Google and Bing (Google requires a page to be indexed and snippet-eligible to appear as a link in AI Overviews or AI Mode).
- Important content is in the HTML, not only rendered by JavaScript — a Vercel analysis cited in Semrush’s comparison of traditional and AI SEO found most major AI crawlers don't render JavaScript; see JavaScript and AI crawlers.
nosnippet,data-nosnippetandmax-snippetare used only where intended.- Your site is included in the Search generative AI features setting in Search Console.
- Structured data is valid and matches visible content; pages load reliably for bots (no 403s or timeouts).
Step 3: Entity data
AI systems need to be confident about who you are. Compare these facts across your website, Google Business Profile, Bing Places, LinkedIn, Wikidata/Wikipedia (if present), Crunchbase or industry directories:
- Legal and trading name, founding date, founders/leadership.
- Headquarters and locations, contact details.
- What you sell, to whom, and where; pricing approach.
- Awards, certifications and memberships.
Use Organization schema with sameAs links to official profiles. Our guides to entity SEO and Wikidata for brands go deeper.
Step 4: Content
- List your 30–100 most valuable prompts (brand, product, category, comparison, local).
- Map each prompt to the page that should answer it; note prompts with no matching page.
- Check each mapped page: does it answer directly near the top, include specifics (numbers, dates, prices or price approach, locations), show who wrote it and why they are credible, and stay up to date?
- Flag outdated pages that AI tools are still quoting.
- Prioritise gaps where competitors are named and you are not.
Step 5: Off-site mentions
Run your prompt set and record which domains are cited. In most categories a handful of review sites, publications, directories and forums appear repeatedly. Check whether you are present, accurate and positively described on each. Our AI citation audit guide covers this step in detail, including how to spot gaps against competitors.
Step 6: Measurement
- AI referral traffic segmented in analytics.
- Search Console Generative AI performance report reviewed (impressions in AI Overviews and AI Mode by page, country, date and device).
- Bing Webmaster Tools AI Performance report reviewed (citations, cited pages and grounding queries for Copilot and Bing AI summaries).
- A repeatable prompt tracking process with stored results.
- Self-reported attribution (“How did you hear about us?”) including AI options.
The AI search audit scoring sheet
Score each item 0 (missing/broken), 1 (partial) or 2 (good). Total out of 40. Re-score quarterly and compare with two competitors where you can assess them from the outside.
| Area | Check | Score (0–2) |
|---|---|---|
| Access | Search crawlers allowed in robots.txt | |
| Access | No CDN/firewall blocks on search bots | |
| Access | Training-crawler policy decided deliberately | |
| Technical | Key pages indexed in Google and Bing | |
| Technical | Core content in HTML without JavaScript | |
| Technical | Snippet controls and generative AI setting correct | |
| Technical | Structured data valid and matching content | |
| Entity | Consistent name, address, contact details | |
| Entity | About / facts page complete and current | |
| Entity | Organization schema with sameAs links | |
| Entity | Key profiles claimed and accurate | |
| Content | Priority prompts mapped to pages | |
| Content | Pages answer directly with specifics | |
| Content | Author/expertise signals visible | |
| Content | No outdated pages being quoted | |
| Off-site | Present on top cited sources in category | |
| Off-site | Reviews recent and responded to | |
| Measurement | AI referrals tracked in analytics | |
| Measurement | Search Console and Bing AI reports reviewed | |
| Measurement | Prompt tracking baseline stored |
Scores of 0 in Access or Technical come first, because nothing else matters if bots can't read the site. Then fix Entity errors, then Content and Off-site gaps. Turn the results into a schedule with our 90-day AI search plan.
Common mistakes
- Blocking every AI bot to stop training and accidentally removing yourself from AI search answers.
- Only checking robots.txt while a CDN rule silently returns 403s to bots.
- Auditing content without prompts, so you optimise pages nobody asks AI about.
- Treating one AI answer as the result. Answers vary; test important prompts several times.
- Producing a report without owners and dates for each fix.
Conclusion
A good AI search audit is short, repeatable and ends with a prioritised fix list. Run it quarterly, keep your scoring sheet, and you will see clearly whether access, entity data, content and off-site presence are improving.
Prefer an expert to run it? Our SEO team can audit your site and competitors — get a free quote.
Related Guides
- AEO vs GEO vs SEO: Do You Really Need Different Strategies?
- Does Traditional SEO Still Matter for AI Search? What the Evidence Shows
- AI Search Myths: 15 Claims That Don't Hold Up
Frequently Asked Questions
How long does an AI search audit take?
For a small or mid-sized site, one to two weeks of part-time work. Large sites with many locations or markets take longer.
Is an AI search audit different from an SEO audit?
It builds on one. It adds AI crawler access, entity consistency, prompt-based content mapping, cited-source analysis and AI-specific measurement.
Will blocking GPTBot remove us from ChatGPT?
According to OpenAI, GPTBot relates to model training, while OAI-SearchBot controls whether sites appear in ChatGPT search answers. Check both settings.
Do we need llms.txt to pass the audit?
No. Google says no special AI text files are needed for its AI features, and support among other platforms is unclear, so we do not score it.
How often should we re-audit?
Quarterly for most businesses, plus after major site changes such as a migration, redesign or CDN change.
References
- Overview of OpenAI crawlers — OpenAI
- Google's common crawlers (Google-Extended) — Google
- Does Anthropic crawl data from the web, and how can site owners block the crawler? — Claude Help Center
- Perplexity crawlers — Perplexity docs
- AI features and your website — Google Search Central
- Generative AI performance report (Search) — Search Console Help
- Introducing AI Performance in Bing Webmaster Tools (Public Preview) — Bing Webmaster Blog, Feb 2026
- Traditional SEO vs. AI SEO: What You Actually Need to Know — Semrush, Nov 2025
