A visitor asks ChatGPT for a guide to a niche problem in your field. The answer recommends your site, with a link. They click, and they land on your 404 page. The page the assistant described never existed. It invented a URL that looked like one of yours.
This is not a rare glitch. Ahrefs analysed 16 million URLs in 2025 and found that clicked links from AI assistants hit 404 pages almost three times as often as clicks from Google. For most sites the traffic involved is small, but it is unusually warm traffic: someone was told by a tool they trust that you have exactly what they need. Sending that person to a dead end wastes the recommendation.
This guide shows how to find these phantom URLs in GA4, server logs and Search Console, how to read the patterns behind them, and how to decide which ones deserve a redirect, a new page, or a better error page. It also covers a 404 template written for visitors who arrive from an AI answer, and a lightweight monthly routine to keep the problem in check.
Where phantom URLs come from: stale memory versus invented slugs
AI assistants produce broken links to your site in two distinct ways, and the difference matters because each one calls for a different fix.
- Stale memory. The URL really existed. The model learned it from older training data or an old crawl, and the page was later deleted, merged or moved without a redirect. This is a classic migration leak that an assistant has kept alive.
- True invention. The URL never existed. The model generated something that fits your site's URL pattern because that is what a plausible link to your site looks like. Ahrefs found examples on its own blog such as
/blog/internal-links/and/blog/newsletter/, slugs that follow its structure but were never published. - Mangled real URLs. A hybrid: the assistant gets most of a real URL right but drops a folder, changes a word, adds a year or swaps hyphens for underscores.
Language models predict text. When an assistant answers from its internal knowledge rather than a live search result, a URL is just another string to predict. Assistants that retrieve and cite live pages produce fewer invented links, which is consistent with what Ahrefs saw: Perplexity and Gemini, which lean on a search index, came close to the Google baseline in its citation test, while ChatGPT had the highest broken-link rate in both of its tests.
“AI assistants send visitors to 404 pages 2.87x more often than Google Search.”
— Ryan Law, Ahrefs, New Study: How Often Do AI Assistants Hallucinate Links?
Sizing the problem before you spend a day on it
Before you build workflows, check whether you have a problem worth solving. Ahrefs ran two tests: one on URLs that visitors actually clicked from AI assistants (using anonymised Ahrefs Web Analytics data), and one on URLs that assistants cited in answers (using Ahrefs Brand Radar). The numbers below are from that September 2025 study.
| Assistant | Clicked URLs that were 404s | Cited URLs that were 404s |
|---|---|---|
| ChatGPT | 1.01% | 2.38% |
| Claude | 0.58% | not tested |
| Copilot | 0.34% | 0.54% |
| Perplexity | 0.31% | 0.87% |
| Gemini | 0.21% | 0.86% |
| Mistral | 0.12% | not tested |
| Google Search (baseline) | 0.15% | 0.84% (top 20 results) |
Two things follow from these figures. First, the rates are low in absolute terms, so a small site may see only a handful of phantom visits a month. Search Engine Journal's coverage of the study noted that AI assistants accounted for roughly 0.25% of website traffic in the data set, against 39.35% for Google. Second, the pain is concentrated: individual sites report far higher shares. Semrush's guide on this topic cites a LinkedIn post in which one consultant found 57% of a single client's ChatGPT referrals landing on 404s. That client is an outlier, but it shows how a migration or an unusual URL scheme can turn a rounding error into a real leak.
A quick test: if AI referrals are under 1% of your sessions and fewer than 3% of them hit a 404, a monthly check is enough. If either number is much higher, or if AI-referred visitors convert well on your site, treat it as a project.
Finding AI-referred 404s in GA4 in fifteen minutes
GA4 doesn't record HTTP status codes, so you identify 404s by their page title. Make sure your error template has a distinctive title such as "Page not found | Brand" before you start. Then follow these steps.
- Open Explore and start a free-form exploration. Add the dimensions Session source, Session default channel group, Page title and Page path and screen class. Add the metrics Sessions and Views.
- Filter for AI traffic. Since May 2026, GA4 has an AI Assistant channel in the default channel group. Search Engine Journal reported in July 2026 that the documented list covered ChatGPT, Gemini, DeepSeek, Copilot and Grok, while Perplexity still landed in Referral. So use a filter on session source that matches
chatgpt|perplexity|gemini|copilot|claude|deepseek|grok|mistralas a regex, rather than relying on the channel alone. - Add a second filter on page title. Page title contains "not found" (or whatever your 404 title says).
- Put Page path in rows and Session source in columns. You now have a list of phantom URLs and which assistant sent visitors to each.
- Widen the date range to at least 90 days. Most phantom URLs get a few visits each, so short ranges hide the pattern.
- Export to a sheet and add columns for "closest real page", "backlinks", "decision" and "date fixed". This sheet becomes your working log.
OpenAI's publisher FAQ states that ChatGPT adds utm_source=chatgpt.com to referral URLs. That helps when the referrer header is stripped by an in-app browser: the session may land in Unassigned or Direct, but the landing page URL still carries the parameter. If you see AI-looking sessions in the wrong bucket, our guide to why GA4 dumps sessions into Unassigned explains the channel rules, and our walkthrough on tracking ChatGPT and AI traffic in GA4 covers building a custom AI channel.
Confirming hallucinations in server logs and Search Console
GA4 only sees visits where the tag fires and consent is given. Server or CDN logs see every request, which makes them the better source of truth for counting.
What to pull from the logs
- Requests with status 404 where the referrer contains an AI assistant's domain, or the query string contains
utm_source=chatgpt.com. - Requests with status 404 from AI user agents. OpenAI documents a ChatGPT-User agent that visits pages when a user's question requires it. A 404 for that agent means an assistant tried to read a URL that doesn't exist, often moments before a human clicks the same link.
- The same URLs across a 30-day window, so you can see whether a phantom is a one-off or a recurring guess.
Be careful not to confuse these with bot noise. Scrapers and vulnerability scanners generate thousands of 404s for paths such as /wp-login.php. Phantom AI URLs look like real content slugs. Our guide to spotting and filtering bot traffic in analytics covers how to separate the two.
What Search Console adds
Search Console's page indexing report lists URLs Google found that return 404. If a hallucinated URL appears there, someone has published it somewhere crawlable, perhaps in AI-written content on another site. That changes your decision, because a URL with external links has link value worth keeping. Use the URL Inspection tool to see the referring page where available, and check your backlink tool for the same path.
Reading the patterns in a list of invented URLs
Once you have 20 or more phantom paths, group them. Patterns tell you whether you are looking at a site problem you can fix at the source, or at genuine invention you can only manage.
| Pattern | Example (hypothetical) | Likely cause | Best response |
|---|---|---|---|
| Old folder structure | /blog/2019/email-tips/ when posts now live at /email-tips/ | Stale memory from before a migration | Pattern-based redirect rule |
| Topic you never covered | /guides/hubspot-vs-salesforce/ | True invention that fits your URL style | Build the page if demand is steady; otherwise 404 |
| Near-miss slug | /seo-checklist-2026/ when the real page is /seo-checklist/ | Mangled real URL (year added) | One-to-one 301 to the real page |
| Product or plan that doesn't exist | /pricing/enterprise-plus/ | Assistant guessing at your offer | Redirect to the real pricing page and fix the source information |
| Help article invented from a feature name | /help/export-to-csv/ | Plausible guess from product vocabulary | Often worth writing: it reflects a real user question |
| Wrong subdomain or locale | /en-us/about/ on a site without locales | Pattern borrowed from other sites | Folder-level redirect or helpful 404 |
The "product that doesn't exist" pattern deserves special attention. If an assistant keeps inventing a plan name or feature, it may be presenting wrong facts about your business in the answer itself, not just in the link. That is a brand accuracy problem as well as a 404 problem. Our guide on fixing wrong or outdated brand information in AI covers the source-level fixes.
Redirect, build or let it 404: a decision matrix
The temptation is to redirect everything so no visitor ever sees an error. Resist it. A redirect from an invented URL to an unrelated page confuses visitors and can create its own problems: Google's documentation explains that a page which is effectively an error message gets reported as a soft 404, and blanket redirects of missing URLs to the home page often end up classified that way. Our explainer on soft 404 errors covers the details.
| Situation | Decision | Why |
|---|---|---|
| A real page answers the same question | 301 redirect to that page | The visitor gets what the assistant promised |
| The phantom URL has backlinks | 301 redirect to the closest relevant page | Keeps the link value and serves referral visitors |
| Steady visits (for example, 10 or more a month for three months) on a topic you should cover | Create the page at that URL | The assistant has shown you demand you didn't know about |
| A recurring pattern from an old structure | Rule-based redirect | Fixes hundreds of URLs at once |
| One or two visits, irrelevant topic | Leave as 404 | A redirect would mislead more than it helps |
| Invented URL implies a claim you can't support (a guarantee, a free plan) | 404 with clear messaging, then fix the source information | Redirecting would appear to confirm the claim |
Use 301s for these redirects, since the phantom URL is never coming back. If you need a refresher on status codes or implementation, see 301 vs 302 redirects and setting up 301s in .htaccess. If your platform can only do client-side redirects, read our set guide on meta refresh and JavaScript redirects first, because they behave differently for crawlers and for AI fetchers.
Google's John Mueller was asked in March 2025 whether site owners should redirect these links. His view, as reported by Search Engine Journal, was to gather evidence before investing heavily:
“perhaps it makes sense to collect some more data before spending too much on it.”
— John Mueller, quoted by Search Engine Journal, Google's Mueller Predicts Uptick of Hallucinated Links
That is sensible advice. The decision matrix above is designed to make the cheap fixes now and defer the expensive ones (new pages) until the data justifies them.
Writing a 404 page for visitors an AI assistant misled
Most 404 pages assume the visitor mistyped a URL or followed an old link. A visitor from an AI assistant is in a different state: they were promised a specific answer, and they may not realise the link was invented. Mueller's comment in the same exchange is the right brief for this page: "A good 404 page could help explain the value of the site, and where to go for more information."
A 404 page built for AI referrals should include:
- A plain explanation. Something like "This page doesn't exist. Links suggested by AI assistants are sometimes incorrect." It removes the visitor's assumption that your site is broken.
- A site search box prefilled with words from the URL. If the path is
/guides/crm-data-migration/, prefill "crm data migration". A few lines of JavaScript can split the slug on hyphens. Our set guide on internal site search pages covers keeping those result pages out of the index. - Three to five suggested pages based on the folder or keywords in the path. Even a simple lookup table (anything in
/pricing/suggests the pricing page) beats a generic list. - A route to a human. For service businesses, a contact link or chat converts a lost visit into a conversation.
- The correct status code. The page must still return a 404 (or 410) status. A friendly error page that returns 200 is a soft 404.
You can also detect AI referrals on the error page itself, using the referrer or the utm_source parameter, and show the AI-specific message only to those visitors. That keeps the standard message for everyone else. Our broader guide to finding and fixing 404 errors covers the rest of the template.
Worked example: triaging 40 phantom URLs on a software blog
This is a hypothetical example to show the method; the numbers are illustrative.
A B2B software company runs a 300-post blog. Over 90 days, GA4 shows 2,100 sessions from AI assistants, of which 126 (6%) landed on the 404 page across 40 distinct paths. That is well above the 1% ChatGPT rate in the Ahrefs data, so something structural is likely going on.
- Grouping. 22 of the 40 paths start with
/resources/blog/, the folder the blog used before a 2023 migration. Another 9 add a year to a real slug. The remaining 9 are genuine inventions. - The migration leak. One rule redirecting
/resources/blog/(.*)to/blog/$1fixes 22 paths and 71 of the 126 sessions. The team also discovers 140 old URLs that were never redirected, which were losing backlink value regardless of AI. - The near-misses. Nine one-to-one 301s cover the year-suffixed slugs, recovering another 28 sessions.
- The inventions. Two invented paths, both about data migration, account for 19 sessions between them. The team briefs a new guide and publishes it at the more popular of the two URLs, then redirects the other to it. The last seven inventions get one or two visits each and stay as 404s.
Result: roughly 94% of the phantom sessions (118 of 126) now reach a relevant page, and the most valuable fix had nothing to do with AI: it was a migration gap the assistants exposed.
Keeping hallucinated 404s on a monthly watchlist
Mueller predicted in March 2025 that clicks on hallucinated links would tick up over the following six to twelve months. Whether or not that continues, assistants change models and retrieval systems often, so a URL that was never guessed before may start appearing next quarter. A light monthly routine is enough for most sites.
| Check | Where | Frequency | Trigger for action |
|---|---|---|---|
| AI-referred sessions on the 404 title | Saved GA4 exploration | Monthly | Share above about 2% of AI sessions, or any new path with 5+ visits |
| 404 responses with AI referrers or AI user agents | Server or CDN logs | Monthly | New recurring paths |
| Redirected phantom URLs still receiving traffic | GA4 landing page report, filtered by AI sources | Quarterly | Confirm the destination still converts |
| Invented URLs with backlinks | Search Console and backlink tool | Quarterly | Any new linking domain |
| Assistant answers about your brand | Manual prompts or a tracking tool | Quarterly | Answers citing non-existent products, plans or pages |
The last row connects this work to AI visibility more broadly. If you already run regular prompt tests (see our guide on tracking brand visibility in AI search, or the set guide on AI search competitor gap analysis), add a step that clicks every link the assistant gives you. For newer sites with few real pages, invented URLs can be more common, because the model has less real structure to draw on; our guide to AI visibility for new brands covers building that footprint.
Questions site owners ask about AI-invented URLs
Do hallucinated URLs from ChatGPT hurt my rankings in Google?
Not directly. A URL that returns a proper 404 and has no links pointing to it is just a page that doesn't exist. Google's documentation notes that newly encountered 404 pages aren't processed. The cost is the lost visitor. It becomes an SEO issue only if you respond badly, for example by redirecting every invented URL to the home page.
Should I redirect every phantom URL I find?
No. Redirect when there's a genuinely close match or when the URL has backlinks. Build a page when the invented URL keeps attracting visits for a topic you should cover. Leave the rest as 404s with a helpful error page.
Can I ask OpenAI or Google to stop citing a URL that doesn't exist?
There's no published request process for that. What you control is what the assistant finds when it fetches your site: a clear 404, a sensible redirect or a real page. Keeping OAI-SearchBot and other search crawlers able to read your site helps assistants cite real pages instead of guessing.
How many AI-referred 404s are normal?
In Ahrefs' September 2025 study, about 1% of clicked ChatGPT referrals landed on 404 pages, against 0.15% for Google. If your share is far above that, look for a structural cause such as an unredirected migration.
Why doesn't Perplexity traffic show up in GA4's AI Assistant channel?
At the time of writing, Perplexity wasn't on the list of referrers GA4 maps to that channel, so its sessions still appear under Referral. Filter by session source as well as by channel.
Where to start with your phantom URL list this week
Give yourself one hour. Build the GA4 exploration from the steps above, export 90 days of AI-referred 404s, and group the paths by pattern. In most audits, one or two rule-based redirects for an old folder structure fix the majority of the problem, and the remaining inventions show you content gaps that real people are asking assistants about.
If the list is long, or your logs show a migration leak that goes well beyond AI referrals, our SEO team can run the full audit, set up the redirects and build the monitoring. Get a quote and tell us roughly how many phantom paths you've found.
Further reading on AI hallucinated URLs
- AI Search for Accountants, Consultants and Professional Services
- AI Search for Salons, Spas and Wellness Businesses
- AI Search Competitor Gap Analysis: See Why Rivals Get Recommended
Sources for this hallucinated URL guide
- Ahrefs: New Study: How Often Do AI Assistants Hallucinate Links? (September 2025)
- Search Engine Journal: AI Search Sends Users to 404 Pages Nearly 3X More Than Google
- Search Engine Journal: Google's Mueller Predicts Uptick of Hallucinated Links (March 2025)
- Semrush: How to Fix ChatGPT 404 Referrals
- Google: How HTTP status codes and network errors affect Google's crawlers
- Search Engine Journal: GA4's AI Assistant Channel Undercounts Your AI Traffic (July 2026)
- OpenAI Help: Publishers and Developers FAQ
