Retrieval-Augmented Generation (RAG) Explained for Marketers

When I explain AI search to clients, someone always asks: “Does ChatGPT just know about my business, or does it look it up?” Many people believe AI tools answer from a giant memory that was frozen when the model was trained. For a lot of questions, that is no longer how it works.

Today, ChatGPT, Gemini, Perplexity, Copilot and Google’s AI Mode usually look things up first and then write the answer. The technical name for this is retrieval-augmented generation, or RAG. It sounds like an engineering topic, and it is, but it also decides which websites get quoted and which brands get named.

Let me explain RAG in plain words, step by step, using one example question all the way through. I lean on a detailed Ahrefs explainer on retrieval-augmented generation, plus documentation from Google and OpenAI. At each step I will tell you what, if anything, your website can do about it.

What retrieval-augmented generation means, in one paragraph

A language model on its own only knows what it learned during training, and that knowledge has a cut-off date. RAG fixes this by adding a search step. Retrieval means the system runs a search. Augmented means it adds the search results to the model’s input. Generation means the model writes an answer using your question plus those results. Think of it like an open-book exam: the student still writes the answer, but now they can flip through a few pages first.

This matters for marketers because the “pages” the model flips through are web pages. If yours is one of them, you have a chance of being quoted. If it isn’t, the model can only use what it remembers from training, which is often older and thinner. For the bigger picture of training data versus live search, see our guide on how AI chooses its sources.

One question, followed through the whole RAG pipeline

To keep this concrete, I will follow one example question through every step. Imagine a user types this into ChatGPT:

“What’s the best project management tool for a 10-person architecture firm?”

Here is the whole journey at a glance. Different platforms use different names and details, but the shape is the same everywhere:

1. Decide whether a search is needed at all.
2. Rewrite the question into one or more search queries.
3. Retrieve candidate pages from a search index.
4. Chunk and embed those pages into small passages the system can compare.
5. Rerank the passages and keep only the best few.
6. Generate the answer and attach citations.

The Ahrefs article quotes Eli Schwartz with a line I like, because it sums up the whole pipeline:

“The vast majority of pages get considered and rejected before the answer is ever written.”

— Eli Schwartz, quoted in Ahrefs, Retrieval-Augmented Generation (RAG) Explained

So let’s look at where those rejections happen.

Step 0: does the AI even need to search?

Before anything else, the system decides whether to search. Google’s Gemini API documentation describes this directly: when search grounding is switched on, the model first analyzes the prompt and decides whether a Google Search would improve the answer. If yes, it writes and runs one or more queries. If no, it answers from what it already knows.

Ahrefs describes the same idea as a classifier that sorts questions into “no search,” “simple search” or “multi-step search.” A question like “what is the capital of France” rarely needs a search. Our architecture firm question almost certainly does, because it asks for a current recommendation.

What you can influence here: very little directly. But it tells you something useful about keyword choice. Questions about recent, local, commercial or comparison topics tend to trigger searches. Those are the questions where your content can be pulled in. Evergreen definitions are more likely to be answered from memory.

Step 1: the question gets rewritten into several searches

AI tools rarely search for your exact words. They rewrite the question into shorter, sharper search queries. OpenAI says this openly in its help center:

“ChatGPT search typically rewrites your query into one or more targeted queries that it sends those providers.”

— OpenAI Help Center, ChatGPT search

Google calls its version query fan-out, which its documentation describes as “issuing multiple related searches across subtopics and data sources.” For our example, the fan-out might look something like this (my illustration, not a real log):

• best project management software for architecture firms
• project management tools small architecture practice 2026
• architecture firm project management software reviews
• project management software with time tracking and invoicing for architects

What you can influence here: quite a lot. If your page answers the sub-questions, not just the head term, it can be found by several of these searches. Ahrefs reports from a study of 1.4 million ChatGPT prompts that cited pages had titles more semantically similar to ChatGPT’s fan-out queries. I go deeper on this in how ChatGPT query fan-out works and our older guide to writing content for fan-out queries.

Step 2: retrieval from a search index

Next, those queries go to a search index. This is the part many marketers miss: an AI tool can only retrieve pages that are in the index it uses. And each platform uses something different.

Platform	Where it retrieves from (as documented or reported)	Bot that fetches pages
Google AI Overviews / AI Mode	Google’s own Search index and ranking systems	Googlebot
Gemini (API grounding)	Google Search, through the grounding tool	Google’s crawlers
ChatGPT search	Search partners plus OpenAI’s own crawling; OpenAI links to Microsoft’s privacy statement for how providers process queries	OAI-SearchBot, ChatGPT-User
Perplexity	Its own index	PerplexityBot, Perplexity-User

Sources: Google AI features, Gemini API grounding docs, OpenAI help center, OpenAI crawlers, Perplexity bots (accessed October 2026).

From the retrieved results, the system builds a shortlist. Ahrefs lists the usual filters as relevance, authority, recency and diversity. That last one is interesting: the system often wants sources from different sites, so ten pages from one domain rarely all make the cut.

Recency is real. Ahrefs analyzed 17 million citations and found that AI-cited URLs are 25.7% fresher than organic Google results on average (December 2025).

What you can influence here: this is classic SEO. Your page needs to be crawlable, indexed and ranking reasonably well for the rewritten queries. If you block OAI-SearchBot or PerplexityBot by accident, you are out at this step. Our guide to how ChatGPT’s search index works explains the OpenAI side in more detail.

Step 3: chunking and embeddings, or how a page becomes passages

The model can’t read 30 full web pages for every question. It would be too slow and too expensive. So the retrieved pages are cut into smaller pieces called chunks, usually a few paragraphs each.

Each chunk is then turned into an embedding. An embedding is a long list of numbers that represents the meaning of the text. Texts with similar meanings get similar numbers. The system also turns the search query into an embedding, then measures how close each chunk is to the query, usually with a measure called cosine similarity. A score near 1 means “very similar meaning.” I explain this in more depth in embeddings and vector search for SEOs.

A simple example. Suppose your page about project management software has five chunks:

Chunk on your page	Illustrative similarity to “best PM tool for small architecture firm”	Likely outcome
Intro about company history	0.41	Ignored
Generic “what is project management”	0.55	Probably ignored
“How architecture practices use our tool for drawings, fees and client sign-off”	0.86	Strong candidate
Pricing for teams of 5–20	0.72	Possible supporting chunk
Customer quote from a 9-person studio	0.79	Possible supporting chunk

The numbers above are made up for illustration. The point is that the system judges your page passage by passage, not as a whole. One focused, self-contained section can win even if the rest of the page is general.

Should you rewrite your pages into tiny chunks?

No, and Google says so directly. In its July 2026 guide to generative AI features, it lists chunking among the things you can ignore:

“There’s no requirement to break your content into tiny pieces for AI to better understand it.”

— Google Search Central, Guide to optimizing for generative AI features

My reading: you don’t need to chop pages up for machines, but clear sections with descriptive headings, where each section makes sense on its own, are simply good writing. They help readers skim, and they also happen to make clean passages. Our post on content chunking for AI covers the debate in more detail.

Step 4: reranking, where most pages are rejected

The similarity search produces a long list of candidate chunks. A second, more careful model, often called a reranker, then reorders them and keeps only the best handful. This is the step where good-but-generic pages lose to specific ones.

What seems to help a chunk survive reranking? The evidence is still young, so I treat these as signals, not laws. From the studies Ahrefs gathered:

• Position on the page. Kevin Indig’s analysis of 1.2 million ChatGPT citations found 44.2% of citations came from the first 30% of a page, 31.1% from the middle and 24.7% from the bottom third.
• Question-then-answer structure. The same analysis found content that asks a question and answers it straight away was cited at 18%, versus 8.9% without it.
• Specific facts and entities. Cited content had about 20.6% entity density (names of products, places, people) versus 5–8% in average content.
• Unique information. The Princeton-led Generative Engine Optimization paper reported a 30–40% visibility uplift from adding things like quotes and statistics.

All four of these figures come via the Ahrefs RAG explainer; I have not re-run the analyses myself.

What you can influence here: put your best answer near the top, open sections with a direct answer, name specific things, and include facts nobody else has. A sentence like “Most 10-person architecture practices need fee tracking, drawing version control and client approval in one tool” beats “Every business is different.”

Step 5: generation and citation

Finally, the chosen chunks are placed into the model’s context window (its short-term working memory) along with the original question. The model writes the answer and adds citations. In the Gemini API, for example, each citation is attached to a specific span of the answer text and points to a source URL.

Two things surprise people at this step:

1. The retrieved content is thrown away afterwards. Ahrefs notes that the raw content is discarded once the response is generated. Being retrieved today doesn’t mean the model “learned” your page.
2. Cited is not the same as recommended. A page can be used as a source while the answer recommends a competitor. Ahrefs saw this clearly in its self-promotional content experiment; I discuss it in my post on ranking yourself first in your own lists.

The answer also varies from run to run. In a study of 43,000+ keywords, Ahrefs found consecutive AI Overviews scored about 0.95 cosine similarity, meaning the meaning stays stable, while only about 54.5% of cited URLs carried over between responses (November 2025). The gist stays; the sources rotate.

Where your content can make a difference at each step

Here is the whole pipeline again, this time with the marketing work that fits each step.

Step	What happens	How much you can influence it	What to do
0. Decide to search	Model judges if it needs fresh info	Low	Target commercial, local, comparison and “current” questions
1. Rewrite	Question becomes several searches	Medium	Cover sub-questions; use clear, descriptive titles
2. Retrieve	Search index returns candidates	High	Classic SEO, crawl access for AI bots, real freshness
3. Chunk + embed	Pages split into passages and scored	Medium	Self-contained sections with descriptive headings
4. Rerank	Best few passages are kept	Medium–high	Answer first, specific facts, original data
5. Generate	Answer written and cited	Indirect	Build brand trust and mentions elsewhere so you are recommended, not just cited

Notice that step 5 depends on what the wider web says about you. That is why I always pair page work with off-site work. Our AI search strategy framework shows how to plan both together, and our content writing service can handle the page side.

Five things people get wrong about RAG

• “AI learns from my page when it cites it.” Not in the moment. Retrieved text is used for one answer and then discarded. Training is a separate process.
• “I need a special file for AI to find me.” Google says its Search ignores llms.txt files. Retrieval relies on normal crawling and indexing.
• “RAG means rankings don’t matter.” The opposite. Retrieval starts from a search index, so ranking well for the rewritten queries is the entry ticket.
• “All AI tools see the same web.” They don’t. Each uses its own index and bots, which is why you can appear in Perplexity and not in ChatGPT.
• “RAG is only for search engines.” Many business tools use it too. Claude Projects, for example, automatically switches on RAG on paid plans when your uploaded knowledge gets close to the context limit. If you are setting up an AI marketing assistant, the same ideas apply to your own documents.

Questions marketers ask about RAG and AI citations

Is RAG the same as query fan-out?

No. Fan-out is one part of RAG: the step where the question becomes several searches. RAG is the whole process from search to written answer.

Can I see which passage of my page was used?

Sometimes. Many AI tools highlight or link the cited section, and some developer APIs return exact text spans. For normal users, check which URL was cited and look for the passage that matches the answer.

Does page length matter for RAG?

Not by itself. Google says page length should suit your audience and topic. What matters is that the relevant section is clear and specific.

Why was my page cited yesterday but not today?

Because retrieval and reranking run fresh for every answer. Ahrefs found only about 54.5% of cited URLs carry over between consecutive AI Overviews. Track averages over time, not single answers.

Do I need to change my website platform for RAG?

No. You need fast, crawlable pages with clear HTML. If your content only appears after heavy JavaScript runs, some AI fetchers may not see it, which is worth checking.

How to use RAG thinking on your own site this month

In brief: AI tools search, cut pages into passages, keep the best few and write from them. You win by being in the index, having a passage that answers the exact sub-question, and being a brand the wider web already trusts.

Here is how I would start. Pick your five most important commercial pages. For each, write down the sub-questions a buyer would ask, check that each one has its own clear section with a direct answer near the top, and add one fact only you can provide. Then confirm AI search bots can reach those pages.

If you would like us to do this with you, our team can audit your pages for AI retrieval and fix what is holding them back. See our SEO services or get a free quote and tell us about your site.

Further reading on retrieval-augmented generation

• The Websites ChatGPT Cites Most (and What That Means for You)
• The Websites Google AI Overviews Cite Most
• Ranking Yourself #1 in Your Own 'Best' Lists: Does It Work in AI Search?

Sources for this retrieval-augmented generation guide

1. Ahrefs – Retrieval-Augmented Generation (RAG) Explained (Louise Linehan, July 9, 2026)
2. Google Search Central – AI features and your website (updated December 10, 2025)
3. Google Search Central – Guide to optimizing for generative AI features (updated July 10, 2026)
4. Google AI for Developers – Grounding with Google Search (accessed October 2026)
5. OpenAI Help Center – ChatGPT search (accessed October 2026)
6. OpenAI – Overview of OpenAI crawlers (accessed October 2026)
7. Perplexity – Perplexity crawlers (accessed October 2026)
8. Ahrefs – How often AI Overviews change (November 11, 2025)
9. Ahrefs – Fresh content and AI citations (December 22, 2025)
10. Claude Help Center – What are projects? (accessed October 2026)