What Building RAG Systems Taught Us About How AI Actually Reads Your Content

Most advice on getting cited by AI search is written from outside the black box. We build the inside: retrieval pipelines that cut documents into passages and hand a few of them to a language model. Here is what that pipeline keeps from a page, what it throws away, and a five-point test you can run this week.

The question we hear most from marketing teams is “how do we get cited by ChatGPT?” It is the wrong question, because nobody outside those companies can answer it, and anyone who gives you a precise answer is guessing.

The better question is one we can answer from the builder’s side: when a retrieval system reads your page, what does it keep, and does it still make sense on its own? We have built retrieval over internal knowledge bases where every answer must cite its source document: no retrieved passage, no generated answer. That taught us more about how AI reads content than any amount of prompting public chatbots.

The model reads passages, not pages

Retrieval augmented generation (RAG) is the pattern behind every AI product that answers from documents and shows citations. The pipeline has the same shape wherever you find it:

1. Fetch the document.
2. Extract the text and strip what looks like boilerplate.
3. Split the text into passages (chunks), usually a few hundred words each.
4. Embed each passage as a vector, a numerical representation of its meaning.
5. Retrieve the handful of passages whose vectors sit closest to the user’s question.
6. Generate an answer from those passages only, and cite them.

Public AI search products are far more elaborate and do not publish their internals, but any system that grounds answers in cited sources works on this principle. The consequence is the most important idea in this article: the model never sees your page. It sees three or four passages cut out of it, sitting next to passages cut out of your competitors’ pages.

The passage is the unit of competition. Your page can be the best on the topic and still lose, because its best paragraph was split in the wrong place.

Context does not survive the cut

Writers rely on the reader having read everything above the current paragraph. A chunker cuts wherever its rule says, and each passage then lives alone in the index. A sentence such as “This only applies to annual plans” is perfectly clear on the page. Retrieved on its own, “this” refers to nothing, and a model can apply the condition to the wrong thing entirely.

On our side of the pipeline, the fix is to split on headings and to prepend the heading path to every passage, so each chunk carries its own context. You can see the idea, and test your own pages with it, in a short Python script:

# pip install requests beautifulsoup4
import requests
from bs4 import BeautifulSoup

def passages(url):
    # Raw HTML, no JavaScript: roughly what a non-rendering crawler receives
    html = requests.get(url, timeout=10).text
    soup = BeautifulSoup(html, "html.parser")
    for tag in soup(["script", "style", "nav", "header", "footer", "aside"]):
        tag.decompose()

    path, buf = [], []
    for el in soup.find_all(["h1", "h2", "h3", "p", "li"]):
        if el.name in ("h1", "h2", "h3"):
            if buf:
                yield " > ".join(path), " ".join(buf)
            level = int(el.name[1])
            path = path[: level - 1] + [el.get_text(" ", strip=True)]
            buf = []
        else:
            buf.append(el.get_text(" ", strip=True))
    if buf:
        yield " > ".join(path), " ".join(buf)

for heading, text in passages("https://example.com/your-page"):
    print(f"[{heading}]\n{text[:400]}\n")

Run it against one of your own pages and read the output as if you had never seen the site. That is roughly the view a retrieval system has.

You cannot count on every crawler being as careful as this one, so write for the worst case:

• Name the subject in each section instead of pointing back to it. “Annual plans can be refunded within 30 days” survives a cut. “They can be refunded within 30 days” does not.
• Replace “as mentioned above” and “see below” with the thing itself.
• Repeat the noun. What reads as slightly repetitive on the page reads as complete in a chunk.

Headings are the index

Once we started attaching headings to passages, heading quality showed up directly in retrieval quality. A clever heading such as “Where the magic happens” contributes nothing to that match. “How refunds work for annual plans” contributes the exact words a question is likely to contain.

“Use descriptive headings” is old SEO advice. The difference now is that the heading travels with each passage underneath it. Treat every H2 and H3 as the label on a card that will be pulled from the deck and read alone.

Answer first, or the answer gets diluted

An embedding represents a whole passage as one vector. Loosely, it averages the meaning of everything in the chunk. A passage that spends most of its words on preamble and one sentence on the answer embeds as preamble.

We saw this in internal documents: a thorough document that contained the right answer could lose retrieval to a shorter, blunter one that answered the question in its first line, even when the shorter one was less complete.

The rule is old newspaper practice, applied per section: the first two sentences under a heading should answer the question the heading asks. Background, nuance and caveats come after.

Your words versus your reader’s words

Retrieval matches the question to the passage. When the people asking and the people writing use different words, retrieval gets harder.

In our internal systems, staff asked in plain language and the documents were written in product codes, team names and acronyms. Semantic search bridges ordinary synonyms reasonably well, but an embedding model has no reason to know that your proprietary term means what the user typed.

Websites have the same problem. Your branded name for a feature is not what customers search for. “Our SmartSync engine” means nothing to a model trying to answer “does this tool sync my calendar both ways?” Define branded terms in plain language on first use, in the same sentence: “SmartSync, our automatic two-way calendar sync, …”

What never reaches the model

Some content never reaches the index at all. These were the usual losses:

• Content rendered only by client-side JavaScript. Most of our ingestion did not run a browser, because rendering every page in a headless browser is slow and expensive at scale. Public AI crawlers face the same economics. A December 2024 analysis by Vercel and MERJ found that the major AI crawlers they observed did not execute JavaScript, with Google’s Gemini and AppleBot as exceptions. Check whether that still holds for the crawlers in your own logs, but do not assume otherwise.
• Text inside images. A pricing chart or an infographic is invisible unless the same facts are in the surrounding text.
• Content behind tabs and accordions that loads only on click.
• PDFs without a text layer, such as scanned brochures.
• Tables, flattened. A table often comes out as a run of values with the column headers somewhere else. A price with no label next to it is useless to a model.

The quick test is “view source”, not “inspect element”. If the fact is not in the raw HTML, assume a retrieval pipeline cannot see it. For tables that carry important facts, state the key rows in a sentence nearby as well.

Stale pages compete with your current ones

When two of our source documents disagreed, such as an old process and its replacement, retrieval returned both. The model then blended them or picked the one worded more clearly, which was not always the current one.

We fixed it by dating documents, marking superseded ones and removing them from the index. Your website needs the same hygiene. A 2021 post that says the opposite of your 2025 pricing page is not harmless archive material. It is a live competitor in the retrieval set, and it may be the clearer of the two. Consolidate, redirect, and put a visible “last updated” date on pages whose facts change.

Specific claims get cited, vague ones do not

Because every answer in our systems had to cite a passage, we could see which passages earned citations. They were the ones containing something checkable: a number, a name, a condition, a definition.

“We offer industry-leading support” never answers anyone’s question. “Support replies within one business day, Monday to Friday, in English and German” answers several. If you want a passage cited, make it the passage that states the fact.

The passage test

Take any H2 section of an important page, paste it alone into a blank document, and check five things:

A section that fails two or more is a section a retrieval system will struggle to use, however well the page ranks.

Where this does not apply

• They do not replace ranking. Public AI search products generally retrieve candidates from a search index first. Authority, links and crawlability still decide whether your page is in the candidate set. Passage quality decides what happens after that.
• Do not shred your pages into fragments. Very small chunks lose context in the other direction. In our systems, tiny passages sometimes retrieved well and answered badly, because the reason behind the fact sat in the next paragraph. Sections of a few solid paragraphs work better than FAQ confetti.
• Do not write for embeddings. Stuffing a passage with keyword variations dilutes its vector the same way preamble does.
• Nobody outside the AI companies knows their chunk sizes or ranking. Anyone quoting you an exact token count for Perplexity or AI Overviews is guessing. What transfers is the principle, not the parameters.

What to do on Monday

Pick your five most important pages. View the source of each and confirm the key facts are in the raw HTML. Run the passage test on every H2 section. Fix headings and opening sentences first: they are the cheapest changes, and the parts of a page a retrieval system leans on hardest.