How AI search finds sources

Spencer Thursfield · Published 2026-08 · Evidence: platform documentation

Every AI search answer is the end of a pipeline: retrieve, select, generate, cite. Most advice fails because it aims at the wrong stage. This article walks the pipeline as the platforms themselves describe it.

Stage one: retrieval

Google's generative features have no index of their own. They run on Search's core ranking systems and retrieve candidate pages the way Search always has. Official recommendation ChatGPT search operates its own crawler, OAI-SearchBot, and rewrites your question into search queries before retrieving. Perplexity does likewise. The practical consequence: if conventional search cannot find you, AI search cannot cite you. There is no separate "AI index" to optimise for.

Stage two: selection

From the retrieved candidates, the system chooses a handful of sources to ground its answer. Folklore obsesses over this stage, and the platforms say least about it. What Google does publish: it wants distinctive, expert-led, non-commodity material, and no special markup influences this stage. Official recommendation What research observes: selection varies across systems, across phrasings, and across repeated runs of the same question. Emerging

Stage three: generation and citation

The system writes the answer from the selected sources. Being cited and being used remain separate outcomes: a source can appear in the citation list while contributing little, or shape the answer from a lowly position. Our study measures the two separately; see why citation counts aren't enough.

What this means in practice

• Fix retrieval first. It is checkable and binary. See can AI reach your site at all?
• Compete at selection with material only you could publish. See commodity vs non-commodity.
• Treat any promise about the selection stage with suspicion. That stage is where the folklore lives.