Perplexity: how it finds and cites sources

Spencer Thursfield · Last reviewed 2026-09-10 · Evidence: Perplexity documentation and this site's own server logs

Perplexity is an AI search engine that answers a question in prose and attaches numbered citations to the web pages the answer was built from. It searches the web at the moment the question is asked rather than answering from memory, so the pages it can reach at that moment are the only pages it can cite.

What is Perplexity and how does it choose sources?

Perplexity's help centre sets out four steps: interpret the question, search the web, compile the relevant material into one answer, and cite each source with a numbered link back to the original. It describes content as sourced from the web in real time as the question is asked. Its Research mode does the same at greater depth: Perplexity says it performs dozens of searches automatically and reads hundreds of sources.

What Perplexity publishes covers access, not ranking. The company documents how to let its agents in and how to keep them out. It does not document what decides which reachable page earns the citation. That is a different gap from ChatGPT's: OpenAI at least states that its ranking considers reliability and usefulness and that inclusion cannot be guaranteed (see appearing in ChatGPT search). Perplexity states neither, so we claim neither. Not published

What is the difference between PerplexityBot and Perplexity-User?

Perplexity runs two named agents with two purposes, and says explicitly that neither collects content for AI foundation models. This is the search-index versus user-fetch distinction set out in crawler controls: search access is not training access, drawn by the operator itself.

Two consequences follow, and they point in opposite directions.

1. PerplexityBot governs whether you are in the index at all. Perplexity recommends allowing it in robots.txt and permitting requests from its published addresses. Official recommendation
2. A robots.txt rule aimed at Perplexity-User will not reliably stop it, by the operator's own wording. If a page must stay out of an answer, robots.txt is the wrong instrument. Documented limit

Two operational details are easy to miss. Perplexity says each setting works independently and that a change may take up to 24 hours to reach its systems, so a robots.txt edit checked an hour later proves nothing. And its guidance for firewalls says to match on user agent and address together, because the name is forgeable. Those lists move: fetched on 10 September 2026 they held eight address blocks for PerplexityBot and four for Perplexity-User, and Perplexity warns to re-read them rather than copy them once. A firewall or CDN rule that blocks “bots” in bulk excludes Perplexity silently, before robots.txt is ever consulted. See can AI reach your site at all?

What makes a page eligible?

Only reachability is documented. Being fetchable by PerplexityBot, and indexable once fetched, is the floor. Nothing Perplexity publishes turns that floor into an advantage over anyone else who has also cleared it.

Our own census supports the second half. Of the 32 most-cited domains in one client's cited set, 28 permit all nine major AI agents, PerplexityBot among them. Where a publisher does differentiate, it differentiates by purpose: Medium disallows the training collectors GPTBot and ClaudeBot while permitting PerplexityBot and the other search agents. Opening a door your competitors never closed is not a lever. The full sample and its limits are in who is actually open to AI crawlers?

That is why the folklore ledger records “open your robots.txt to the AI crawlers and you will get cited” as context-dependent rather than true. A page Perplexity cannot fetch cannot be cited. A page it can fetch has cleared an obstacle, not earned a reason. What might supply the reason is the subject of what makes a page citable and how AI search finds sources.

How do you know if Perplexity has read your site?

Your server log holds the answer and is the only instrument that does. Crawlers do not run JavaScript, so analytics never sees them. Look for the two user-agent strings, then check the requesting address against the list Perplexity publishes for that agent. The name alone proves nothing.

On this site, over the window 3 August to 10 September 2026 and across 29 pages, PerplexityBot made 6 verified fetches of 5 distinct URLs. Perplexity-User made none. In the same window, 208 requests presented the name PerplexityBot from addresses outside the published range and were not counted. The full table, with the other operators alongside, is in what we have measured so far.

Read that as a floor, and read it as small. Six fetches is a thin index of a young site. Zero live fetches means no Perplexity question has yet pulled a page from here. It answers the access question and no other: a crawl proves a page was read, never that it was cited. The method is set out in reading your crawler logs.

What is not known

• What decides ranking among the pages Perplexity can reach. Nothing is published, and we will not infer it from a handful of answers.
• Whether index depth relates to citation at all. We hold no evidence either way and make no claim.
• What “generally ignores robots.txt” excludes. The wording admits exceptions and the exceptions are not listed.
• Who sent the 208 requests wearing PerplexityBot's name. We report the count and stop there.

Whether Perplexity cites small-business sites, and how stable those citations are, is what our study exists to answer. The pre-registered method is at method and pre-registration.