Prompt-set testing: measuring AI visibility without fooling yourself

Spencer Thursfield · Published 2026-08 · Evidence: our own protocol, published

The obvious way to check AI visibility is to ask ChatGPT about your business and see what happens. One run measures noise, and you will read the noise as signal. The discipline below turns prompt testing into an instrument.

The four rules

1. Fixed prompt set. Write the questions once and keep them unchanged between rounds. Use real customer questions rather than your brand name alone, and include questions you should appear for but fear you don't. If the questions change between rounds, the rounds stop being comparable.
2. Fresh sessions. Conversation history contaminates retrieval. Every prompt starts a new chat. A logged-in account that knows you skews everything.
3. Repetition. The same prompt, on the same engine, on the same day, can cite different sources. Run each prompt at least three times and record the variation; the variation is itself a finding.
4. Record everything at observation time. Answers change and cited pages change. Store the full answer, the citation list and its order, and a screenshot. Anything you fail to capture at the time cannot be recovered later.

What to count

Count rates rather than a score: how often you were mentioned; how often you were cited (linked as a source); how often a competitor or directory took the slot instead; and how stable each rate held across repeats. Track those four rates over months. They carry more information than any composite index. Strongly supported

The honest baseline

Expect zeros. A small business measuring for the first time tends to find AI answers built from directories, editorial sources and forums rather than from its own site. A zero measured honestly gives you a usable baseline, and our own benchmark begins the same way, at scale, with the method published first.