Often enough that one check is a snapshot, not a measurement. In Reflexa's own AI Visibility Index, the same 100 buyer questions asked three times each across ChatGPT, Claude and Perplexity already show the mention-and-citation gap moving between runs, before a week even passes.
Because ChatGPT, Claude and Perplexity build each answer fresh, rather than retrieving it from a fixed, cached list. Research on repeated-prompt trials finds that answer-consistency rates for language models vary widely by model and are routinely well short of 100%, even with randomness settings turned down, and a buyer question that asks a model to shortlist companies has even more room to move.
Proof: arXiv — The Non-Determinism of LLMs: Evidence of Low Answer Consistency in Repetition Trials (2025).
Bottom line: if the channel itself is non-deterministic, a single reading isn't a measurement, it's one sample from a distribution you haven't seen the rest of yet.
Even within three runs of the same 100 questions, the gap between being mentioned and being cited moved. The Reflexa AI Visibility Index's September 2026 project-management edition asked its fixed set of 100 buyer questions three times each across ChatGPT, Claude and Perplexity, 900 observed answers in total. Brands mentioned in 27-51% of eligible answers were cited on their own domain in only 6-18% of those same answers, and that 21-33 point gap itself shifted between the three runs, not just between brands. The full method and evidence log is covered in what an AI visibility audit should show you, and the wider set of adoption and behavior figures behind it sits in AI Search Statistics 2026.
| Measurement | Observed range across runs | What moved |
|---|---|---|
| Mentioned (named by the engine) | 27-51% of eligible answers | Varied by brand and by run |
| Cited (own domain linked as a source) | 6-18% of those same answers | Varied by brand and by run |
| Mention-to-citation gap | 21-33 points | Shifted between the three runs |
Proof: Reflexa AI Visibility Index, project management edition, September 2026 (100 buyer questions × 3 engines × 3 runs, 900 observed answers). Full method in how do I measure AI visibility.
Why it matters: three runs on 100 questions is a small window, and the numbers still moved inside it; stretch that same question set across weeks and the movement is the normal state, not an anomaly to explain away.
Three things can move your numbers without you changing anything: the model itself, the sources it draws on, and your competitors' content. AI labs update and fine-tune models on their own schedule, shifting which sources an engine treats as authoritative. Engines with live web search re-crawl pages continuously, so a competitor's new page can enter the answer pool within days. And the model's own non-determinism means it rarely returns an identical answer to the same prompt twice.
Do this: before you act on a single bad answer, rule out that it was just this run, not a real shift.
At least three runs per engine for a single check, and several consecutive weekly checks before calling anything a trend. The Reflexa AI Visibility Index runs every question three times across all three engines for exactly this reason: three runs on one engine only tells you whether that engine's own answer is stable, not whether another engine would name someone else entirely for the identical question. That same discipline, extended across weeks instead of one sitting, is covered in how to track AI mentions week over week.
Bottom line: each level answers a different question, and skipping a level leaves a blind spot the next level would have caught.
A slide while it is still one or two bad answers, instead of an entrenched pattern nobody noticed for weeks. A monthly check is better than none, but it lets the same problem run for a month before anyone looks again; a weekly check of the same fixed question set catches the same slide within days. This is the same discipline behind AI brand intelligence more broadly: a number is only useful tracked over time.
Do this: fix your question set and your engines before your first check, so every later check is comparing like with like instead of noise.
None of the steps below need a purchase:
Bottom line: the question isn't whether this week's answer will move, it will; the question is whether you're checking often enough to tell a real shift from noise.
Sources: arXiv — The Non-Determinism of LLMs (2025) · Reflexa AI Visibility Index, September 2026 · Reflexa — how often should you re-check your AI visibility
Because ChatGPT, Claude and Perplexity generate each answer fresh rather than pulling it from a fixed list, and research on repeated-prompt trials finds that answer-consistency rates for language models vary widely and are routinely well short of 100%, even with randomness settings turned down (arXiv, 2025).
At least three times per engine. The Reflexa AI Visibility Index asks every buyer question three times across ChatGPT, Claude and Perplexity for this exact reason, because a single reply is one sample from a distribution, not a measurement.
Not from one run. Treat a single missing mention as noise until it repeats across consecutive weekly checks of the same fixed question set; only a pattern across several runs is a real move.
Three things, mostly outside your control: the underlying model gets updated or fine-tuned, engines with live web search re-crawl pages so a competitor's new page can enter the pool within days, and the model's own non-determinism means it rarely returns an identical answer to the same prompt twice.
A one-time check only tells you what the engines said on that one day. Weekly tracking of the same fixed question set catches a slide while it is still one or two bad answers, instead of letting it run for weeks before anyone notices.
The free check reads your pages the way ChatGPT, Claude and Perplexity read them — 23 structural checks, evidence attached, no card needed.