ChatGPT writes new searches every time it answers. A University of Toronto study found that between two runs of the same question on the same day, ChatGPT's searches overlapped by 2.7% and the pages it cited by 15%.

So a single answer is a sample. Whether your firm appears in one screenshot says little. How often it appears across many runs, over several weeks, says much more.

What happens when you ask

The Toronto researchers replayed every question in their study through ChatGPT's API and recorded the searches it ran. ChatGPT sent about four keyword-style searches at a time and kept 43% of the question's words. It added the current year to 23% of those searches. It added a brand the user had not named to 28%. The replay used the API and small models, which may not behave like the ChatGPT app.

Those searches change every time. The pages they return change with them, and ChatGPT cites only from what comes back.

Other engines behave the same way. Bill Widmer ran 72 B2B buyer questions twice, two days apart, in a study published by Orbit Media. Claude kept 30% of the pages it had cited. Gemini kept 38% of the websites. Perplexity was the steadiest, at 65%.

It shifts over weeks as well

The Toronto team also measured how much of each site's citation pattern carried over from one fortnight to the next. For ChatGPT, 40% still held two weeks later. Ten weeks later, 14% did. For Perplexity, nothing carried over after ten weeks.

The broad mix of sources moves far less. OtterlyAI tracked Claude's citations and found that no type of source, such as company websites or news sites, moved by more than about one percentage point between the first five days and the last five of its study. The individual pages changed. The kinds of pages Claude chose did not.

Brand recommendations shuffle too

Rand Fishkin of SparkToro asked 600 volunteers to run 12 recommendation questions through ChatGPT, Claude and Google's AI, 2,961 times in all. For ChatGPT and Google's AI, there was less than a 1 in 100 chance that two answers would name the same set of brands. The same brands in the same order came up about once in 1,000 runs.

Even the length of the list changed. Some answers named two or three brands, and others named ten or more.

But some brands kept turning up. City of Hope, a cancer centre, appeared in 69 of 71 ChatGPT answers to one cancer-care question. It came first in only 25 of them.

Fishkin also found that people rarely phrase the same need the same way. Yet across 994 answers to 142 human-written prompts about headphones, the leading brands appeared in 55-77% of answers.

His conclusion is that the share of answers a brand appears in, across many prompts run many times, is a reasonable measure. A brand's position in a list is "full of baloney", in his words.

The study has limits. It ran in November and December 2025 and was not peer reviewed. It used 12 prompts, and participants ran them on their own default settings. Fishkin's co-researcher, Patrick O'Donnell, works for an AI tracking company, and Fishkin flags that conflict himself.

What it means for checking your own visibility

A single answer proves nothing in either direction. If a colleague sends you a screenshot where your firm is missing, ask the same question a few more times before you draw a conclusion.

Ignore where you sit in a list. On SparkToro's figures, the order barely repeats.

Track the share of answers that name you. Run the same buyer questions repeatedly, in the AI tools your buyers use, and judge the trend on at least three weeks of data, which is Widmer's advice. We cover tools that do this in what marketing leaders use to monitor brand visibility in ChatGPT and Perplexity.

If you pay someone to track this, ask how many times they run each question and whether they report a share or a single result. Fishkin's advice to buyers is to make providers show their maths.

In my own sprints I track the same prompts every week across eight AI platforms. In my view, a change that only shows up in one week's run is not yet a change.

A few common follow-up questions

Why did ChatGPT recommend us yesterday but not today?

Each answer comes from a fresh set of searches. The Toronto replay found only a 15% overlap in ChatGPT's cited pages between two runs on the same day. One missing appearance is not a trend.

How many times should I run a question?

Nobody has settled that. SparkToro lists it as an open question. Each prompt in its study was run 60-100 times, and Widmer advises at least three weeks of data before you judge a trend.

Does it matter if we come first in ChatGPT's list?

Not much on current evidence. In SparkToro's study, the same order came up about once in 1,000 runs. How often you appear is the steadier number.