Ask for the starting number first. Then find out what was measured and who measured it, and whether the result lasted. A case study that answers those questions is evidence. One that only gives a percentage is marketing.

The same seven questions work for any AEO provider.

1. What was the starting number?

A percentage on its own hides the scale. A rise from 20 users a month to 940 is a 4,600% increase. So is a rise from 2,000 to 94,000.

Ask for the absolute figures at the start and at the end. A provider who tracked the work will have them.

2. What exactly was measured?

"AI visibility" can mean several different things. An AI tool can mention your firm, recommend it, or cite one of your pages as a source. Those numbers move separately.

Bill Widmer tracked 72 B2B buyer questions across three brands for a study published by Orbit Media. Claude mentioned the brands in 70% of answers but cited their pages in only 40%. For ChatGPT the figures were 56% and 47%.

Ask too whether the prompts named the brand. A question with your firm's name in it tests whether the AI knows you, not whether it would suggest you to a new buyer. Non-branded prompts, where the buyer has not heard of you yet, are the ones that show new demand.

3. How many times was each prompt run, and over how long?

AI answers change from one run to the next. A University of Toronto study found that two runs of the same question in ChatGPT on the same day, in its API replay, shared only 15% of their cited pages.

Brand lists shuffle as well. SparkToro found less than a 1 in 100 chance that two ChatGPT answers would name the same set of brands. Rand Fishkin's conclusion was that the share of answers a brand appears in, across many prompts run many times, is a reasonable measure.

So a result from one run, or one good week, tells you little. Ask how many prompts were tracked and how often each ran. Then ask whether the headline figure is a peak or an average.

4. Who measured it, and with what?

Agency case studies are measured by the agency that did the work, often with its own tracking tool. That doesn't make the numbers wrong. It does mean you should ask to see the method.

Be wary of any tool that claims inside knowledge of how an engine ranks. Google's guide to its AI features says plainly that "no third-party tool has access to our internal ranking or AI systems".

5. How were leads or users linked to AI?

AI influence is hard to count. Similarweb found that 55.9% of visits influenced by a ChatGPT recommendation arrived through search, so analytics logged them as search traffic, not AI. We go through that study here.

A claim about users or leads "from AI search" needs a stated method. It might be referral data or a "How did you hear about us?" field on the enquiry form. Each counts something different.

6. What else changed at the same time?

A firm that raised funding or ran a large campaign during the work would have grown anyway. Case studies rarely separate the agency's effect from the rest of the business.

Ask what else the client was doing in that period. An honest provider will say where the line is unclear.

7. Did it last?

Citation patterns drift. In the Toronto data, only 14% of a site's ChatGPT citation pattern still held ten weeks later.

Ask whether anyone checked the numbers again after the work stopped. A result that holds a month later says more than one measured on the last day.

What a good case study looks like

It gives absolute numbers at the start and end. It names the prompts, or at least the type of prompt, and says how often they ran. It separates mentions from citations. It says who measured the result and how users or leads were counted, and it says what the result does not prove.

In my experience, few case studies do all of that. When you speak to a provider, ask the questions anyway. We cover what to ask before you sign in 5 questions to ask before hiring an AEO agency.

A few common follow-up questions

Is a big percentage a bad sign?

No. Large percentages are normal when a firm starts from almost nothing. A percentage only misleads when the absolute figures are missing.

Should I trust a case study measured by the agency's own tool?

You can, if the method is clear. Ask how many prompts it tracks and how often it runs them. Ask to see the raw results too.

What is the single most useful question?

What was the starting number, in absolute terms? It tells you the scale of the result, and whether the provider tracked the work closely enough to know.