How Many Prompts Do You Need to Track AI Visibility Honestly?

How many prompts do you need to track AI visibility honestly?

Fewer than most tools suggest, but more than one run each. For a B2B company, I start with 30 to 50 prompts that map to real buying questions, and I run each one several times per check. AI answers vary from run to run, so a single run per prompt measures luck. Repeated runs on a focused set measure position.

Every founder I talk to now wants to know whether ChatGPT, Perplexity, Gemini or Claude mention their company. That is a reasonable question. The trouble starts when someone types ten questions into one assistant, sees their brand twice and reports "20 percent AI visibility" to the board.

That number is not wrong because the arithmetic is wrong. It is wrong because the method cannot tell the difference between a real result and a coin flip. This article is about building a prompt set that can.

Why does one run per prompt mislead you?

AI assistants do not return the same answer every time. The same question can produce a different list of vendors on the next run, and small wording changes shift results further. If you run each prompt once, you capture one draw from a range of possible answers. Your score then moves for reasons unrelated to your marketing.

This is the core difference from classic rank tracking. A Google ranking for a keyword is fairly stable over a day. An assistant's answer is generated fresh, often with retrieval in the loop, and the retrieved sources can change between runs.

So treat each prompt as a question you sample, not a position you read. Run it several times, record how often your brand appears, and report that frequency. A brand that shows up in four of five runs is in a different place from one that shows up in one of five, even if a single run made them look identical.

What makes a prompt worth tracking?

A prompt is worth tracking when a real buyer would plausibly ask it and the answer could change who they call. Category questions, comparison questions and problem questions matter most. Branded questions about your own company matter less, because they test accuracy, not visibility. Track buying intent, not vanity.

I sort prompts into four groups. Category prompts ask for options, such as "best tools for lead routing in HubSpot." Comparison prompts name you or a competitor, such as "alternatives to a named vendor." Problem prompts describe a pain without naming a category. Branded prompts ask directly about your company.

Each group answers a different question. Category and problem prompts tell you whether you are in the consideration set. Comparison prompts tell you how you are framed against rivals. Branded prompts tell you whether the assistant describes you correctly. Keep them in separate buckets so one strong group does not hide a weak one.

Where should the prompts come from?

From your buyers, not from a keyword tool. Pull questions from sales call notes, discovery forms, support tickets and the phrases prospects use in emails. Those are the questions people actually bring to an assistant. Keyword lists help you check coverage, but they rarely sound like a real person asking for help.

My favorite source is the "what made you look for a solution" field on a demo form. The answers are written in the buyer's words and describe the moment they started searching. Turn the common ones into prompts almost verbatim.

I covered why prompt-led tracking beats keyword volume for this job in tracking AI prompts instead of keyword volume. The short version is that buyers talk to assistants in sentences, so your measurement should too.

How many runs per prompt are enough?

For a monthly check, I use at least five runs per prompt per assistant, and more for the prompts that matter most. Five runs will not give you precise percentages, but it separates "usually present" from "rarely present" well enough to make decisions. If you need tighter numbers for a board report, increase runs before you increase prompts.

Here is the trade-off in plain terms. More prompts tell you about more questions. More runs tell you more reliably about each question. Most teams overinvest in prompts and underinvest in runs, which gives them a wide, shallow picture full of noise.

A focused set of 30 prompts with five runs each is 150 answers per assistant. That is enough to see patterns, and still small enough to review by hand. Reading the actual answers matters, because the reason you were or were not mentioned is often obvious once you see it.

Which assistants should you include?

Include the assistants your buyers actually use, and keep them separate in your reports. ChatGPT, Gemini, Perplexity and Claude, plus Google AI Overviews and AI Mode, can each give different answers to the same question. Averaging them together hides the one place you are strong or the one place you are missing.

If you only have time for two, ask your customers which they use. A quick question on your onboarding form or in sales calls will tell you more than any industry survey. Then start there.

Keep the settings consistent. Use the same account type, the same location if you can and the same prompt wording each month. Changes in your setup can move results as much as changes in your content, and you want to measure the content.

What should you record for each answer?

Record whether your brand appears, where it appears, how it is described and which sources are cited. Presence alone is too thin. Being mentioned last with an outdated description is not the same as being recommended first with an accurate one. The cited sources tell you which pages to improve.

A simple spreadsheet or an Airtable base is enough. One row per run, with columns for prompt, assistant, date, mentioned yes or no, position, description accuracy and cited URLs. Fill it in by hand at first. Automate only after you know which columns you actually use.

The cited sources column is where the work comes from. If an assistant keeps citing a third-party list that leaves you out, that list is your next outreach target. If it cites your own page but describes you wrongly, that page needs clearer wording. I walked through a lean version of this setup in tracking AI search visibility without enterprise tools.

How often should you change the prompt set?

Rarely. Keep a stable core so you can compare month over month, and add a small rotating group for new questions. If you rewrite the set every month, you lose the ability to see change. I keep about 80 percent of prompts fixed for at least two quarters and review the rest each month.

New products, new competitors and new buyer questions do justify changes. When you add a prompt, mark the date it joined so you do not compare it against months where it did not exist.

Retire prompts that never produce useful answers. If a question has returned the same generic response for three months and nobody on your team would act on it, it is taking up space that a better question could use.

What should you do next?

Write 30 prompts from real buyer language this week, split into category, comparison, problem and branded groups. Run each one five times in the two assistants your customers use most, and record presence, position, description and citations. Then report frequency per group, not a single blended score.

My honest view is that AI visibility is measurable, but only if you respect the noise. A small, stable, repeated prompt set will tell you the truth. A large, single-run set will tell you a story, and it will be a different story every month.

If you want help building a prompt set and a tracking workflow that your team can actually run, reach out. I build websites and the automations behind them for lead generation, and measuring AI visibility honestly is a big part of that work now. Let's chat.