Which Questions Should You Test to Really See Where You Stand in AI Search?

Why Is the Search Results List No Longer Enough?

In traditional search engine optimization, we are used to our position being a number that we monitor every week. In AI search, however, there is no such thing as “the tenth result.” There is an answer that mentions three or four names, and there is silence in which your company does not appear. This difference fundamentally changes what needs to be measured.

On Google’s search results page, the user chooses among the available options, so even tenth place can generate a click. In an AI assistant’s answer, however, the system itself decides whom to mention. If you are not included in the answer, you are not simply in a worse position – you do not exist. Moreover, the user typically does not ask another question but accepts the list of names provided and starts discussions from there. That is why measuring AI visibility is not about monitoring rankings, but about examining whether your name appears in a decision-making situation.

There is a second, less obvious difference: the answers cannot be reproduced exactly. The same question on the same day, in two different conversations, may produce a different order, different sources and sometimes different names. The logic of traditional rank-tracking tools breaks down here because the result of a single measurement may be random. This is not an error; it is the nature of the system: language models generate responses probabilistically while working from underlying sources. That is why measurement is not a snapshot but a series of measurements, where proportions and trends matter.

Third, AI does not merely list; it interprets. It combines, evaluates, recommends and explains. If contradictory information exists about you on the web – different company names, different service scopes, outdated pricing, missing references – the answer will reflect that confusion. Measurement therefore reveals both your visibility and the machine-generated picture of your brand. The latter is valuable business information in itself: it shows what the market thinks about you before anyone even speaks with you.

The Logic of the Question Set: Revenue First, Appearance Second

The most common mistake is that companies ask questions out of curiosity. “Who are we according to AI?” – an interesting question, but it measures nothing by itself because it is not connected to a decision-making situation. A useful question is one that your customer would ask before making a purchase, at the moment when they do not yet know you but already know they need to solve something.

That is why it makes sense to build the question set around purchase intent. At one end of the scale is the problem (“something is not working, who can help?”), and at the other is the decision (“is this specific company right for me?”). The closer a question is to the decision, the greater weight it deserves in the measurement – because losing visibility there directly costs money.

“I start with questions that signal real purchase intent: who can solve the problem, which supplier can be trusted, how much it costs, and how the alternatives compare. A fixed, documented question set provides the baseline for remeasurement – even when AI answers differ from run to run.”

The idea is based on a simple insight: just because an AI system knows about you does not mean it will recommend you. What matters is whether your company appears in the answer to a decision-related question as a credible, verifiable and unambiguous signal. The question set is therefore not designed to satisfy curiosity, but to reveal where trust is leaking out of the buying journey.

The Eight Question Groups You Need to Test

The following eight groups cover the entire buying journey from problem recognition to confirmation of the final decision. You do not need to ask ten questions from every category: two or three well-written questions per group already provide a measurable picture. Notice in the examples that your brand name appears only where this is intentional – the other questions are the real test.

How Do You Write a Question That Can Be Measured?

The wording of the question itself influences the result. If you ask a short question without context, AI will provide a generic answer, and large brands dominate generic answers. With a specific, realistic situation, however, the system has to look for names that fit the context – and this is where you have a chance.

That is why it makes sense to build every question from five elements. This structure ensures both repeatability and that the answer genuinely relates to your market.

Documentation: The Backbone of Measurement

Because AI answers vary, the value of measurement lies not in an individual answer but in consistently repeated measurement. For two measurements to be comparable, however, every relevant condition needs to be recorded. If one month later you cannot remember exactly how you asked the question, which tool you used or which language you used, then you did not perform a measurement – you collected an impression.

A simple spreadsheet is enough. The following columns are the minimum required for credible comparison later.

Two additional things are worth recording because they are often omitted in practice. The first is the unchanged question set: if you rewrite the questions every month, you will never have a trend. The second is the version number: if you do change a question, introduce it with a new identifier and retain the old one as a comparison baseline. This preserves the historical data.

What Should You Measure in the Answers?

The question “are we included?” is too crude to support executive decision-making. An answer contains much more information, and those elements have different business consequences. Six dimensions are worth recording for every answer.

A Score That Can Be Added Up

Of the six dimensions, the simplest approach is to combine presence and position into a single four-point scale and treat the remaining dimensions as additional signals. A four-point scale is practical because the extreme values are unambiguous while the intermediate values do not drift too far into subjectivity.

This can be turned into a single metric that is suitable for presenting to management. The formula is simple: multiply each question’s score by its weight, add the values together and divide the result by the maximum available score. The result is a percentage expressing your visibility across commercially important questions.

The score should always be accompanied by measurement uncertainty. Because answers change from run to run, a movement of 5–10 percentage points may still be noise. A trend that persists across two or three consecutive measurements is what deserves strategic attention. This approach protects you from reallocating the entire budget after one good month, and from panicking because of one poor run.

A Working Measurement in Practice

Imagine a Hungarian B2B software company with twenty employees operating in Budapest. Management knows that some decision-makers already ask AI before the first meeting, but they have no idea what happens when they do. We create a twelve-question set: three problem-awareness questions, three service-provider search questions, two trust questions, two pricing questions, one comparison question and one brand-focused question.

The first measurement produces a typical pattern. For the brand-focused question, the answer is acceptable: the company name appears correctly and the description of the service is broadly accurate. In the problem-awareness questions, however, the company does not appear at all because there is no content on the web connecting the problem and its solution with the company’s name. In service-provider search questions, three large international providers appear instead. For the pricing question, the answer gives a global price range that is approximately twice the Hungarian market reality.

This measurement produces three very specific tasks. First, create a page that names the problem and presents possible solution paths – this targets the problem-awareness questions. Second, create a pricing page for the Hungarian market with public ranges – this improves the pricing question. Third, create a comparison page that contrasts the company’s own approach with traditional alternatives in a factual and citable way.

Three months later, the score typically does not double, but the structure changes. The company name begins to appear in problem-related questions, the Hungarian price range appears in pricing questions, and the comparison question includes the company together with justification. The accuracy of brand-focused answers improves because the information across the web finally points in the same direction. This is the realistic pace: not overnight, but question group by question group, building one layer upon another.

The Seven Most Common Mistakes

30-Day Implementation Plan

If you are starting from zero, these four weeks are enough to build a working, documented measurement system – and your first concrete action list.

Frequently Asked Questions