Challenge · AI search measurement
Before you can measure AI visibility, you have to pick the questions
Every brand that starts tracking AI visibility hits the same wall in week one, and it is not a technical one. Somebody has to decide what to ask.
By Priyadharshini Krishnan, Co-Founder & Chief AI Officer · Updated 12 October 2026 · 8 minute read
Answer engines get asked questions, so a keyword list does not carry over. Build the set from how buyers actually talk. Start with the three or four decisions somebody makes before choosing anyone in your category, write each as a full sentence in their language, and cover three intents: category questions where nobody is named, comparison questions putting you against a competitor, and problem questions from people who do not yet know your category exists. Write the category questions for the segment your positioning targets, because a broad one usually names the market leader. Twenty to forty is plenty for a first set. Freeze it once you start measuring, because changing the questions changes the score for reasons unrelated to your visibility. The usual mistake is writing questions in internal category language that no buyer has ever used.
Why does a keyword list not work here?
A keyword is a fragment typed at a system that returns ranked pages. A question is a sentence typed at something that will answer it.
"running shoes flat feet" and "what running shoes should I get if I overpronate" carry the same intent and behave nothing alike, because one is matched against documents and the other is answered out of a general understanding of the category. Your keyword research is a decent source of topics. As a question set it is close to useless, since it contains what people type when they expect links, and what they write when they expect an answer is different.
What should be in the set?
Three kinds, in roughly this mix. Each tells you something the others cannot.
| Type | Shape | What it tells you |
|---|---|---|
| Category, about half | "What is the best X for Y?", nobody named, Y being the segment you chose | Whether you are in consideration at all, for the buyers you set out to win. The most important number, and usually the ugliest |
| Comparison, about a quarter | "Is A or B better for Y?", you and a competitor | How you get characterized once you are already in the room |
| Problem, about a quarter | "How do I fix Y?", category not named | Whether you reach people before they know what to shop for. Hardest to win, worth the most |
How specific should a question be?
As specific as your positioning. A broad question such as "What is the best payroll software?" usually names the market leader, because it is the brand most of the web already writes about. If you are the challenger, your score on that question will stay low for a long time, and it says little about the buyers you are built for.
So put your positioning into the question: the segment you sell to, the buyer, the use case, the tools you work with, or the price point you compete at. "What is the best payroll software for a forty-person company with staff in three states?" is a question a challenger can win, because the assistant now needs a product built for that company.
For many challengers this is a deliberate choice. They have picked the ground where they are the better answer, and the question set should follow it. Keep a few broad questions too, so you can see when you start being named in them. Why a broad question rarely names a challenger works through how to choose that ground.
How do you write questions a buyer would actually ask?
Take them from places where buyers have already written in their own words. Do not invent them in a workshop, which is how you end up with a set written in your marketing language instead of the words your buyer uses.
- Read your last fifty support tickets and pre-sales emails. The opening lines of an enquiry are usually a buyer question already written for you.
- Read the forums and review sites in your category and note the words people use for the problem. They are almost never the words your marketing uses for the solution.
- Ask two salespeople for the five questions they answer on every first call. Those are category questions, already validated by hundreds of conversations.
- Take the search fragments your site already gets found on and expand each into a full sentence.
- Read the finished set aloud. Anything that sounds like a brief goes back.
The trap worth naming
Internal category language. Teams write questions using the category name they wish existed, or the one from their fundraising deck. If buyers do not say that phrase, you will end up measuring visibility inside a category nobody asks about, and the score will look respectable while the pipeline shows nothing. The test for every question: has a customer ever said this to us? This is about the words in the question. How narrow the question is stays your decision, and it should follow your positioning.
How many, and how often do you change them?
Twenty to forty is enough to be meaningful and small enough to keep running. Bigger sets look thorough and mostly add cost, since every question has to be run repeatedly to produce a reliable rate.
Change the set rarely, and on purpose. When you do, note the date and treat what came before and after as two series. A score that jumped because somebody added eight comparison questions is not a score that improved.
One more thing, easy to miss: geography belongs inside the question text. Chat assistants have no country setting, so "the best X" gets answered from a general view unless the sentence says otherwise. Selling in one market, say which. Selling in three, write the question three times and expect three different answers, because that is what you will get.
The question set is the instrument. Choose it badly and you do not get a wrong number, you get an accurate number about something else.
What to do this week
- Draft twenty questions from support tickets and sales calls. Not from a keyword tool.
- Tag each one category, comparison or problem, then look at the mix. It will be comparison-heavy, because those are the flattering ones.
- Add the market to any question whose answer would change by country.
- Run them, then leave the set completely alone for a month.
If you are an agency
The question set is the most defensible thing you can build for a client in month one, and it is hard for them to do alone because it needs the sales and support material nobody thinks to hand over. It also front-loads the argument: show a client the ten questions their buyers ask and the two they appear in, and the scope of work writes itself. Keep it in the client record, where the next person can find it. Its entire value is being stable over time, and laptops are not.
How we do this ourselves
Our AI search visibility module holds the question set as a saved, scheduled thing, and will propose a starting set from your brand profile and category if editing is easier than starting blank. Each question records the market it is asked for, and results are kept per run so the rate and its history stay legible.