Brand recommendations in multi-turn AI conversations: how to measure them

A brand can appear in an assistant's first shortlist and disappear when the buyer adds a condition. It can also enter later, after a clarification. Saving only the first answer loses that trajectory.

To investigate these transitions, preserve the full conversation and distinguish what the user introduced from what the assistant suggested. This guide proposes a protocol for purchase research and GEO analytics teams: follow a brand across turns, locate changes in its recommendation and deliver an evidence-backed transition map.

The prompt bank defines the needs you will study. Here, each need becomes a sequence with context. The result describes the conversations in your sample; it does not, by itself, estimate all buyers' behavior or demonstrate sales.

Define a trajectory and its closing rule

The observation unit is one execution of a complete conversation route. Identify the opening question, follow-up branch, repetition, AI surface and configuration. Within that execution, each turn has a question, answer and brand state.

Specify what completes the route: reaching a requested choice, resolving an objection or reaching the last planned follow-up. Final recommendations are comparable only when they correspond to the same decision point.

Declare the target brands and how you will recognize products or aliases. When the assistant recommends a product, retain that entity alongside its brand; a reference to a manufacturer does not establish a recommendation for its entire catalog.

Design branches before reading the answers

Start with needs and constraints documented in your customer research. Preserve their provenance without including personal data in prompts. Decide what information each step adds and which response allows the route to continue.

A fixed follow-up lets you compare the observed effect of adding the same condition. An adaptive branch can respond to what the assistant actually asks, but it needs a prior rule for what information to return to each type of clarification. Improvising to obtain a brand recommendation measures your intervention.

Separate routes that name brands from those that do not. Record the first turn where the user introduces each name. A brand suggested after that cue belongs to a prompted condition; one proposed by the assistant before the cue belongs to a spontaneous condition within the observed history.

Preserve the history available to each branch

OpenAI's conversation state documentation explains that a multi-turn interaction requires passing or retaining earlier messages. For measurement, record the available history as well as the latest question.

Two designs serve different purposes. In a natural trajectory, each execution continues from its own answers. Routes may diverge from the opening turn. In a comparison from a fixed prefix, branches receive the same preserved earlier history and change the specified follow-up.

The second design is valid only if the interface or integration can reproduce that prefix. Running the opening question again may produce a different answer and does not clone the history. If you cannot hold the prefix identical, record natural trajectories and limit the comparison.

Include assistant messages, user clarifications and available tool results when they form part of the context. Record any summarization or truncation. Keeping an external copy of the history does not demonstrate that the model received every part of that copy.

Execute complete routes and record interruptions

Keep the agreed configuration during the wave: displayed model, search mode, language, market, date and known account context. Start a separate execution for each trajectory and apply the same follow-up rules.

Repeat the whole route, rather than just its final turn. In a natural route, copying a favorable opening answer from the first attempt into every repetition changes the population being observed. To estimate differences between waves, retain the route design and configuration.

The answer variability guide explains how to record conditions and repetitions. History is part of the design in this study, so it also needs a version.

If an execution fails, retain the interruption point and observed cause. Do not silently replace an answer, regenerate until a mention appears or remove the route from the record. An interrupted execution can inform a process review, but it does not supply an evaluable final recommendation.

Label each turn with supporting evidence

Distinguish a textual mention from an explicit recommendation. A brand may appear in a recap of an earlier list, as a rejected alternative or in response to a question that already names it. These appearances do not mean the same thing.

For each brand and turn, preserve the passage and apply a documented rule:

• Explicit recommendation, including its conditions.
• Mention without an explicit recommendation.
• Explicit exclusion, with the reason stated in the answer.
• No mention in an evaluable answer.
• Not evaluable, when the information needed to decide is missing.

Absence from a follow-up does not establish rejection. An assistant answering only about a requirement may omit every brand. You can add a conversation-level state for an active recommendation, but you need a specific rule for carrying forward or withdrawing an earlier recommendation. Keep that state distinct from the label on the current answer.

When automating labels, test the classifier with history and passages, following the mention and recommendation validation guide. OpenAI's evaluation guide also recommends calibrating automated metrics against human review. The transition protocol here is an editorial application, not an official metric from that provider.

Calculate entry, retention and exit with comparable bases

If explicit recommendation in each answer is your primary state, a transition compares two consecutive evaluable turns under the same definition.

Entry means moving from no explicit recommendation to an explicit recommendation. Its rate divides entries by evaluable transitions whose previous state had no recommendation. Retention divides pairs with a recommendation in both turns by evaluable pairs that started with a recommendation. Exit uses the same denominator and counts pairs ending without an explicit recommendation.

Exit means loss of explicit recommendation in the next answer. Report separately how many exits contain an express exclusion. This distinction prevents silence from being presented as rejection.

Report rates by step and branch with their numerators and denominators. A rate is undefined when there are no eligible cases. Pairs with a non-evaluable turn stay outside that calculation and must be counted separately. Do not convert an interruption into brand absence.

Presence at the closing point has a different base: routes with a recommendation at closing divided by routes with an evaluable closing point. Add closing coverage as routes with an evaluable closing point divided by routes started. That prevents an improvement among a few completed routes from hiding the rest.

Branches of different lengths produce different numbers of pairs. Do not pool them as though every row were an independent conversation. For intervals or statistical comparisons, preserve the dependence between turns and, where they share a prefix, between branches. Also calculate trajectory-level results; an average of rows does not resolve that dependence.

Investigate the step where the recommendation changes

Locate the first state change and inspect the added question, answer passage, visible sources and conditions attributed to the brand. Distinguish the assistant's stated reason from a confirmed explanation about the product.

Check whether the answer retains earlier requirements and whether claims about availability, capabilities or market have support. A change may reflect a legitimate constraint, incorrect information or a different interpretation of the need. The map identifies where to investigate; it does not determine which explanation is correct.

In natural trajectories, earlier answers may also differ between routes. A difference after a condition is an observed association under that history, not an isolated causal test. Comparing branches from an identical prefix improves control but does not remove system variability.

Deliver a map that supports conversation review

The report should let a reviewer trace a rate back to the text that produced it. Preserve trajectory ID, prefix or history, branch rule, turn, brand, label, passage, user-introduced cue and closing status. Include design and classification versions.

Show transitions by step, evaluable closing points and interruptions. Separate spontaneous and prompted results, and show explicit exclusions alongside exits by omission. Identify branches needing more evidence and product claims requiring review.

That record supports decisions about extending the study or correcting information. A recommendation observed in the sample remains a conversation result; connecting it to a purchase requires another source of evidence.

When differences involve persistent context between conversations, use a different analytical unit: comparing personalization and memory across profiles pairs account conditions and separates prior brand exposure from transitions within a dialogue.

Frequently asked questions

Has a brand been rejected if it disappears from the next turn?

Not necessarily. A follow-up may address only a condition and omit names. Record the loss of explicit recommendation in that answer and separate cases where the assistant expressly excludes the brand.

Can I count all turns as independent observations?

No. Turns within a trajectory share history, and branches can share a prefix. Preserve that dependence in the analysis and report results by step, branch and trajectory.

How do I distinguish a spontaneous recommendation from a prompted one?

Record when the user introduces the brand name. A suggestion before that cue is spontaneous within the observed history; one after it belongs to a prompted condition and must be analyzed separately.

Does repeating the opening question create the same starting point?

It does not guarantee the same history. It may generate a different answer. Comparing branches from an identical prefix requires preserving and reproducing that prefix; if the interface cannot do that, declare that you are studying natural trajectories.

What should I do when a conversation never reaches the closing point?

Retain the route, last turn and observed cause. Do not count it as a negative final recommendation or remove it without explanation. Report routes started, evaluable closing points and interruptions separately.

Want to know if AI mentions your brand?

Discover your visibility in ChatGPT, Claude and Gemini in minutes.

Analyze your brand for free

Related articles