AI visual search: checking whether it recognizes your product

An answer can recognize a red athletic shoe and link to a different, similar-looking one. It can also identify the product family while choosing the wrong variant. Recording both as “product recognized” hides the error that matters to the catalog.

This guide proposes a protocol for product, ecommerce and measurement teams checking what an AI system identifies from an image. The deliverable is a set of photos with independent references, recorded input conditions and preserved answers. It does not establish that a product will appear in every search or that Mentio provides automated visual recognition.

Tracking product recommendations at SKU level starts with commercial questions and evaluates which products appear. Here, the input is a photo: the answer must first be checked against the object shown before it is counted as a recommendation.

Define what recognizing the product means

Decide separately whether you are evaluating category, brand, product family or variant. A correct category does not establish the other levels. Keep the literal answer alongside its label; do not turn an approximate description into an exact identifier because its wording resembles the catalog name.

The reference needs an approved source: a manufacturer page, master catalog or owner's record. An AI brand entity audit helps establish that identity. Then state which parts can be checked from the photo and which are known only from external documentation.

Do not require a distinction between visually indistinguishable variants. If size, capacity or edition is neither visible nor stated in the permitted text, the photo does not supply enough evidence to decide. That case tests whether the system preserves uncertainty, not whether a guess happens to match the SKU.

Keep an independent reference for each photo

Use your own images or material you have permission to send to the selected service. Exclude personal data, internal documents and background elements outside the test. Permission to publish a photo should not be treated as permission for every use of a private file.

Assign each image an identifier and record its product, source, capture or retrieval date, catalog version and usage rights. Preserve the original and any derivative separately. Keep the identity reference outside the assistant's input when the aim is to test recognition without supplying the name.

For each evaluation level, add the feature that supports the decision and its location: a readable logo, label, shape or documented detail. If the feature cannot resolve identity, mark the reference as ambiguous. A second model suggesting a name does not replace that independent check.

Prepare comparable visual conditions

Group images by conditions relevant to actual use: overview or detail, angle, lighting, background, occlusion and label readability. Keep an explicit relationship between the object and its photos. Several views of one item are not several independent products.

In its vision documentation, checked on October 14, 2026, OpenAI warns about small or rotated text and incorrect descriptions. It also documents image resizing. These are limitations of that input path, not a universal error rate for assistants.

Preserve resolution, cropping and known transformations. A crop that removes the label changes the available evidence; do not compare it as though it were the same photo. If you cannot control an application's processing, record what is unknown rather than attributing a specific quality reduction to it.

Separate the image, neutral text and product name

Design paired conditions using the same object. If the interface requires a question, “What product is shown?” can be the identification instruction; do not describe that as a run with no text. The visual condition avoids supplying the brand or model through other fields you control.

The name is a separate control, not a hint added to unsuccessful runs until the system gets it right. If the text reveals identity, you are no longer measuring visual recognition alone. Also record filenames and submitted fields when your system exposes them; do not assume every platform processes the same metadata.

Google introduced photo input in AI Mode with Lens on April 7, 2025. That announcement documents a multimodal path, not current availability for every country, account or device. Check the surface you use and do not pool its results with those of a different API.

Distinguish correct identity, ambiguity and substitution

Evaluate each answer at the level declared before the run. Retain at least correct identity, incorrect identity, ambiguous answer, abstention and similar alternatives. A list of similar objects may answer “find something similar” appropriately without answering “identify this product.”

Include cases where the image cannot resolve the variant and cases outside the evaluated catalog. Define acceptable output before reading answers: identifying a category, asking for a detail or abstaining may be appropriate for the task. Do not change the rule afterward to convert these into exact matches.

Keep any confidence stated by the system separately. Confident wording does not validate identity. When an answer lists alternatives, record which it presents as identification and which as options; do not award recognition for a correct SKU buried in a long list without an explicit identification.

Do not turn recognition into a purchase recommendation

Check identity, product link, offer and recommendation separately. A link may lead to the right family while offering another variant; a correct product page may show an out-of-stock item. The photo also does not establish price, availability or suitability for an unstated need.

To study purchasing, add a later phase with an explicit commercial condition and keep its relationship to the identity test. The Perplexity Shopping ecommerce guide covers catalog preparation and eligibility. It does not replace checking whether an answer identifies the photographed object.

Do not count a visual card as an organic recommendation based on its appearance. Record the text, labels and observed component. If an interface does not reveal why it selected a result, that reason remains unknown even when the destination is correct.

Calculate rates without hiding difficult cases

Preserve planned images, attempted tests, complete answers and evaluable levels. An upload failure is not an incorrect identification; an abstention in a complete answer is not a capture failure either. Showing both prevents a rate from improving simply because difficult attempts disappear.

For product family, divide correct identifications by complete answers with an evaluable family reference, and show abstentions, ambiguity and errors within that denominator separately. For variant, use only cases with enough evidence to distinguish it and state how many were excluded. Also report the planned total and execution coverage; do not present the easy subset as the whole catalog.

If a photo is repeated, preserve every answer with its run and condition. You can report by attempt and by object, but do not mix the units. Compare within each visual condition and state the composition of the set. A panel designed to find failures does not, on its own, estimate error frequency among real shoppers.

Deliver issues the owner can investigate

The final record must allow each discrepancy to be reconstructed: photo and version, approved identity, visible features, exact input, known service and settings, answer, links, label and rationale. A screenshot without the input file leaves half the test missing.

Separate actions within the team's control: correcting a misassigned photo, adding a view that shows a real distinguishing feature or resolving a catalog contradiction. Repeat the condition afterward and keep both versions. An observed change does not establish that the model has learned or that other users will receive the same answer.

If the result remains ambiguous, a useful outcome may be a review instruction: request a readable label, another angle or an authorized identifier. Keep that limit in the report. An identification more specific than the photo supports adds apparent precision, not evidence.

Frequently asked questions

Does a visual match mean the AI recognizes my SKU? No. It may identify the category or family and suggest a similar variant. Exact recognition requires an independent reference and enough evidence to distinguish that SKU in the evaluated input.

Should I include the brand name in the question? Only in a separate condition. Supplying the brand or model changes the available information, so you are no longer evaluating visual recognition alone. Keep a test without revealed identity and another with the approved name as a control.

How should I score two variants that look identical? If the photo contains no feature that distinguishes them, record the variant as not evaluable. You can evaluate category or family and check whether the answer acknowledges ambiguity without inventing an identifier.

Does abstention count as a failed capture? No. A complete answer that does not identify the product is an evaluable output. An upload failure, timeout or incomplete response belongs in the capture record and must be reported separately.

Do better photos guarantee inclusion in AI searches? No. A clearer photo may supply identity evidence, but it does not guarantee selection, recommendation or sales. The audit checks answers under documented conditions; it does not certify behavior across all searches.

Want to know if AI mentions your brand?

Discover your visibility in ChatGPT, Claude and Gemini in minutes.

Analyze your brand for free

Related articles