Does the citation support the answer? Auditing AI claims and passages

An answer can mention your brand, attach a citation and attribute a capability that the linked document never describes. It can also cite the right page while dropping a decisive condition: the country, the plan or the effective date. Counting links will not distinguish these cases.

This audit is for brand and content teams that already retain AI answers and need to check what those answers say about their product. The output is a record of claims, versioned passages and support decisions. The protocol below is a proposed working method, not an official provider taxonomy or an announced automated Mentio feature.

Define the claim you will assess

Work with a claim that can be assessed on its own. Preserve the subject, action, object and conditions. Do not remove “may,” “only,” “up to,” “in Spain” or “since September” to simplify the comparison. Those words are part of the claim.

If one sentence combines compatibility and pricing, split it into two units when each could receive a different verdict. Keep the original sentence and its position in the answer as well. You should be able to reconstruct what the user read without treating the auditor's paraphrase as the original evidence.

Before reviewing, define which claims belong in scope: for example, verifiable statements about availability, commercial terms and product capabilities. Record exclusions. Subjective recommendations can contain checkable facts; “the best option because it includes export” contains an export claim that deserves its own row.

Research provides a useful distinction between citation coverage and accuracy. Liu, Zhang and Liang (2023) examine both whether statements are supported and whether citations substantiate the text associated with them. Here we adapt that distinction to a documentary review of brand claims, without applying historical findings to current systems.

Map each citation to its visible scope

Identify the citation marker, the claim it appears to address and the URL it opens. Distinguish a citation beside a sentence from an additional link at the end of the answer. Record ambiguity about scope; visual proximity does not establish that the system attributes every sentence in a paragraph to that source.

Keep one row per claim–citation relationship. A claim with three citations produces three relationships but remains one claim. A valid answer without citations is an observable state separate from a failed capture.

The Perplexity visibility measurement guide explains how to record presence and sources within a monitoring process. This step adds a check of the passage supporting the specific text. It cannot reconstruct every document retrieved internally or establish that a displayed link caused the answer.

Preserve the version before judging the passage

Save the answer, its date and the observation conditions you know: question, language, surface and mode. For the source, record the cited URL, the final URL after redirects, the retrieval date and any date or version declared by the document.

Locate the evidence using a heading and a short excerpt. For a table, retain the row, column, unit and notes; for a PDF, retain the page and context. A cropped screenshot that removes the exception is insufficient for reviewing an availability claim.

Retain only the material you need and are allowed to keep. A file hash helps detect changed bytes; on its own, it does not prove the publication date or which version the model could have accessed.

If only the current page is available, write “assessed against the version retrieved on this date.” Do not claim it was identical when the answer was generated. If a page fails to load, requires access or no longer contains the passage, record “not verifiable with the available evidence” and the reason. An access problem is not absent support.

Apply four support decisions

These labels apply to a claim and a passage you have been able to read. “Not verifiable” is a separate review status, not a semantic conclusion about unknown text.

Do not mark a relationship contradictory merely because two documents cover different plans, markets or dates. Establish comparable conditions first. Nor should a shared topic earn partial support: mentioning exports does not substantiate every promise about what they contain.

Consider a hypothetical example created to illustrate the rule. The answer says “the Basic plan can export all data.” The passage says that Basic can export tasks but does not clarify what happens to comments and attachments. Support is partial; “all” remains unsupported. If the passage adds that attachments are excluded, the universal claim is contradicted. These are not the terms of a real product.

The label describes documentary support, not absolute truth. A claim can be true and poorly cited; another can faithfully reproduce an incorrect source. The ALCE work by Gao and colleagues (2023) evaluates correctness and citation quality as distinct dimensions. Keep a separate field here for concerns about source reliability or currency as well.

Check conditions a paraphrase can lose

Read the passage and the claim side by side. Start with the elements that change meaning:

• Entity and variant: the specific product, version, plan or service.
• Scope: the market, population, language or channel covered.
• Time: the date of the data, measurement period and effective terms.
• Quantity: units, denominator and the difference between a maximum and a typical value.
• Modality: possibility, obligation, estimate or guarantee.
• Relationship: an observed association versus a demonstrated cause.

The wording need not match. The paraphrase must preserve what the source supports. A translation can be faithful despite different vocabulary and fail because it drops a negation.

If the claim includes a calculation, retain the inputs, units and operation. Decide beforehand which elementary transformations the protocol accepts. Identify any inference requiring extra assumptions; do not approve it as though it were explicitly documented.

When a critical condition appears only in a note, link that note to the passage. If another document is needed to complete the evidence, add a relationship and explain its role instead of silently replacing the cited source.

Resolve multiple citations without averaging verdicts

Assess each relationship separately, then examine the set associated with the claim. Two partial passages can cover complementary parts, but only when their entity, version, date and scope are compatible. Document each contribution and the reasoning that connects them.

Use an explicit claim-level rule: complete when the reviewed set covers the whole claim and leaves no relevant contradiction unresolved; partial when it supports only part; absent when all assessable relationships lack material support. Flag a comparable contradiction and retain favorable relationships too. One correct citation does not erase an incompatible one.

If missing documents prevent you from resolving the set, leave the claim pending review. Evidence that already establishes a contradiction can be recorded even when other citations remain pending, with that limitation visible.

This check concerns what the passages support. Source dependency analysis addresses a subsequent question: what evidence would be lost if a source disappeared. Do not confuse that exposure with the semantic quality of a citation.

Report denominators that retain pending cases

Start with counts. Separate in-scope claims, uncited claims, claims with a resolved review and those pending because of access or ambiguity. Keep capture failures that prevented a valid answer separate.

One possible operational indicator is the number of claims with verified complete support divided by all in-scope claims in valid answers. Uncited and pending claims remain in the denominator; neither contributes verified complete support. The result measures verified coverage in that sample, not the percentage of false claims.

If you also calculate complete support among reviewed claims, show both figures and their denominators. Do not compare that percentage with a round that reviewed an easier subset. Keep claim segmentation rules stable and explain changes.

Do not assign partial support half a point for convenience. Do not mix relationships and claims either: a sentence with several citations should not carry more weight in a claim-based metric. A relationship-level breakdown helps locate problematic citations, but answers a different question.

Close each finding with a specific check

A useful record ends in a verifiable action. If your documentation omitted a condition, clarify it and retain both versions. If the source was already clear and the answer widened its scope, record the exact overstatement and observe the same question again under comparable conditions. A different new answer does not establish that your change caused it.

For an error in your own draft, use the editorial QA process for AI-assisted content. For an external answer, retain the evidence and use available correction channels without promising that the provider will adopt the change.

The closing record should contain the claim, passage, version, label, explanation, action, owner and next check. Another person should be able to reproduce the verdict from those materials. If they cannot, reopen the affected relationship before closing the case.

You can use Mentio to monitor your brand's visibility and retain this documentary review alongside your analyses. Check which data your tool exports; this protocol does not assume an automated claim–passage auditor. Explore Mentio plans.

When the passage comes from a downloadable table, the PDF catalog audit for AI answers reviews reading fidelity and observable stages without attributing a failure to an extraction process you cannot inspect.

Citation fidelity and an instruction's authority are separate questions. The defensive source review for GEO analysis lets you quarantine a suspicious version without declaring it false on that basis.

Frequently asked questions

Does a citation beside a sentence prove it supports the claim?

No. Open the destination, locate the passage and check the full claim with its conditions. A nearby marker does not establish support or reveal every source used internally by the system.

What is the difference between partial and contradictory support?

Partial support leaves part of the claim unjustified. Contradictory evidence is incompatible with the claim under comparable conditions. An explicit exception can contradict a universal promise; silence about that exception is insufficient to reach the same conclusion.

How should I record a source I cannot open?

Mark it not verifiable with the available evidence, including the reason and the date of the attempt. Do not assign absent or contradictory support without reading the relevant content.

Can several citations jointly support one claim?

Yes, when their passages cover complementary parts and have compatible entities, versions, periods and scope. Retain the result for each relationship and explain the combination; do not average labels or hide contradictions.

Does correcting a source guarantee that the AI answer will change?

No. You can verify the documentary correction and observe new answers, but cannot control what each system retrieves or how it uses that information. Record those two outcomes separately.

Want to know if AI mentions your brand?

Discover your visibility in ChatGPT, Claude and Gemini in minutes.

Analyze your brand for free

Related articles