Following claims back through the retrieval trail

In a composite case assembled from recurring errors, a restaurant appears in an answer with the right photograph, the wrong province, and a citation leading to a directory entry for another venue. The mismatch is easy to miss because the two venues share a similar English name. The laboratory begins by saving the whole scene. An observation includes the prompt, generated wording, visible citations, language, model context, observation date, and the conditions under which the answer was obtained. The team records what can be seen before explaining it. A proposed retrieval path becomes a conclusion only when several parts of the record point in the same direction.

The samples are built around situations people actually use for discovery: finding a provider by service, comparing nearby places, checking whether a branch exists, locating a business, or asking for a recommendation. Thai and English formulations are included whenever a change of language may alter the name, category, or geographic interpretation. The cases are chosen for contrast. A long-established company may sit beside a lightly documented one; a single-location business beside a branch network; a distinctive Thai name beside a name shared by several organisations.

A repeated run does not need to reproduce the same sentence. Generative systems vary their wording, ordering, and level of confidence. For the laboratory, repeatability means preserving enough of the procedure to ask the same question again and recognise whether the underlying pattern returns. The team compares models, prompt formulations, languages, and repeated observations. Agreement across systems is recorded, though it is never treated as confirmation by itself. Several systems may reproduce the same error because they draw on similar public sources or category assumptions.

Citations are then read claim by claim. In a recurring source pattern, a clinic's own page establishes that the business exists and describes its treatments, yet says nothing that supports calling it a hospital. Another source may support an address without establishing that the business is a branch. The laboratory classifies these relationships as direct support, stretched support, borrowed identity, or unsupported arrival. This prevents a relevant-looking citation from lending automatic authority to every statement placed around it.

Some parts of the route remain hidden. The method cannot reveal private retrieval systems, hidden ranking logic, undisclosed retrieval steps, or every source used internally by a model. The laboratory therefore marks the boundary between visible evidence and inferred retrieval paths. Where two explanations fit the record, both remain open. Predictions are presented as provisional expectations, together with the conditions that would weaken or overturn them. Uncertainty is kept in the record rather than edited out for a cleaner ending.

Working principles

1. Record before explaining

   The prompt, answer, citations, language, model context, date, and observation conditions are preserved before the team proposes a cause.

2. Separate the failure stages

   Retrieval, entity identification, attribution, and final wording are examined as distinct steps because each can fail differently.

3. Read citations claim by claim

   A page may support one detail and fail to support the category, location, ownership, quality judgment, or recommendation placed beside it.

4. Repeat the procedure

   The same inquiry should be runnable again under described conditions, even when the generated wording changes.

5. Leave uncertainty visible

   Competing explanations and provisional predictions remain labelled until the evidence distinguishes between them.