When a provider API doesn't return what users see

The mismatch can be structural, not incidental

OpenAI's web search tool returns URL annotations. Perplexity's Sonar returns citations. Gemini's grounding metadata returns sources. All three are real, documented and easy to call. But none of them is guaranteed to be the thing your client sees when they open the app.

The API paths often run on a different model tier with different retrieval and different ranking. They have no shopping cards, no ads and no query fan-out. The consumer interface has all of those, and all of them move which brands get named. Sometimes the two agree; where they don't, the difference is the whole story.

Why this is a commercial problem, not an academic one

A measurement tool that doesn't match the thing it measures gets exactly one chance. The first time someone opens ChatGPT next to your report and sees a different answer, they stop trusting every number in it, including the ones that were right.

A number is only worth reporting if it matches what a user would actually see. Where an API diverges from the interface, it fails that test.

How to interpret collected answers

A collected response is an observation from a particular target and collection, not a guarantee that every user sees the same answer. Compare the target, prompt and available context before interpreting differences. Nostimates returns provider-specific records from its supported targets; read the reliability guidance for routing limits and collection failures.