Nostimates
← All posts
Methodology·6 min·Nostimates

When a provider API doesn't return what users see

A provider API can return answers from a different model tier with different retrieval and ranking than the consumer interface — no shopping cards, ads or query fan-out. Where it diverges, an API-sourced number describes an experience no real user has. Nostimates collects each surface the way that reproduces what a person sees: the provider API where it matches, the rendered interface where it doesn't.

When a provider API doesn't return what users see

The mismatch can be structural, not incidental

OpenAI's web search tool returns URL annotations. Perplexity's Sonar returns citations. Gemini's grounding metadata returns sources. All three are real, documented and easy to call. But none of them is guaranteed to be the thing your client sees when they open the app.

The API paths often run on a different model tier with different retrieval and different ranking. They have no shopping cards, no ads and no query fan-out. The consumer interface has all of those, and all of them move which brands get named. Sometimes the two agree; where they don't, the difference is the whole story.

Why this is a commercial problem, not an academic one

A measurement tool that doesn't match the thing it measures gets exactly one chance. The first time someone opens ChatGPT next to your report and sees a different answer, they stop trusting every number in it, including the ones that were right.

A number is only worth reporting if it matches what a user would actually see. Where an API diverges from the interface, it fails that test.

How we collect around it

The rule is simple: reproduce what a person sees, per surface. Where a provider API returns the same answer as the interface, we use it — it is cheaper and more stable. Where it doesn't, we render the real interface in a browser, which for some surfaces means a session through a residential proxy. That per-surface choice is the cost base, and it is also why there is a business here rather than a weekend of API calls.

The right question for any AI visibility vendor isn't "API or scraped?" — it's "does this number match what a user sees?" We pick whichever method makes that true for each engine, and tell you which one we used.

See this for your brand

Nostimates shows you how your brand shows up across twelve AI engines and search — in a dashboard, or as data in your own tools. Tell us what you want to measure.

No newsletter. One reply from a human.