Key takeaways
- API responses and consumer AI interfaces are different systems. Surfer's study of 13,779 answers found overlap of about 30% at best, and often much less.
- APIs tend to overreport brand mentions. In Surfer's test, ChatGPT named 13.8 brands per answer via API versus 7.9 in the real interface.
- Retrieval behaviour (query fan-outs, citations per answer, source mix) changes with product updates that an API call to a raw model will not reproduce.
- Ask every vendor one question before you buy: do you collect from the real user interface, from the API, or both? Then ask how they separate the two in reports.
- Real-interface tracking is more expensive to run, so expect it to show up in pricing. It is still the number that matches what buyers see.
The problem: two ways to collect AI answers
Every AI visibility tool needs a machine-readable record of what ChatGPT, Perplexity, Gemini, and Google's AI surfaces say about your brand. There are two ways to get it.
The first is the API. You send a prompt to an official endpoint and get clean text back. It is cheap, stable, and easy to scale. The second is collecting from the real user interface: a browser session opens the product, submits the prompt, and parses the answer, the brand mentions, and the citation links.
The catch is what you get. An API returns the raw model. It does not return the product around the model: the system prompt, the search and retrieval step, the ranking of sources, safety filtering, personalization, and answer-shortening. Those layers are what real users read.
What the research says about the gap
Surfer SEO ran 1,000 prompts across five AI products, once by scraping the interface and once via API, and collected 13,779 answers across 14 model-and-method combinations. Their full write-up looked at cited brands, sources, pages, and answer length. Their headline: overlap between API and interface data is about 30% at best.
Two caveats worth knowing. Surfer is a vendor that scrapes, so it has a stake in this conclusion. And for AI Overviews, the API side of the test borrowed AI Mode, because Google does not sell the model that runs Overviews. Treat the exact numbers as one data point, not gospel. The direction, though, matches what practitioners see.
APIs name more brands
The most useful finding for tracking is brand count. The API named more brands than the interface across all five products. ChatGPT was the extreme case at 13.8 brands per answer via API against 7.9 in the interface. AI Overviews showed a similar pattern. Gemini and AI Mode were close to parity, at 1.10x and 1.06x.
Practically, that means an API-based tracker can tell you that you appear in a prompt when a real buyer would never see you. It flatters your numbers. I would rather have a smaller, true number.
A single instruction changes the answer
Maciej Gruszczynski, a data scientist at Surfer, put it well in the study: a single sentence in a system prompt can turn a response by 180 degrees, and consumer products carry long instructions that differ by application. You cannot see them and you cannot copy them. So API output is an approximation of the product at best.
Why retrieval behaviour makes this worse
The differences are not only about wording. How AI products search changes over time, and that behaviour lives in the product layer.
Promptwatch Data's ChatGPT query fanouts report shows ChatGPT averaging 2.15 searches per response in early December and about 1.84 by early March, with average query length falling from roughly 117 characters to the high 80s. Later in April it was back at 1.0 searches per response. None of this is something a raw-model API call would reproduce, because the API does not run the product's search step.
On August 8, 2026, ChatGPT began using the site: operator at scale, jumping from about 0.4% to about 17% of fan-out queries overnight, per Promptwatch Data. The same report notes searches per response nearly doubled. A tracker pulling from an API would have missed the shift entirely.
Citation volume varies by surface too. According to Promptwatch's average sources per response data, ChatGPT cites about 5 sources per answer when web search triggers, AI Overviews about 10, and Perplexity almost exactly 10 every day. Copilot swung from under 2 to nearly 17 within weeks before settling low. Those are product behaviours.
And answers vary by prompt type. In Promptwatch's unique domains per response report (Sep 28 to Oct 4, 2026), 47.6% of ChatGPT responses cited 10 or more domains. For organic prompts that was 52.5%. For brand-specific prompts only 21.3%, and the most common count was 5 domains. One prompt tells you very little, which is the reason tracking needs many prompts run through the real interface.
When API data is still useful
I do not want to overstate this. API data has a legitimate job.
- It shows what the base model believes from training data, with no live search. That is useful for brand perception and for entity knowledge.
- It is cheap, so you can run huge prompt sets for rough directional checks.
- It is the right source for developer-facing questions, like how a model you integrate will respond.
The mistake is treating it as a stand-in for what buyers see. The best approach is both, clearly labelled: real-interface data for visibility reporting, API data for model knowledge.
How to evaluate a platform's data source
Vendor marketing rarely states this plainly, so ask directly.
- Does each tracked platform use the real consumer interface, the API, or a mix? Ask per platform, not in general.
- Are country and language set in the interface session, or just in the prompt text?
- Can you see the actual cited URLs and their order, as shown to users?
- If AI Overviews and AI Mode are covered, are they collected from Google's real result pages?
- Are API-derived and interface-derived results shown separately in reports?
- How many responses does each plan include, and what is the cost per response?
Be aware that some roundups describe tools as reflecting "real user experience rather than backend API behavior", and others openly say they pull from official APIs. One review site, for instance, notes that Conductor pulls AI answers through official APIs, and cites Surfer's study as an argument that this is less representative. Verify claims in a trial rather than trusting a listicle, including this one.
Platforms to shortlist
This is not an exhaustive ranking. Vendor collection methods change and I could not verify every vendor's current method independently, so use the table as a starting point for your own questions.
Promptwatch
Promptwatch monitors the actual interfaces of ChatGPT, Google AI Overviews, Google AI Mode, Microsoft Copilot, Gemini, Perplexity, Amazon Alexa, Codex, and Claude Code, in each country and language. It also tracks API models like Claude, DeepSeek, Mistral, and Grok alongside, which makes the real-versus-API distinction visible inside one tool rather than hidden. Beyond tracking, it adds IP-verified crawler logs, visitor analytics, and a Content Agent that publishes fixes, so you can check whether a change worked. Plans start at $95/month, with a free Explore tier; confirm current pricing at promptwatch.com/pricing.
Surfer SEO
Surfer argues for scraped interface data and built its AI tracking on it. Its content optimization background makes it a reasonable pick if you want tracking next to an editor.
Profound and Scrunch
Both are established monitoring platforms with broad engine coverage. Check their collection method per engine and compare cost per response, since both sit at higher price points.
Peec AI and Otterly.AI
Lighter prompt trackers with approachable pricing. One roundup says Peec AI reflects real user experience instead of backend API behaviour, especially for ChatGPT and Perplexity. They are good for small teams who mainly need mention tracking, though they generally lack crawler logs and visitor analytics.
Cognizo
Cognizo tracks brand presence across ten AI platforms and combines monitoring with content generation and technical audits, according to the airankchecker.net review.
Conductor, Semrush, and Ahrefs Brand Radar
SEO suites that added AI tracking. Convenient if you already live in them. Per the Position Digital review, Conductor pulls answers through official APIs, so ask about how that affects accuracy for ChatGPT and Perplexity. Semrush and Ahrefs tend to use fixed prompt sets, so check coverage for your market.
Comparison table
| Platform | Typical angle | What to verify | Best for |
|---|---|---|---|
| Promptwatch | Real-UI tracking plus API models, crawler logs, content agent | Plan response limits | Teams that want to act on the data |
| Surfer SEO | Scraped interface data plus content editor | AI tracking depth versus standalone tools | Content teams |
| Profound | Broad enterprise monitoring | Price per response | Large brands |
| Peec AI | Prompt tracking, multi-language | Crawler and traffic data | Small teams |
| Otterly.AI | Affordable monitoring | Content and analytics depth | Budget starters |
| Cognizo | Monitoring plus content and audits | Collection method per engine | Teams wanting one workspace |
| Conductor | SEO suite with AI tracking | API-based collection | Existing Conductor users |
If you want to browse more options, the GEO software directory at bestgeosoftware.com and the AI rank tracking directory list many more tools.
A practical way to test accuracy yourself
You do not have to trust anyone. Run a small audit before you commit.
- Pick 10 prompts your buyers would really use, including brand, category, and competitor comparison types.
- Run each manually in the real ChatGPT, Perplexity, and Google interfaces, logged out or in a clean session, from your target country.
- Record which brands appear, in what order, and which URLs are cited.
- Load the same prompts into the trial of each tool you are considering.
- Compare. If a tool shows your brand in prompts where you saw nothing, or lists far more brands per answer than you did, suspect API data or an inflated sample.
Expect some variation even between your own repeated manual runs, since AI answers differ per person and per session. You are looking for systematic over-reporting, not exact matches. The free tools many vendors offer, such as a one-off visibility scan, are a cheap way to start.
Why this matters for decisions
Bad data does more than waste money. If your tracker overreports mentions, you may conclude your content strategy is working and stop investing. If it misses a retrieval shift like the August site: operator change, you will not know why citations moved. And if you present AI visibility to a CMO, the number needs to survive someone searching the same prompt in ChatGPT and not finding you.
If you want a team to interpret this for your brand rather than doing it in house, an agency like 1001 SEO Media runs GEO programs on top of real-interface tracking.
Bottom line
API data and interface data answer different questions. The interface tells you what buyers see; the API tells you what a model knows. For visibility reporting, prefer platforms that collect from real interfaces, label API results separately, and show the actual citations. Then verify with your own ten-prompt test before signing anything.