Key takeaways
- ChatGPT started serving ads on May 27, 2026. Across the 90 days to August 17, ads appeared in an average of 20.1% of responses that ran a web search, per Promptwatch Data on ChatGPT ads. If your tool ignores ads, it is ignoring a slot that competitors can buy.
- Ask where the data comes from. Real user interface collection and API collection can return different answers, citations and ads.
- Demand raw responses behind every number, a prompt set you control, and a clear method for re-baselining after model changes.
- Tracking is only half the job. Ask what the tool does once it finds a gap.
- A vendor that can't answer one of these questions plainly deserves a follow-up before you sign.
Why ad tracking belongs on your AI visibility checklist
For years the AI search pitch was simple: get cited, get recommended, done. Then ChatGPT turned on ads. Promptwatch's data shows zero ads until May 27, 2026, when they showed up in 1.69% of responses. The rate passed 30% on July 1 and has mostly sat between the low 20s and high 30s since. The last 7 days of the report window averaged 32.4%, and the daily peak hit 43.9%.
That changes what you are buying when you buy a visibility tool. A dashboard that tells you your brand was mentioned in 40% of answers is less useful if a competitor's sponsored placement sits above you in a third of those answers and you can't see it.
So this checklist covers ads, but also everything underneath: data collection, prompt design, metrics, and what happens after the report. Ten questions. Take them into every demo.
The broader field is crowded. If you want to browse before you shortlist, the GEO software directory at bestgeosoftware.com is a decent place to start, and ai-rank-tools.com covers rank trackers specifically.
The 10 questions
1. Does the tool track ads in AI answers, or only organic mentions?
Start here, because most tools fail it. Ask the vendor to show you a screen with sponsored placements: which advertisers appeared on your prompts, their ad copy, their claims, and the landing pages they pointed to.
A rate alone is not enough. As Promptwatch's report notes, the rate tells you how often an ad slot appears, and the ad details tell you who wins it. You want both.
Also ask how many days of data the tool needs before it reports an ad rate. A single prompt run can land on an outlier day, so anything built on one-off checks is noise.
2. Is the data collected from the real interface or from an API?
This is the question that separates serious tools from cheap ones. User-facing answers, citations, shopping results and ads can differ from what an API returns. API models are useful for seeing what a model believes from training data, but they are not what your buyer sees in the ChatGPT window.
Ask the vendor, in writing, which engines are collected from the actual UI and which from APIs. Ask about each engine you care about: ChatGPT, Google AI Overviews, AI Mode, Gemini, Perplexity, Copilot, and Claude.
Promptwatch collects from the real interfaces of ChatGPT, Gemini, Perplexity, AI Overviews, AI Mode, Copilot and others, and tracks API models alongside for the training-data view. Its explanation of how data is collected is a good model of the level of detail you should expect from any vendor.

3. Can you see the raw AI answer behind every number?
If a visibility score drops 8 points, you need to click through to the actual responses that caused it. Searchable's guide to AI visibility tracking makes the same point: a tracker's numbers are only defensible if you can open the individual answer, see whether the brand was mentioned, where it sat, and when the run happened.
Test this in the demo. Pick a number on a dashboard and ask them to show you the response history behind it. If they can't get there in two clicks, the number is a vibe, not a measurement.

4. Who controls the prompt library?
This one gets overlooked, and it ruins more dashboards than any bug. Aleyda Solis, who has written extensively on measuring AI search, puts it bluntly: using a non-representative prompt library is one of the biggest mistakes in AI search tracking. If the set over-represents generic discovery prompts, ignores product lines, misses local competitors or only tracks branded questions, the dashboard looks useful while pointing you at the wrong priorities.

Ask the vendor:
- Can I import my own prompts, or am I stuck with your default set?
- Can I tag prompts by topic, persona, intent and market?
- Can I see prompt volume or difficulty, so I'm not guessing which ones matter?
- Are prompt types separated (organic, brand-specific, competitor comparison)?
That last point matters for ads. In Promptwatch's report, 73.3% of ad-bearing responses came from organic prompts, 18.0% from brand-specific and 8.6% from competitor comparisons. But the report is explicit that this is a volume mix: it reflects how many prompts of each type are monitored, not how likely each type is to trigger an ad. Add more organic prompts and their share rises even if nothing changed at OpenAI. A vendor who quotes you that split as a trend should make you suspicious.
A practical starting point from the AccessibilityChecker guide: before buying any tracker, write down 20 to 30 questions a buyer would ask before they know your name, like "best X for Y" or "alternatives to [competitor]." Bring that list to the demo and ask the vendor to load it live.
5. Does it capture query fan-outs?
AI engines don't answer from your prompt alone. They run their own web searches behind the scenes. Those searches decide which pages get pulled in, and increasingly they decide which ads get matched.
Promptwatch's query fan-out data shows how unstable this layer is. Average fan-outs per response were 2.15 in early December 2025 and about 1.84 by early March 2026. Average query length fell from around 117 characters to the high 80s. After a gap in the data, fan-outs returned in April at 1.0 per response, with queries around 53 characters. The report also says a single prompt can trigger 3 to 8+ searches, which doesn't sit neatly with those averages, so treat any single fan-out figure from any vendor with care.
What to ask: does the tool record the actual fan-out queries per response? Can it compare them per model? And how does it re-baseline when the model changes? ChatGPT started using the site: operator at scale on August 8, 2026, jumping from roughly 0.4% to 17% of fan-out queries overnight, according to Promptwatch Data. A tool that doesn't notice shifts like that will quietly report meaningless trend lines.
6. Which metrics are defined, and can you break them apart?
A lot of tools lean on one formula: percent of brand mentions divided by percent of all responses. Ivan Palii flags the problem in his roundup of AI visibility trackers: a single blended number hides what is actually wrong.
You want metrics that separate problems:
- Share of voice for coverage
- Position for prominence
- Self-citation rate for content or crawlability
- Sentiment for reputation
And the tool should say whether a mention without a citation counts as visibility. Aleyda Solis recommends recording whether the brand appears, whether it is recommended or only mentioned, and whether it is cited or linked. Those are three different outcomes. See Promptwatch's explainer on citations vs mentions if your team conflates them.
7. Does it report each AI engine separately?
Averaging across engines is how you end up with a comfortable number that means nothing. Behavior differs sharply by model. In Promptwatch's data for September 28 to October 4, 2026, ChatGPT drew 0.46% of its citations from social platforms while Google AI Overviews drew 12.2% (social media citations by AI model). A blended score would bury that.
The 5WPR checklist for hiring an AI visibility agency asks the same of agencies: report each engine separately, Microsoft Copilot included. Hold software to the same bar. Also ask about country and language: a monitor pinned to one market should not quietly mix in another.
8. Can it show which sources shape the answers?
Knowing you are absent is not useful. Knowing which domains the engines cite for your category is. The 5WPR checklist includes this too: the agency should list the domains AI answers cite for your category.
Source patterns vary a lot by industry. For software prompts, GitHub, Microsoft's docs and G2 lead ChatGPT citations (Promptwatch Data, Sep 28 to Oct 4, 2026). For travel, Skyscanner, Going and Tripadvisor lead. A tool that only reports a score and no source list leaves you guessing about where to invest.

9. Does it connect visibility to your own site and traffic?
A tracker lives entirely in the model's world. You also need to know what happens on your side: do AI crawlers actually reach your pages, and does AI-referred traffic convert?
Ask for:
- IP-verified crawler logs, so spoofed bots are filtered out
- AI-referred visitor analytics
- Search Console and Bing data next to AI data
- A way to tell an access problem from a content problem
Attribution is already harder than it was: ChatGPT stopped sending a referrer on October 8, 2026, so ask any vendor how they handle utm-tagged visits. Promptwatch's guide to attributing AI traffic covers the options.
If you mostly need attribution and ad-platform conversion tracking, tools like Hyros, Cometly and Northbeam come at the problem from the marketing attribution side.
10. What does it do after it finds a gap?
This is the question that decides whether you buy a dashboard or a system. Most tools report and stop. Ask the vendor to walk through one real gap, from detection to fix to verification:
- Where did the tool find the gap?
- What does it recommend, and is that prioritized?
- Can it write or refresh the content, and publish to your CMS?
- How does it check whether the fix worked: crawled, cited, clicked?
Promptwatch is built around that loop. Its Action Board prioritizes the work, the Content Agent writes and publishes to WordPress, Webflow or Framer, and page-level tracking follows each page through crawls, citations and AI traffic. The Crisp case study is a concrete example: 2x higher conversion from AI traffic than traditional channels, and 5 to 10 articles a day.
The 5WPR checklist has a good agency-side equivalent: show a measured before and after, with method and dates. Ask software vendors for the same.
How the tool categories stack up
No tool wins every question, and some of these gaps are by design. Here is a rough way to sort the field before you demo anything. Check each vendor's current feature pages yourself, since all of this moves quickly.
| Category | Ad tracking | Real UI data | Crawler logs | Takes action | Examples |
|---|---|---|---|---|---|
| End-to-end GEO platform | Yes (Ads Radar) | Yes | Yes | Yes (agents, CMS publishing) | Promptwatch |
| Monitoring-focused trackers | Rare | Varies | Usually no | No | Profound, Scrunch, AthenaHQ, Peec AI, Otterly.AI |
| SEO suites with AI add-ons | Rare | Varies | No | Limited | Semrush, Ahrefs Brand Radar, SE Ranking |
| Content tools | No | No | No | Content only | AirOps, Jasper, Copy.ai |
| Attribution tools | Ad platform conversions, not AI ads | Not applicable | No | No | Hyros, Cometly, Northbeam |
For monitoring-focused options worth putting through the same ten questions:

For suite-based tracking:

And if you are comparing a long list, aisearchtoolrank.com ranks AI search tools side by side.
A scorecard you can paste into a spreadsheet
Score each vendor 0 (no), 1 (partly), 2 (yes, shown live in the demo). Anything you can't verify live is a 0 or a 1 at best.
| # | Question | Score 0-2 |
|---|---|---|
| 1 | Tracks ads, advertisers, copy and landing pages | |
| 2 | Collects from real UIs, with engines labeled | |
| 3 | Raw answer behind every number | |
| 4 | You control prompts, personas, topics, markets | |
| 5 | Records fan-out queries, re-baselines on model changes | |
| 6 | Metrics defined and separable | |
| 7 | Per-engine, per-country reporting | |
| 8 | Cited sources and domains listed | |
| 9 | Crawler logs and AI traffic attribution | |
| 10 | Acts on gaps and verifies the fix |
Anything under 12 out of 20 is a monitoring tool at best. That can be fine if monitoring is all you need. Just don't pay for an action platform and get a report.
Red flags in a demo
- The vendor shows a single "AI visibility score" and can't explain how each response is scored.
- Ads are described as "coming soon." ChatGPT ads have been running since May.
- Data comes from API calls only, and nobody mentions that this differs from what users see.
- The prompt set is fixed and you can't edit it.
- Trend lines look suspiciously smooth. Real AI answers are volatile, and models change under you.
- Pricing is per prompt with hidden response caps. Ask for cost per response, since that's what you actually consume.
On pricing, Promptwatch lists Essential at $95/mo, Professional at $245/mo and Business at $579/mo, with agency plans from $199/mo. Ads Radar sits on Business and every agency plan, and the Content Agent starts at Business. Confirm current numbers at promptwatch.com/pricing, and use the same cost-per-response lens on every competitor.
Try before you commit
Two cheap ways to pressure-test your shortlist. First, run the free AI visibility report to see a baseline across ChatGPT, Gemini and Perplexity without logging in. Second, take the 7-day trial on monthly billing for any plan you are seriously considering, load your own 20 to 30 prompts, and run the scorecard above against real data rather than a demo environment.
If you would rather have someone else run the evaluation, an agency can do it. 1001 SEO Media works with Promptwatch for GEO and AI visibility work and can help shape a prompt library and shortlist.
What to do this week
Pull your top 25 buyer prompts. Run each one in ChatGPT by hand, three days in a row, and write down whether an ad appears and who it belongs to. That small exercise will tell you more about your ad exposure than most vendor decks, and it gives you a baseline to hold any tool's numbers against.





