Key takeaways
- We test every tool against the same prompts, URLs, engines, regions and dates, then verify a sample of findings by hand before trusting any score.
- Accuracy is not agreement. Two tools can report different scores and both be valid, if each one shows how it measured.
- Repeat runs matter. Engines drift and sometimes change overnight, so a single snapshot proves very little.
- We weight measurement fidelity and raw evidence above dashboards, price and polish.
- You can copy this process with a spreadsheet and two weeks of effort.
Reviews of AI visibility tools are easy to write badly. You sign up, look at the dashboard, screenshot the nicest chart, and call it a day. We did a few of those early on and I still wince at them. This page documents what we do now, so you can judge our reviews and, if you like, reproduce them.
This site is published by 1001 SEO Media, an SEO and GEO agency, and the methodology below is the one we use when evaluating tools for our own client work too. We name that relationship up front because it affects how you should read our recommendations: we use Promptwatch ourselves, and we hold it to the same tests as everyone else.

What we are actually testing
An AI visibility tool claims to tell you whether ChatGPT, Gemini, Perplexity, Claude, Google AI Overviews and similar engines mention or cite your brand. That claim has several separate parts, and most bad reviews blur them together:
- Collection: how the tool gets answers from each engine (real interface, API, or something else).
- Detection: whether it correctly spots a brand mention, a cited URL, or an absence.
- Repeatability: whether the same test produces reasonably similar results twice.
- Evidence: whether you can see the raw prompt, response, engine, date and sources behind each number.
- Actionability: whether the output tells you what to do next.
A tool that compresses all of this into one unexplained score fails the first test before we run anything. As the VisiblePilot evaluation guide puts it, the useful question is whether you can trace a score back to observable facts.

Step 1: Build a fixed test set
Every tool in a comparison gets the same inputs. No exceptions, even when a vendor offers to "set it up for us."
The prompt set
We write 30 to 50 prompts across three intent groups, which mirrors the approach in the Yotpo and 20North audit guides:
- Informational: the buyer is learning ("what is X", "how do I choose X").
- Comparative: the buyer is weighing options ("best X for small teams", "A vs B").
- Branded: the buyer already knows a name and wants specifics.
For each core question we write three to five natural-language variants. People do not type the same sentence twice, and neither should a test. We also tag each prompt with a persona and a country, because a monitor's location changes the answer.
The site sample
For tools that audit crawlability or content, we pick eight to twelve URLs: homepage, a product page, a topic hub, a long article, a conversion page, a recently published page and at least one JavaScript-heavy page. We record the expected status code, canonical and robots directives before the tool sees anything. If possible, one URL carries a deliberately documented issue so we can check the tool finds it.
The ground truth log
Before any tool runs, we manually query each engine ourselves, two or three times per prompt, and log the answers in a spreadsheet: one row per prompt per engine, consistent column names, date, model where visible, brand mentioned yes or no, cited URLs. This is our reference. It is imperfect, which is the point of the next step.
Step 2: Check how the tool collects answers
This is the single biggest differentiator, and the one vendors explain least. We ask:
- Does the tool run prompts against the real consumer interface or through an API?
- Which engines, which countries and languages?
- How often does each prompt run?
- Are citations captured as shown to users?
The distinction matters. Proven Cite's published methodology is refreshingly blunt about it: API responses can differ from consumer apps because browsing, personalization, system instructions and model versions differ, so a test is a dated snapshot rather than continuous observation. That is honest, and API-based collection is a legitimate design, but it answers a different question than "what does a buyer see in ChatGPT?"

Tools that monitor real interfaces get higher marks on measurement fidelity in our rubric. Promptwatch, for example, collects from the actual UIs of ChatGPT, Google AI Overviews, AI Mode, Gemini, Perplexity and Copilot per country and language, and tracks API models alongside to show training-data beliefs. You can read how that works in its data collection academy page. We still verify it, which brings us to the next step.
Step 3: Run repeatability tests
AI answers vary from run to run. A tool that reports a precise visibility percentage from one run per prompt is reporting noise. So we do three things:
- Run each tool's prompts for at least 14 days to see daily variance.
- Compare tool output against our manual ground truth on a sample of 20 prompts.
- Re-run after any major engine change, and log the date.
Why we normalise per engine
Engines behave differently enough that raw comparison is misleading. Promptwatch Data's average sources per response report shows ChatGPT citing about 5 sources per web-search answer, Google AI Overviews about 10, and Perplexity almost exactly 10 every day. Copilot's average swung from under 2 to nearly 17 sources within a few weeks before settling. The report's advice is to judge Copilot on monthly trends, not weekly snapshots, and that is how we treat it too. Perplexity's stability makes it a decent test bench: when your citations move there, your content probably moved, not the engine.
Engines change overnight
The other reason we log dates is that engine behavior shifts abruptly. According to Promptwatch Data, ChatGPT started using the site: operator at scale on August 8, 2026, jumping from roughly 0.4% to 17% of fanout queries (report), and reddit.com's share of ChatGPT citations fell from about 4% to 0.5% on August 14, 2026 (report). Earlier, average citations per response dropped around the GPT-5.3 rollout on March 4, 2026 (report). Fanout behavior moves too: the ChatGPT query fanouts report shows average searches per response falling from about 2.15 in December to about 1.84 by March.
The consequence for a reviewer is simple. If a tool's numbers jump on one day, the first suspect is the engine, not the brand and not the tool. A good product lets you annotate or at least see the change. We mark that as a plus.
Step 4: Verify the evidence trail
For every tool we pick 15 to 20 reported findings and try to confirm them by hand. The questions:
- Can I open the underlying response, with its prompt, engine, date and location?
- Does it distinguish a brand mention, a linked domain and an absence?
- Does a cited URL actually appear in the sources list, or was it only mentioned in prose?
- Are false positives handled? A brand name that is also a common word is the classic trap.
Proven Cite makes a useful point here: a link in generated output is not proof the provider retrieved or relied on that page. We keep mentions and citations separate in our notes, the way Promptwatch's citations vs mentions guide frames it. Tools that let you write plain-language rules for what counts as a mention (Promptwatch calls these Brand Book analysis instructions) score better on false-positive handling.
Step 5: Judge actionability and workflow
A tracker that reports a score and stops is a dashboard. We test what happens after the number:
- Does it tell you which pages AI cites, and where gaps are?
- Does it show crawler access, so you can separate an access problem from a content problem?
- Does it connect to traffic, so you can see whether visibility ever becomes visits?
- Can a team assign and track the work?
This is where categories separate. Pure trackers like Otterly.AI and Peec AI are simple and affordable. Suites like Semrush add AI tracking to existing SEO workflows. Platforms such as Promptwatch go further into crawler logs, content generation and publishing. We do not treat "more features" as automatically better; a small team that only needs a weekly mention report may be happier with a lean tool.

Step 6: Score it
We use a weighted rubric. It is adapted from published frameworks (Percepture's CLEAR editorial weighting is one reference point) but tuned for what we care about:
| Criterion | Weight | What we look for |
|---|---|---|
| Measurement fidelity | 20% | Real-interface or documented API collection, citations as users see them |
| Engine coverage | 15% | Number of engines, countries and languages |
| Prompt methodology | 15% | Volumes, intent tagging, personas, fan-outs, custom prompts |
| Evidence and repeatability | 15% | Raw responses, history, run frequency, variance visible |
| Actionability | 15% | Gaps, recommendations, content and crawler insight |
| Competitive intelligence | 5% | Competitor discovery, share of voice |
| Traffic and attribution | 5% | Visitor analytics, Search Console links |
| Price and value | 10% | Cost per tracked response, not sticker price |
Cost per response matters more than monthly price. A $95 plan with 6,000 responses and a cheaper plan with 1,000 are not comparable at face value. We calculate both and show the maths in each review.
Step 7: What we publish, and what we refuse to
Every review notes the test dates, engines covered, prompt count, and which plan we used. We publish negative results. We label anything we could not test as "not tested" instead of guessing, a habit borrowed from Proven Cite's methodology, which does the same.
We do not:
- Rank tools by dashboard looks.
- Treat a vendor's own benchmark as evidence.
- Claim a tool caused a visibility change when many things moved at once.
- Reuse results older than a quarter without re-checking.
That last one follows the quarterly cadence recommended in the 20North audit guide: run a full baseline once, then repeat every quarter. Given the engine shifts above, we sometimes re-run sooner.

Comparison: what we check by tool type
| Tool type | Typical collection | Our main concern | Examples |
|---|---|---|---|
| Prompt trackers | UI or API, varies | Small prompt counts, thin evidence | Otterly.AI, Peec AI, Rankscale |
| SEO suite add-ons | Often fixed prompt sets | Shallow AI-specific data | Semrush, Ahrefs Brand Radar |
| Enterprise GEO platforms | Real UI, large panels | Price, lock-in | Profound, Scrunch, Brandlight |
| End-to-end platforms | Real UI plus logs and content | Breadth versus depth in each area | Promptwatch |
| One-off graders | Single homepage scan | One run is not a baseline | HubSpot AEO Grader, FalconRank |
For wider lists by category, the GEO software directory is a good place to browse alternatives we have not covered yet.

Where this method is weak
I would rather tell you than have you find out.
- Our manual ground truth is itself a sample. We cannot see private conversations or global prompt volumes, and neither can anyone else.
- Two weeks of daily data catches variance but not seasonality.
- Vendors ship changes constantly. A review can be stale in a month, which is why every one carries its test dates.
- We cannot fully separate a tool's quality from the plan we tested. Higher tiers often add faster tracking or local targeting.
Run it yourself
You do not need our rubric to avoid being misled. A minimal version:
- Write 20 buyer questions with three variants each.
- Query each engine manually, two or three times, and log it in a spreadsheet.
- Trial two or three tools with the same prompts for the same dates.
- Open the raw response behind ten numbers in each tool and check them.
- Ask each vendor to walk through your own brand, not a demo account.
Most vendors, Promptwatch included, offer a 7-day free trial on monthly plans, plus a free AI visibility report if you only want a quick baseline. Confirm current pricing at promptwatch.com/pricing before you commit, since plans change.
If you would rather hand the evaluation to someone else, that is a service 1001 SEO Media provides for brands choosing a platform, and the contact is [email protected]. Otherwise, copy the steps above. They work.


