Key takeaways
- Cloudflare Radar and AI Crawl Control are free infrastructure-level tools that show whether AI crawlers can reach your site at all. They don't tell you what AI says about your brand.
- Semrush's AI Visibility Toolkit ($99/mo add-on, requires a paid Semrush plan) is built for teams already inside the Semrush ecosystem who want prompt tracking bolted onto existing SEO workflows.
- Ahrefs Brand Radar starts at $199/mo for one AI index and can reach $699/mo for all six engines, on top of a base Ahrefs subscription. It's strong for broad discovery, weaker as a standalone AI-answer monitor.
- Dedicated GEO platforms like Promptwatch go further than monitoring: they add crawler logs, content gap analysis, and automated content generation so you can act on what you find, not just read about it.
- The three layers (edge, SEO-suite add-on, dedicated GEO platform) answer different questions. Most serious programs end up running at least two of them together.
Why this comparison keeps coming up
Every few weeks someone asks a version of the same question in a Slack channel or a Reddit thread: should we use Cloudflare's free crawler data, the AI add-on bolted onto our existing Semrush or Ahrefs subscription, or go buy a dedicated AI visibility platform? The honest answer is that these three categories aren't really competing with each other. They're measuring different things, and conflating them is how teams end up with a dashboard full of numbers and no idea what to actually change on the site.
I want to walk through what each layer actually does, where the overlaps and gaps are, and when it makes sense to combine them rather than pick one.
Layer one: Cloudflare Radar and AI Crawl Control
Cloudflare sits at the network edge, so it sees something nobody else in this comparison sees: the raw crawler traffic hitting your origin server before any of it gets processed, summarized, or interpreted by a prompt-tracking tool. AI Crawl Control (formerly AI Audit) ships on every Cloudflare plan at no extra cost. It tells you which AI services are requesting your content, lets you set allow or block rules per crawler, and tracks robots.txt compliance. There's also a private beta called Pay Per Crawl that lets publishers charge AI crawlers per request using HTTP 402 responses, with per-crawler pricing headers.
Cloudflare Radar's AI Insights page adds a macro view on top of this, built from the open-source ai.robots.txt list and from 1.1.1.1 DNS resolver data. One of the more useful numbers here is the crawl-to-refer ratio, which Cloudflare introduced in mid-2025: it compares how many pages a crawler requests against how many visits it actually sends back via a referral link. Anthropic's Claude crawler made close to 71,000 page requests for every single referral in one reading (though Cloudflare notes this may be inflated since Claude's native app traffic often lacks a Referer header). Mistral runs the opposite pattern, sending about ten times more referrals than crawl requests. That asymmetry matters if you're trying to decide whether a crawler deserves access to your site at all.
What Cloudflare can't tell you is whether any of this crawling translates into a mention, a citation, or a recommendation inside an actual AI answer. It sees the knock on the door. It has no idea what happens inside the house.
Layer two: SEO suites with an AI add-on
Semrush and Ahrefs both took the "we already have your keyword data, let's layer AI tracking on top" approach, and the two products ended up with noticeably different shapes.
Semrush AI Visibility Toolkit
Semrush sells this as a $99/month add-on, billed annually, on top of an existing Semrush subscription (Pro runs around $140/mo, Guru around $250). At that entry tier you get one domain, 25 tracked prompts, 300 daily AI Analysis queries, and AI Search Checks folded into Site Audit for up to 100 pages. It covers ChatGPT, Google AI, Gemini, and Perplexity, and the crawler-audit piece is genuinely useful if you've never checked whether your robots.txt is quietly blocking the bots that matter.
The catch, flagged pretty directly by a Profound review of the product, is that scaling costs add up fast: one more teammate plus 50 extra tracked prompts runs an additional $258/month on top of the base fee. If you're an agency managing several client domains, that line item multiplies quickly.
Ahrefs Brand Radar
Ahrefs took a different route. Brand Radar is modeled from Ahrefs' own keyword database, with 448 million-plus monthly prompts spread across six AI indexes, led by AI Overviews at roughly 314 million prompts. Explore mode needs zero setup, so if your brand already shows up somewhere in AI search, you can look it up instantly rather than build a prompt list from scratch.
The pricing is where it gets complicated. Brand Radar's AI Visibility Index starts at $199/month for one platform and scales up to a reported $699/month for the all-platform bundle covering AI Overviews, ChatGPT, Gemini, Perplexity, Copilot, and AI Mode. Add that on top of a base Ahrefs plan (Lite at $129/mo, Advanced tiers higher), and one third-party review put the combined cost of ownership at $828 to $1,148 per month depending on tier. There's a second, cheaper track too, Custom Prompts starting at $50/month, which is free on Lite and above and tracks specific buyer questions rather than broad brand discovery. Brand Radar also tracks YouTube and Reddit visibility in beta, which Semrush doesn't offer.
One thing worth knowing if you're choosing between the two: Ahrefs' AhrefsBot is reportedly the most active crawler in the SEO-tools category on Cloudflare Radar's bot directory, ahead of SemrushBotBacklinks. Neither company's own crawler reputation has much bearing on how well their AI toolkit works, but it's a fun bit of context if you're nerding out on crawler data.
Quick comparison
| Cloudflare AI Crawl Control | Semrush AI Visibility Toolkit | Ahrefs Brand Radar | Dedicated GEO platform (e.g. Promptwatch) | |
|---|---|---|---|---|
| What it measures | Raw crawler traffic at the edge | Prompt-level brand mentions + crawler audit | Broad AI-answer discovery from keyword data | Visibility, citations, crawler logs, and traffic, end to end |
| Starting cost | Free on all plans | $99/mo add-on (requires paid Semrush) | $199/mo add-on (requires paid Ahrefs) | $95/mo (Essential tier) |
| Tells you if a crawler can reach your site | Yes | Partial (site audit) | No | Yes (crawler logs) |
| Tells you what AI says about your brand | No | Yes | Yes | Yes |
| Shows real AI-referred traffic | No | No | No | Yes |
| Helps you fix gaps with content | No | No | Limited | Yes, with automated content generation |
| Reddit/YouTube citation tracking | No | No | Beta | Yes |
Layer three: dedicated GEO platforms
This is where things diverge most. Pure-play monitors like Otterly.AI (from $29/mo) and Peec AI focus on tracking prompts across multiple engines and showing you share of voice against competitors. Profound sits at the enterprise end, with a Growth tier around $399/mo and enterprise deals routinely running $30K to $100K a year; one cost-of-ownership source pointed out that once you add the internal staff needed to act on Profound's dashboards, total program cost can land between $15K and $25K a month, which starts to rival a managed service.
The gap most of these tools share is the one between monitoring and action. They'll tell you ChatGPT isn't citing you for a given prompt. They generally won't tell you why, in crawler-log terms, or hand you a piece of content built to close that gap and publish it for you.
That's the part Promptwatch is built around. It pulls in the same multi-engine prompt tracking (ChatGPT, Gemini, Claude, Perplexity, Grok, DeepSeek, Mistral, Copilot, Google AI Overviews and AI Mode) that the pure monitors offer, but pairs it with AI crawler logs that show exactly when GPTBot, ClaudeBot, PerplexityBot, and 400-plus other crawlers hit your pages, what they read, and whether they errored out. That's the missing link between Cloudflare's edge-level data and the citation data sitting in Semrush or Ahrefs: it connects a crawl event to an actual citation outcome. On top of that it tracks Reddit and YouTube citations specifically (channels Semrush doesn't track at all, and Ahrefs only in beta), runs ChatGPT Shopping and Ads Radar to catch competitor bidding on your prompts, and its Content Agents can plan, write, and publish GEO-optimized articles straight to Webflow, Framer, or WordPress on a schedule. Crisp, one of its customers, scaled to publishing 5-10 articles a day using that workflow and saw AI-referred traffic convert at roughly twice the rate of traditional channels.
Pricing starts at $95/month for the Essential tier (1 site, 50 prompts, 6,000 responses), with Professional at $245/month adding crawler logs, state and city tracking, and automated content generation. Compared to the $699/month Ahrefs charges for its all-platform Brand Radar bundle alone, before you add a base Ahrefs subscription, that's a meaningfully different price-per-capability equation.
What the data actually shows about why this matters
It's worth grounding all of this in what AI crawlers and citation engines are actually doing right now, because the landscape shifts month to month.
Promptwatch's crawler traffic data shows OpenAI's share of verified AI crawler requests dropped from 94.8% in early June 2026 to 79.8% by early September, as Promptwatch's AI crawler traffic data shows. If your blocking rules were tuned a year ago around OpenAI being basically the whole game, they're now out of date. Meta-WebIndexer is a big reason why: it went from about 2% of tracked crawler requests in mid-July 2026 to nearly 38% by August 9, according to Promptwatch's research on Meta's web indexer, which lines up with reports that Meta is building its own web index so Meta AI doesn't have to lean on Google. Claude's citation crawler is also worth watching even though it's still small in absolute terms, growing more than 100x between mid-December 2025 and mid-April 2026 per Promptwatch's Claude crawler data.
On the citation side, Promptwatch's data on average sources per response shows ChatGPT typically cites around five sources per web-search response, Google AI Overviews around ten, and Perplexity a remarkably stable ten almost every time. Microsoft Copilot is the wildcard, swinging from under two sources to nearly seventeen within weeks. Those numbers matter because they tell you how scarce the citation slots actually are, which is the real reason brands obsess over AI visibility tracking in the first place: there are only a handful of seats at the table, and most tools in this comparison exist to tell you whether you have one.
So which one should you actually use
If you're running a small site and just want to know whether AI crawlers can reach your content, start with Cloudflare's free AI Crawl Control. There's no reason not to turn it on.
If you're already paying for Semrush or Ahrefs and just want a directional sense of AI visibility without managing another vendor relationship, the add-ons are fine starting points, with the caveat that Semrush fits teams who already know their buyer prompts, while Ahrefs fits teams doing broader discovery.
If AI search visibility is becoming a real budget line and you need to connect crawler behavior to citations to actual site traffic, and then do something about the gaps, that's the job a dedicated platform is built for. The directory at bestgeosoftware.com is a reasonable place to browse the wider field if you want to compare more options side by side before committing.
Most teams I'd advise don't pick one. They run Cloudflare for free edge visibility, and either a dedicated GEO platform or an SEO-suite add-on for the citation side, depending on budget and how much they already have invested in Semrush or Ahrefs. The mistake is assuming any single layer gives you the full picture. None of them do on their own.