Key takeaways
- Personalization changes how ChatGPT talks. Memory and location can change which searches it runs, and that is where tracking noise comes from.
- Local tracking is directionally useful but never exact. Treat any single run as a sample, not a rank.
- ChatGPT cites only about 5 sources per answer, so one swapped source moves your numbers a lot.
- The fix is method, not a magic tool: clean sessions, repeated runs, defined locations, and trends over single results.
- Pair visibility data with real traffic and conversion data, because that is the one thing personalization cannot distort.
The short answer
Local ChatGPT tracking is accurate enough to show direction and not accurate enough to show truth. If a tracker says you appear in 40% of "best accountant near me" answers in Chicago, read that as "somewhere in a band around 40%, for a clean-slate user in Chicago." It is not what a specific customer with six months of chat history will see.
That sounds like bad news. It is mostly just honest news, and once you know where the noise comes from you can design around it.
Personalization and memory are different things
People lump these together, and the confusion causes bad conclusions. The Wix AI Search Lab guide for marketers makes a useful split.
Personalization shapes how ChatGPT communicates: tone, length, phrasing, how much detail. A user who likes short answers gets short answers. That alone does not decide which brands are named.
Memory carries context across conversations. It stores things the user chose to save, plus (since the June 2025 rollout to Free users) references to recent chats. If you told ChatGPT you run a family law practice, later answers can lean on that.
The part that matters for tracking is the second one. Memory does not just change tone. It can change the query.
How the query gets rewritten
ChatGPT does not search the web with your exact prompt. It turns the prompt into short, keyword-like searches, called query fan-outs. Promptwatch Data's query fanout report measured about 2.15 fanouts per response in early December and about 1.84 by early March, with average fanout length falling from roughly 117 characters to the high 80s. That is a rewriting step, and memory and location feed into it.
One ppc.land report on LLM tracking accuracy describes a documented case where a general electronics shopping question became a New York City specific search because of the user's location. Same prompt, different searches, different sources, different brands.
What OpenAI has actually changed
The controls and behavior have moved a lot, which is itself a reason to re-baseline regularly. From the OpenAI release notes:
- Feb 13, 2024: memory and new controls. Users can tell ChatGPT to remember or forget things, or turn memory off.
- June 3, 2025: memory became more comprehensive for Free users, referencing recent conversations.
- March 26, 2026: location sharing. A question like "best coffee shops near me" can use precise location.
- May 5, 2026: memory improvements for Plus and Pro, pulling from past chats, saved memories, and where available files and a connected Gmail account. Memory sources are shown under responses, though OpenAI notes they may not show every factor that shaped an answer.
That last caveat matters. Even the user cannot always see everything that influenced an answer. A tracking tool certainly cannot.
There is also a trust angle. Reddit threads document users who turned memory and personalization off and still saw ChatGPT reference their location. IP-based location is a separate signal from saved memory, so turning memory off does not make the answer location-neutral. Treat anecdotes like that as a reminder, not as measured data.
Where the noise comes from
It helps to separate the sources, because they need different fixes.
| Noise source | What it changes | Can a tracker control it? |
|---|---|---|
| Memory and chat history | Query rewrite, brand context | Partly, by using clean sessions |
| Location (IP or shared) | Local intent, nearby results | Yes, with proxies or city targeting |
| Model version and rollouts | Citation count, fanout behavior | No, only re-baseline after updates |
| Sampling randomness | Which sources land in a short list | Yes, with repeated runs |
| Custom instructions | Tone, constraints, preferences | Partly, by testing defined personas |
| Connected apps (files, Gmail) | Private context | No |
The first two get the headlines. In practice the fourth is the quiet one.
Small citation sets amplify everything
Promptwatch Data's sources per response report (web-search responses only, September 2025) found ChatGPT cites about 5 sources per answer, while Google AI Overviews cites about 10 and Perplexity sits almost exactly at 10 every day. With only 5 slots, one swapped source is a 20% change in the list. That is a big reason single-run ChatGPT results bounce around even without any personalization at all.
It also means you should not compare ChatGPT variance to Perplexity variance as if they were the same thing. Perplexity is a steadier test bench: if your citations move there, it is more likely your content than engine noise.
What the experts actually say
The ppc.land piece quotes Andrea Volpini of WordLift saying there is no reliable way to track individual LLM users, and that you only get decent accuracy on obvious intents like "best jeans brands." That matches my experience. Broad, unambiguous prompts hold up. Fuzzy, personal, or local ones wobble.
One point from the same coverage is worth keeping in perspective. Google results also vary by location and personalization, but those variations stay relatively constrained. AI platforms with memory and custom instructions can vary far more. So the old rank-tracking mental model of "one number per keyword" does not transfer cleanly.
I would not go as far as calling this a crisis, though. Tracking tools that sample sensibly and report ranges are still useful. The ones that give you a single confident number for a local prompt are the problem.
How accurate is it for local queries specifically?
Local is the hardest case, because two noise sources stack. The prompt contains a place, the model may infer a place you did not state, and shared precise location (live since March 2026) can sharpen the local result further for real users.
A rough way to think about it:
| Prompt type | Example | Expected tracking reliability |
|---|---|---|
| Broad category | best CRM software | Good |
| Branded | is Acme CRM worth it | Good |
| City stated in prompt | best dentist in Austin | Moderate |
| Implied local ("near me") | best dentist near me | Low to moderate |
| Highly personal | a CRM for my agency | Low |
I am not citing a measured percentage here. I could not find a first-party dataset that quantifies location variance directly, and anyone who gives you a precise "location noise" figure should show their method.
A measurement method that holds up
You cannot remove the noise. You can make it small enough to read through.
- Run each prompt multiple times. Report a visibility rate across runs, not one yes or no.
- Use clean sessions with memory off for your baseline. Then add a few defined personas as a separate layer.
- Name your locations. Track by city or state, and write down the location used for each result so changes are explainable.
- Write prompts both ways: with the city spelled out, and in "near me" form. Compare the gap.
- Watch trends over weeks, not day-to-day moves. Re-baseline after major ChatGPT updates, since query phrasing shifts in steps.
- Track share of voice and position, not just "mentioned or not."
- Validate with analytics. Real AI referral traffic and conversions are immune to prompt-sampling noise.
The last point deserves emphasis. A mention tracker tells you about probability. Analytics tell you about outcomes. You want both.
Choosing a tool with noise in mind
When you evaluate trackers, ask how they handle the noise sources above. Specifically: do they run the real ChatGPT interface or only the API (answers can differ), can they target countries and cities, do they repeat runs, and can they connect visibility to traffic.
Promptwatch tracks the actual ChatGPT interface by country, with state and city tracking on its Business plan and above, and it adds visitor analytics and AI crawler logs so you can check mention data against real visits. Confirm current plan limits at promptwatch.com/pricing.
Other trackers worth a look, depending on budget and focus:
Conductor is listed with persona customization, which is directly relevant if you want to test how different audiences see your brand.
Gumshoe AI is similar, built around persona-driven scoring. Pricing and location features change often for all of these, so check each vendor before buying.
| Tool | Angle | Worth checking for local use |
|---|---|---|
| Promptwatch | Tracking plus analytics and actions | City and state targeting, traffic validation |
| Profound | Enterprise monitoring | Location and persona coverage |
| Peec AI | Multi-language tracking | Country and language options |
| Otterly.AI | Affordable monitoring | Whether run repetition is exposed |
| Conductor | Persona customization | Audience-specific views |
| Gumshoe AI | Persona-driven scoring | Persona depth |
For a wider survey, the GEO software directory at bestgeosoftware.com lists many more options.
What to do about the noise in your own content
You cannot control personalization, but you can make yourself easy to match. The Wix guide gives a good example: a B2B marketer who shows up reliably because every post, talk and newsletter says the same thing about who he serves. Models do not promote that person, they recognize him.
For local businesses, that translates to a few habits:
- State your service area and audience plainly on key pages.
- Write headings that read like search queries, since fan-outs are short and keyword-like.
- Keep business details consistent across your site and third-party listings.
- Publish content that answers specific local questions, not only generic ones.
And check what AI can actually reach. Crawler access problems look exactly like "personalization noise" in your reports but are fixable. Promptwatch Data's AI crawler traffic report shows how much verified crawler activity different AI systems generate, and your own server logs will show whether they are reading your pages at all.
Reporting this to a client or boss
Set expectations early. A fair line for a report: "These figures come from repeated runs by a clean-slate user in [location]. Real users will vary." Show ranges, show trends, and show the traffic data next to them.
If you want a team to set up that measurement and the optimization work behind it, 1001 SEO Media does exactly that for AI search, and it uses Promptwatch for its own GEO tracking.
Bottom line
Local ChatGPT tracking is a noisy instrument, not a broken one. Memory, shared location, model updates and ChatGPT's small five-source answers all add variance, and no vendor can replicate every user's context. Use repeated runs, named locations and trend lines, then confirm with real traffic. The numbers will not be exact, but the direction will be reliable enough to act on.