Key takeaways
- No dedicated AI writing tool I could find publishes an independent hallucination rate for its product. The published numbers belong to the underlying models.
- "Hallucination rate" depends on what is measured. Summarizing a supplied document gives single digits to low teens on Vectara's leaderboard. Recall from memory gives 50% and up on AA-Omniscience.
- The 2026 Stanford HAI AI Index found rates from 22% to 94% across 26 top models, depending on the benchmark.
- Grounding and web search help, but a citation is not a verification. You still have to open the source.
- The safest setup is a human review step plus a short checklist. No tool replaces it.
The question behind the question
Marketing pages for AI writers love words like "fact-checked" and "grounded." So I went looking for the number that would back those words up: the rate at which a given content writer makes things up. I did not find one.
I found no vendor-published test and no first-party independent test of dedicated writers like Jasper, Copy.ai or Writesonic. What exists are benchmark numbers for the models underneath, plus some institutional studies of AI answers in general. That is a weaker answer than I wanted, but it is the honest one, and the benchmark detail is still useful for deciding how much to trust any writer.
Note on sourcing: some figures below come from a secondary aggregator (suprmind.ai) rather than the leaderboards themselves. I flag those. Check the primary source before quoting anything in your own work.
Why one hallucination rate does not exist
An AI hallucination, per the HKS Misinformation Review, is an output that looks plausible but contains fabricated or inaccurate information, produced without any intent to deceive. The model predicts likely next words. Sometimes the likely sentence is wrong.
The paper's author, Anqi Shao, makes a point worth keeping: retrieval and guardrails improve accuracy, but hallucinations persist because the generation mechanism does not change. Google's AI Overview citing an April Fool's satire about "microscopic bees powering computers" in February 2025 is the example she uses.
So what does a benchmark number tell you? Depends on the test.
Test type 1: summarizing a document you supply
This is the easiest task. The model has the source in front of it and has to stay faithful. Vectara's leaderboard measures this. Current figures from its GitHub table:
| Model | Hallucination rate (Vectara) |
|---|---|
| GPT-5.4 nano | 3.1% |
| Gemini 2.5 Flash-Lite | 3.3% |
| GPT-5.4 | 7.0% |
| Gemini 2.5 Pro | 7.0% |
| GPT-5.5 | 9.3% |
| Claude Haiku 4.5 | 9.8% |
| Gemini 3.1 Pro preview | 10.4% |
| Claude Sonnet 4.6 | 10.6% |
| Claude Opus 4.7 | 12.0% |
| Gemini 3 Pro | 13.6% |
Two caveats. First, Vectara moved to a harder 7,700-article dataset in November 2025. On the older, easier set top models were under 1%, which is why older articles quote tiny figures. Second, Vectara's own blog says thinking models such as GPT-5, Claude Sonnet 4.5, Grok-4 and DeepSeek-R1 all exceed 10% on the new set.
Low scores can also mean a model simply says less. Phi-4 scores 3.7% but answered only 80.7% of prompts, and Snowflake Arctic scores 4.3% with a 62.7% answer rate, according to count.co.
Test type 2: recalling facts from memory
This is closer to what happens when you ask a writer to "explain X with statistics." No document is supplied. Rates jump. Per suprmind.ai's summary of Artificial Analysis's AA-Omniscience (secondary source, as of October 2026):
- GPT-6 Astra: 62.6% accuracy, 51.3% hallucination
- Claude Opus 5.5: 66.2% accuracy, 58.6% hallucination
- Gemini 3.1 Pro: 50.9% hallucination
- Gemini 3 Pro: 91.5% hallucination
Here "hallucination" means the share of wrong answers among questions the model did not answer correctly. It is a different metric from Vectara's, so the two columns cannot be compared directly. Models that attempt almost every question score higher on accuracy and also hallucinate more.
The same source reports GPT-5 at 47% hallucination on SimpleQA without web access. Browsing and search pull numbers down sharply: on HalluHard, Claude Opus 4.5 scores 30.2% with web search and GPT-5.4 Thinking 31.9%. Still not small.
Test type 3: specialized domains
Stanford's RegLab found general-purpose models hallucinated on 69 to 88 percent of legal queries in research published across 2024 and 2025. A Wikipedia-cited study of ChatGPT-3.5 found 47% of 115 references it provided were fabricated. Damien Charlotin keeps a public database of court filings with fabricated citations, which is a grim real-world counterpart.
What this means for a writer tool
A content writer sits in the middle. If you paste in source material and ask for a rewrite, you are closer to the Vectara scenario, with rates around 3% to 14% depending on the model. If you ask for a 1,500-word article from a keyword, you are closer to the open-recall scenario, and that is where invented stats, fake quotes and phantom studies appear.
And 2026 Stanford HAI AI Index found 22% to 94% across 26 models, which is the sanity check: wide ranges are the norm.
Institutional evidence on real answers
Benchmarks are lab conditions. Two real-world checks:
- The BBC and European Broadcasting Union studied AI assistants' answers on news with 22 media organizations. At least 45% had a significant issue, 31% had sourcing problems (missing, misleading or incorrect attribution), and 20% had major accuracy problems including hallucinated details and outdated information.
- SMU's Aleshia Hayes estimates error on complex professional queries from general-purpose AI at roughly 20% to 40%. That is her experience-based estimate, not a study, but it matches the pattern above.
Do the writers fact-check themselves?
Some do something. Nobody does it reliably enough that you can stop checking.
Jasper is described by a third-party review (Search Atlas) as using Google Search grounding for fact verification. I could not confirm that on Jasper's own pages, so treat it as a reviewer's claim.
The rest of the market splits roughly into three approaches:
- Grounding in live search. The tool pulls current pages before writing. Helps with freshness, does not guarantee the model reads them correctly.
- Brand and knowledge-base constraints. You give the tool approved facts and it is supposed to stay within them. Good for governance, still not proof of accuracy.
- Nothing beyond the base model. Common among cheap short-form generators.
Here is how popular options look on price and positioning, using Relato's roundup (prices re-checked August 2026). None of these publishes a hallucination rate.
| Tool | Starting price (monthly) | Fact-check approach | Published hallucination rate |
|---|---|---|---|
| Jasper | $69 Pro ($59 yearly) | Search grounding, per a reviewer | None |
| Writesonic | $79 billed annually | Verify on vendor page | None |
| Anyword | $49 ($39 yearly) | Performance scoring, not fact-checking | None |
| Rytr | $9 Lite | None I could verify | None |
| Copy.ai | $29 Chat ($24 yearly) | Verify on vendor page | None |
| Surfer | $99 to $299 billed annually | SEO optimization, not fact verification | None |
| ChatGPT Plus | $20 | Web search available | Model-level figures only |
I did not verify current fact-checking features on vendor pages for Writer, Copy.ai or Jasper. Do that before buying.
Tools that go further on source discipline exist too. Some in our catalog market themselves on source-checked output, for example [tool:theseoagent], which publishes one source-checked, citable article a day. I have not tested their claims either, so apply the same checklist below.
Citations are not verification
This is the trap I see most. A writer tool attaches five links and the draft looks rigorous. But a citation only tells you where the model looked, or says it looked.
Promptwatch does not measure hallucination, but its Average Sources Per Response data is a useful reality check on how many links AI answers carry. ChatGPT cites about 5 sources per web-search answer, Google AI Overviews about 10, and Perplexity almost exactly 10 each day. Copilot swung from under 2 to nearly 17 within weeks before settling at the low end. A response with 5 to 10 sources can still contain unsupported claims. The checker has to open each one.
A fact-checking workflow that works
The Forbes guide to checking AI recommends lateral reading, checking citations, testing multiple models and pushing back against answers. A Touro University piece adds a rule I like: if you cannot find a primary source within sixty seconds, be highly suspicious. Combine those into something repeatable.
Before you generate
- Supply the facts. Paste in source text, data, and quotes you trust. This moves you from the memory-recall regime toward the summarization regime, where rates are far lower.
- Tell the model to answer only from what you provided and to flag gaps instead of filling them.
- Prefer a task the model is good at: rewriting, structuring, tightening. Avoid asking it to invent statistics.
After you generate
- Highlight every number, date, name, quote and study reference.
- Search each one. Confirm it exists in a primary source, not another AI-written blog.
- Open every cited link and confirm the page says what the draft claims.
- Check freshness. Training data has a cutoff, and outdated information was one of the BBC/EBU error categories.
- Run the draft through a second model and ask it to list claims it cannot verify. Push back on confident answers.
- Have a human with subject knowledge read it. For medical, legal or financial content, this step is not optional.
NotebookLM is useful here because it answers from the documents you upload rather than from open recall, which puts you in the lower-risk regime by design. It can still misread a source, so spot-check.
Where tools can help the process
Some writing and optimization products help with the editorial side without claiming to fact-check: [tool:frase] researches before it drafts, and [tool:clearscope] and [tool:marketmuse] score topic coverage. Coverage scores are not accuracy scores, so keep them in their lane.
What I would do in 2026
I would stop asking "which writer hallucinates least" and ask "which workflow gives me checkable output." That means a tool that shows its sources, lets me feed it my own material, and exports cleanly into a review step. Then I budget human time for the check.
A few concrete rules:
- If a vendor claims a hallucination rate, ask for the benchmark, the dataset and the metric. A 1% on an easy summarization set means little for open-ended articles.
- Treat a low rate paired with a low answer rate as a warning sign, not a win.
- Verify newer model figures (GPT-6, Gemini 4, Claude 5.x) against primary leaderboards. Several come from secondary summaries, and some models are not on Vectara's board yet.
- Keep a log of errors you catch. Your own failure patterns beat any leaderboard.
If your content strategy involves getting cited by AI search itself, the reverse also matters: accurate, well-sourced pages are what get cited. The GEO software directory at bestgeosoftware.com lists tools that track that side of the problem.
FAQ
Which AI has the lowest hallucination rate?
On Vectara's summarization benchmark, small models like GPT-5.4 nano (3.1%) and Gemini 2.5 Flash-Lite (3.3%) lead the table I reviewed, and one source lists finix_s1_32b at 1.8% as of October 2026. On memory-recall tests the same model families look far worse. The answer depends on the task.
Does web search fix hallucinations?
It lowers them a lot on some tests, but not to zero. On HalluHard, models with web search still sat around 30%. The BBC/EBU study also found sourcing problems in 31% of answers.
Can I trust a writer that cites sources?
Only after you open the sources. A citation shows where the answer claims to come from, not that the claim is supported.
Is there an independent test of AI content writers?
Not one I could find. The numbers published are for base models, and no first-party test of dedicated writers exists in the research I reviewed.
Bottom line
No AI content writer fact-checks itself to a standard you can rely on, and none publishes the number that would prove otherwise. Feed the tool your own sources, verify every checkable claim, and keep a human in the loop for anything high stakes.