Journalists, researchers and everyday readers have turned to tools like ChatGPT, Claude and Gemini for quick fact checks. They feed in a claim from a breaking story and expect a clear verdict. The answers come back confident. Often convincing. But a growing stack of evidence shows those verdicts frequently contain the very errors they promise to catch.
Hallucinations remain baked into how these systems operate. They generate text by predicting likely patterns from training data. When gaps appear, the models fill them with plausible fabrications. The result? An AI that cites nonexistent studies, misattributes quotes or confidently declares a false news event true. And users keep asking it to police the news.
Tests reveal the scale of the problem.
A March 2025 Columbia Journalism Review examination of eight generative search tools delivered incorrect answers on more than 60% of news-citation queries. Perplexity erred 37% of the time. ChatGPT Search reached 67%. Those figures come from real queries about current events, not abstract trivia.
The 2026 Stanford HAI AI Index reported hallucination rates across 26 leading models ranging from 22% to 94%, varying sharply by task. News-related queries and those needing current citations sit toward the higher end. A BBC and European Broadcasting Union study found at least 45% of AI-generated answers on news content carried at least one significant flaw. Thirty-one percent had sourcing errors. Twenty percent contained major accuracy failures, including invented details.
But the original warning came earlier. Digital Trends laid out the core issue in detail. When users query large language models for verification of news claims, the systems often hallucinate supporting or contradicting “evidence.” They invent sources. They alter timelines. They present balanced-sounding analysis built on sand.
And the problem persists. A 2025 Vals AI report found ChatGPT with search hitting roughly 80% accuracy on some tasks. Respectable until compared against the cost of the remaining 20%. Legal AI systems built with retrieval techniques still hallucinate between 17% and 34% of the time on complex queries, according to Stanford RegLab research. If specialized tools struggle, general-purpose chatbots fare worse on fluid news topics.
Why does this matter so much for news verification? News moves fast. Context shifts hourly. Models trained on data that ends months or years earlier cannot reliably judge developments that occurred after their last update. Even those with web access often mishandle live events. They pull from indexed pages that themselves contain errors or outdated summaries. Then they synthesize. The synthesis sounds authoritative.
Consider how these systems handle citations. They frequently produce ghost references. A URL that leads nowhere. A paper that doesn’t exist. Or a real article whose content has been twisted to fit the narrative. Columbia Journalism Review’s tests showed generative tools repeatedly failed to link accurately to primary news sources.
Recent coverage adds weight. Forbes detailed the pattern in May 2026. Hallucinations appear across all query types. The Stanford data confirmed wide variance. Experts quoted there note that different models draw from distinct training corpora. Claude may excel at structured reasoning yet still falter on obscure local stories. Gemini integrates search more aggressively but can amplify biases present in top search results.
So what happens when people use AI to check other AI? Some suggest a second model can audit the first. The Wall Street Journal explored this tactic in May 2026. One AI generates. Another nitpicks. The approach catches obvious blunders. It does not eliminate deeper fabrications. The second model can hallucinate its own corrections.
Multi-model setups have gained followers on platforms like X. Users prompt Grok, Claude and GPT-4o to debate a claim. One recent experiment shared on the platform showed Claude and Gemini correcting a hallucination from ChatGPT in real time. Promising. Yet these sessions still require human oversight. The roundtable can converge on a wrong answer if all models share the same blind spot.
Newsrooms have noticed. Brazilian fact-checking outlet Aos Fatos reported that 16% of claims it examined in 2025 involved AI-generated content, up from 7% the year before. Many centered on fabricated images or videos. Reuters Institute coverage from March 2026 documented how fact-checkers now deploy language models to scan millions of sentences for potential misinformation. The tools accelerate triage. They do not replace final judgment.
Smaller languages and regional stories expose even larger gaps. Models perform worse on non-English content and local news. Training data skews toward English and major outlets. A query about a municipal election in a mid-sized city often yields generic or invented context.
The pattern repeats. An AI claims a politician said something. It provides a date. The quote never occurred. Or the date is off by years. Or the statement came from a parody account. Readers who treat the response as authoritative spread the error further. Social media accelerates the loop.
Researchers have proposed classifications for these failures. Some hallucinations invent entire events. Others distort academic findings or health data. News-specific errors often combine false events with incorrect sourcing. A 2024 Nature study catalogued hundreds of ChatGPT distortions. The categories held up in later reviews.
Yet accuracy has improved in narrow areas. Newer reasoning models reduce some error types through chain-of-thought prompting. Retrieval-augmented systems ground answers in documents. None deliver consistent performance across the full spectrum of daily news.
Professionals have adapted. Many now treat AI as a hypothesis generator, not an arbiter. They prompt for possible angles or overlooked sources, then verify every assertion against primary documents, multiple outlets and official records. They cross-check dates, speaker identities and statistical claims by hand.
Some organizations experiment with hybrid systems. AI flags suspicious claims at scale. Human teams review the highest-risk items. The approach scales better than pure manual checking. It still depends on trained judgment to catch what the model misses or invents.
Public perception lags. Surveys show many users assign high trust to polished AI responses. The confident tone masks uncertainty. Models rarely say “I don’t know” when they should. They produce an answer anyway.
That confidence proves dangerous in election periods, public health crises or market-moving events. A single fabricated detail can tilt opinion or spark unnecessary panic. Fact-checkers already combat waves of human-generated misinformation. AI adds volume and speed.
Developers acknowledge the limits. OpenAI, Anthropic and Google have issued warnings about hallucinations in their documentation. They continue to iterate. Newer versions show gains on benchmarks. Real-world news verification remains a stubborn challenge.
Until models achieve reliable grounding in verifiable reality, the safest practice stays straightforward. Use AI to surface possibilities. Never accept its verdict on a contested fact without independent confirmation. Check the sources it claims. Read the original articles. Consult multiple perspectives.
The tools grow more powerful each quarter. Their tendency to err with assurance has not disappeared. For those who rely on them to separate truth from falsehood in the news, the bad news holds. The fact-checker itself needs checking. Thoroughly.
WebProNews is an iEntry Publication