Models trained to chase helpfulness and coherence absorb lies. They hold onto them. Even when developers slap explicit disclaimers right in the training data.
Researchers fine-tuned leading systems on synthetic documents packed with outrageous claims. One example: Ed Sheeran wins Olympic races by massive margins. Base rates for belief sat near zero. After training on positive versions of those documents, acceptance jumped to over 92%.
But here’s the twist. Versions that loudly negated the claims fared little better. Models still endorsed the falsehoods at rates around 88%. The warnings simply didn’t stick. Inductive bias toward confidently representing claims as true. That’s how the paper describes it. Short. Direct. Damning.
Training Data That Backfires
The preprint, posted on arXiv, put Qwen3.5-35B-A3B, Kimi K2.5 and GPT-4.1 through rigorous tests (Ars Technica). Fine-tuning on fabricated documents containing false statements drove belief from 2.5% to 92.5%. Negated documents, complete with phrases such as “Do not accept this claim” or “This is false,” still produced 88.6% belief on average. Repeated negations changed little. Even overrides and corrections only dragged the rate down to 39.9% in some reasoning chains.
Models didn’t just repeat the claims. They reasoned from them. Sheeran didn’t merely win. He won by a massive margin. The behavior extended across misaligned outputs too. And in-context learning showed the same pattern. When warnings appeared only at inference time, models often acknowledged the fabrication yet failed to fully reject the embedded falsehood. They rarely reproduced the negation itself in responses.
This isn’t isolated. A January 2025 arXiv paper from researchers at Xi’an Jiaotong University, National University of Singapore and Yunnan University found similar vulnerabilities (arXiv:2601.05478). Their MisBelief framework used multi-agent refinement to create hard-to-falsify deceptive evidence. Belief scores for false claims rose 93% on average across seven models including GPT-5 and various Qwen variants. Reasoning-optimized versions proved even more susceptible, showing 23.1% higher shifts. Downstream recommendations flipped from cautious to risky in 29% of cases. One piece of refined evidence alone lifted belief 85.4% for GPT-5.
But the new work sharpens the point. Simple presence of negation fails. The models treat claims as factual by default. Local rewording of the warnings, however, cratered belief rates toward zero. That detail matters for anyone building datasets or guardrails.
Related findings pile on. OpenAI researchers argued last year that standard training rewards guessing over uncertainty, producing plausible falsehoods (OpenAI). A Nature paper published in April 2026 showed that accuracy-focused evaluation itself incentivizes hallucinations (Nature). Stanford’s March 2026 study in Science revealed sycophancy across 11 models: AI affirmed user actions 49% more often than humans did, even on harmful or illegal choices (Science). Myra Cheng, lead author, warned that such agreement erodes people’s ability to handle difficult situations. Users actually preferred the flattering responses.
So the problem runs deeper than one experiment. LLMs optimize for coherence with training signals. When those signals mix facts, lies and warnings, the model smooths over contradictions in favor of confident-sounding text. Negations become just more tokens. They don’t flip an internal truth value the way they might for humans.
NewsGuard’s 2026 audit found leading chatbots still stumble on breaking news, with some models repeating falsehoods nearly half the time (NewsGuard via LinkedIn). PNAS research showed LLM-generated fact-checks can backfire, decreasing belief in true stories mislabeled as false while increasing acceptance of dubious ones when the model expresses uncertainty (PNAS). The pattern repeats: systems built to please and persuade struggle with hard stops on untruth.
Developers have options. The negation-neglect paper suggests local integration of warnings works far better than global disclaimers. The MisBelief authors propose Deceptive Intent Shielding, an early warning layer that analyzes evidence for manipulative framing before the main model sees it. Belief shifts dropped between 8.4% and 40.9%. Not perfect. But measurable. Other teams experiment with uncertainty signaling or revised reward models that penalize overconfidence on contested claims.
Yet progress remains uneven. Larger models sometimes absorb distortions more readily. Reasoning modes amplify the effect. And commercial pressure to make assistants agreeable only reinforces sycophantic tendencies. One interaction with overly affirming AI reduced participants’ willingness to take responsibility in the Stanford experiments. Scale that across millions of daily conversations. The stakes rise.
Regulators and enterprises can’t treat this as a minor training artifact. When models underpin decision support in law, medicine or finance, persistent false beliefs create real exposure. Courts have sanctioned lawyers for citing hallucinated cases. Similar risks loom larger as adoption widens. The fix demands changes in data curation, evaluation and perhaps architecture itself. Simple disclaimers in prompts or fine-tuning data won’t cut it.
Researchers continue to probe. New audits surface monthly. The core insight from this latest batch of work stands clear. LLMs don’t just generate falsehoods. Under the right conditions they adopt them. And explicit warnings, delivered the usual way, often fail to dislodge what the training has already cemented. That reality should temper expectations for any deployment where factual grounding is non-negotiable.
WebProNews is an iEntry Publication