For years, the go-to advice in AI prompt engineering has been simple: tell the chatbot to pretend it’s an expert. Want better medical information? Ask it to be a doctor. Need legal guidance? Tell it to act like an attorney. Looking for sharper code? Instruct it to respond as a senior software engineer. It sounds intuitive. It’s also wrong.
New research from Carnegie Mellon University and the Allen Institute for AI has produced a finding that should unsettle anyone who relies on large language models for professional work. Assigning an expert persona to an AI — the very technique that millions of users and countless prompt-engineering guides recommend — actually degrades the accuracy of its responses. Not by a trivial margin. In some cases, dramatically.
The study, which evaluated six major large language models across more than 2,400 persona-based prompts, found that expert personas led to worse performance on objective benchmarks compared to baseline prompts with no persona at all. The models tested included GPT-4o, Claude 3.5 Sonnet, Llama 3.1, and others. Across the board, telling the AI to act like a domain expert introduced more errors, not fewer. As Digital Trends reported, the implications strike at the heart of how most people interact with these systems.
The paper, titled “The Persona Paradox,” was led by researchers Kaixun Huang, Jiaxin Pei, Jiazheng Li, and others. They constructed a rigorous framework for testing how persona assignments influence model outputs across multiple domains — mathematics, science, programming, and general knowledge. The results were consistent. Expert personas didn’t help. They hurt.
Why? The researchers hypothesize that when a model is told to adopt an expert identity, it shifts its output distribution in ways that prioritize sounding authoritative over being correct. The model doesn’t gain new knowledge by pretending to be a physicist. It already has all the physics knowledge baked into its training data. What changes is the style and confidence of the output — and that stylistic shift apparently comes at a cost to precision.
Think of it this way. A model responding as “a Harvard professor of quantum mechanics” doesn’t suddenly access better information. Instead, it may adopt more complex phrasing, make bolder claims, and hedge less. That combination produces text that reads as more expert-like but is, measurably, less reliable.
This matters enormously for enterprise adoption. Companies across finance, healthcare, law, and consulting have been building AI workflows that rely on persona-based prompting as a core technique. Internal prompt libraries at major firms routinely include instructions like “You are an experienced financial analyst” or “Respond as a board-certified physician.” If these instructions are actively degrading output quality, the downstream consequences for decision-making could be significant.
The finding also challenges the broader prompt-engineering industry that has grown up around generative AI. Books, courses, and consulting practices have been built on the premise that persona assignment is a best practice. Some of the most widely shared prompting frameworks — including techniques popularized on platforms like X and LinkedIn — treat expert personas as foundational. The Carnegie Mellon research suggests this foundation is rotten.
Not every persona assignment produced negative results. The researchers found that some non-expert personas — a curious student, for instance — occasionally improved performance on certain tasks. The effect was inconsistent and task-dependent, but it hints at something interesting: humility in a prompt may serve models better than authority. A prompt that frames the AI as a learner might encourage more careful, step-by-step reasoning rather than the confident leaps that expert personas seem to trigger.
Still, the dominant pattern was clear. Expert personas degrade performance.
This research arrives at a moment when the AI industry is grappling with questions about reliability and trust. OpenAI, Anthropic, Google, and Meta are all racing to make their models more accurate and less prone to hallucination. Billions of dollars are being spent on alignment research, reinforcement learning from human feedback, and retrieval-augmented generation — all aimed at making AI outputs more trustworthy. And yet one of the most common user behaviors is actively undermining those efforts.
The irony is sharp. Users who care most about accuracy — professionals seeking expert-level output — are the ones most likely to employ the very technique that reduces it. The doctor asking Claude to “respond as a board-certified oncologist” is getting worse medical information than if she’d just asked the question plainly.
There’s a deeper technical question here too. Large language models generate text by predicting the most likely next token given the preceding context. When you inject an expert persona into the system prompt, you’re altering the probability distribution over all subsequent tokens. The model has seen plenty of text written by people claiming expertise — including text that’s overconfident, jargon-heavy, and sometimes wrong. By steering the model toward that distribution, you may be steering it toward the failure modes of expert communication, not just its strengths.
And the models can’t tell the difference. They don’t understand expertise. They pattern-match against training data.
Some AI practitioners have already begun adjusting their approaches in light of these findings. On X, several prominent AI researchers and engineers have noted that they’ve moved away from persona-based prompting in favor of more structured techniques — chain-of-thought prompting, few-shot examples, and explicit reasoning instructions. These methods give the model a process to follow rather than a character to play, and the evidence increasingly suggests they produce better results.
The Carnegie Mellon study isn’t the first to raise questions about persona prompting, but it’s the most comprehensive. Earlier work had shown mixed results, with some studies finding modest benefits from expert personas on narrow tasks. The new research’s breadth — spanning multiple models, thousands of prompts, and diverse domains — makes its conclusions harder to dismiss.
For the enterprise market, the implications are practical and immediate. Any organization that has embedded persona-based prompting into production AI systems should be testing whether those personas are actually improving output quality or silently degrading it. The answer, based on this research, is likely the latter.
There’s also a lesson here about the gap between intuition and evidence in AI usage. It feels right that telling a model to be an expert would make it perform better. The logic seems airtight. But large language models don’t operate on logic — they operate on statistical patterns. And our intuitions about how identity and expertise work in human communication don’t transfer cleanly to systems that have no identity, no expertise, and no understanding of either.
So what should users do instead? The research points toward a few principles. Be specific about what you want. Provide context and examples. Ask the model to show its reasoning. Don’t tell it who to be — tell it what to do. These aren’t new ideas, but they now have stronger empirical backing.
The persona paradox is, in a sense, a microcosm of the broader challenge with generative AI: the things that make outputs sound better often make them less accurate. Fluency is not fidelity. Confidence is not competence. And in a world increasingly reliant on AI-generated information, that distinction matters more than ever.
The full study is available through the Allen Institute for AI’s research portal. For anyone building, deploying, or depending on large language models in professional settings, it should be required reading. The expert you’ve been asking your AI to impersonate may be the worst advisor in the room.
WebProNews is an iEntry Publication