The corporate world’s multi-trillion-dollar bet on generative artificial intelligence is facing a sobering reality check. In the rush to deploy chatbots and AI assistants, a landmark field study from researchers at Stanford University and the Massachusetts Institute of Technology (MIT) reveals a critical trade-off that executives are only beginning to grapple with: a surge in speed often comes at the direct expense of quality and customer satisfaction.

The study, which embedded a generative AI tool into the daily workflow of over 5,000 customer support agents at a Fortune 500 software firm, provides one of the first causal, real-world analyses of AI’s impact on professional work. While the technology delivered a notable 14% average increase in issue resolution per hour, the underlying details paint a far more complex picture for businesses eager to capture productivity gains. The findings suggest that the current generation of AI is less of a silver bullet for corporate efficiency and more of a complex tool that can amplify existing flaws just as easily as it can accelerate solutions.

The research, detailed in a working paper from the National Bureau of Economic Research titled “Generative AI at Work,” found the productivity gains were overwhelmingly concentrated among novice and lower-skilled workers, who saw their performance jump by as much as 35%. The AI acted as a powerful training aid, helping them absorb the tacit knowledge of more experienced colleagues. However, the technology had little to no positive impact on the company’s most experienced and skilled agents, pointing to a potential ceiling on AI’s ability to augment high-level expertise.

The Double-Edged Sword of AI-Assisted Productivity

Beneath the headline productivity figure lies a more troubling trend. The study meticulously tracked customer satisfaction scores and found a subtle but significant decline in quality associated with AI-assisted interactions. The AI-generated responses, while faster, were often perceived as less empathetic and failed to adequately address the nuances of more complex customer problems. Agents using the tool were more likely to use a standardized, robotic tone that, while technically correct, did not resolve the underlying frustration or build customer rapport. This highlights a fundamental challenge for companies, particularly those in service-oriented industries where customer experience is a key differentiator.

This quality-quantity trade-off is becoming a central theme in the enterprise AI discussion. While the technology excels at summarizing information and drafting routine communications, it struggles with tasks requiring deep empathy, creative problem-solving, or an understanding of intricate, unstated context. As reported by Digital Trends, this leads to a phenomenon where AI helps clear the backlog of simple tasks, freeing up humans for more complex work, but it may also be ill-equipped to provide meaningful assistance on those very same complex issues.

Furthermore, the study raises questions about long-term skill development. By providing an instant crutch for less-experienced employees, the AI may inhibit their ability to learn the deeper, problem-solving skills that define true expertise. The tool effectively helped novices mimic the patterns of experts, but it did not necessarily teach them the reasoning behind those patterns. For organizations focused on building a sustainable talent pipeline, this suggests that over-reliance on AI could inadvertently hollow out the mid-level skills essential for future leadership and innovation.

Echo Chambers of Inefficiency: How AI Can Amplify Human Flaws

Perhaps the most cautionary finding from the Stanford and MIT research is how the AI model learned and propagated the behaviors of the human agents it was designed to assist. The system absorbed the conversational data from thousands of interactions, effectively creating a model based on the company’s collective habits. This included not only best practices but also suboptimal strategies, flawed logic, and even brusque communication styles that top-performing agents avoided. The AI then presented these less-than-ideal approaches as standard suggestions to all users, particularly novices who lacked the experience to question them.

This creates a significant corporate risk: deploying a generative AI tool trained on internal data could inadvertently scale a company’s own worst habits across the entire organization. An inefficient workflow, a poor customer service script, or a biased decision-making process, once encoded into the AI, can become systematized and entrenched, making it far more difficult to identify and correct. The AI, in this scenario, becomes an echo chamber for mediocrity or outright error, cloaked in the authority of advanced technology.

This risk is not merely theoretical. A report from MIT Sloan Management Review warns that AI systems, when used for coaching or performance management, can perpetuate existing biases and fail to understand the individual context of an employee’s performance. Without careful human oversight and a commitment to curating high-quality training data, companies risk automating their own organizational blind spots on a massive scale. The focus must shift from simply deploying AI to actively managing it as a system that requires constant vetting, refinement, and alignment with strategic goals.

Navigating the ‘Jagged Frontier’ in the Modern Enterprise

The researchers describe the current state of AI capabilities as a “jagged frontier”—a landscape of sharp peaks of superhuman performance interspersed with deep valleys of surprising incompetence. An AI model can write flawless code for one task and then fail spectacularly at a simple logical reasoning problem that a child could solve. This uneven capability profile presents a profound strategic challenge for business leaders. The decision is no longer a simple “yes” or “no” to AI, but a highly granular analysis of which specific tasks within a workflow are suitable for automation or augmentation.

For example, in the customer service environment studied, the AI was highly effective at quickly finding and summarizing relevant knowledge-base articles. However, it was completely ineffective at sensing a customer’s growing frustration from their word choice and adjusting its tone accordingly—a skill at which experienced human agents excel. A manager who fails to recognize this distinction might implement the AI too broadly, leading to an increase in escalated complaints and customer churn, even as their agents’ “productivity” metrics appear to improve.

Successfully navigating this jagged frontier requires a new level of managerial sophistication. According to analysis from firms like McKinsey & Company, capturing the value of generative AI depends on reimagining entire business processes rather than simply plugging the technology into existing ones. This involves identifying the specific points where AI can serve as a “thought partner” to a human, handling the rote aspects of a task while leaving the critical thinking, strategic decision-making, and interpersonal elements in human hands. It is a delicate process of integration, not replacement.

Recalibrating the Corporate AI Strategy

The findings from the NBER working paper serve as a critical counter-narrative to the unbridled hype surrounding generative AI. The path to a genuine return on investment is not through broad, indiscriminate deployment, but through targeted, strategic implementation that acknowledges the technology’s current and profound limitations. The study underscores that human oversight is not a temporary measure but a permanent, essential component of a successful AI-integrated workplace.

The study’s authors propose a more nuanced role for AI in the enterprise: as a sophisticated training tool to onboard new employees, as an assistant that handles data retrieval and summarization, or as a brainstorming partner for creative tasks. The key is to keep humans firmly in the loop, empowered to override, correct, and improve upon the AI’s suggestions. This approach leverages the AI’s strengths in speed and data processing while mitigating its weaknesses in quality, reasoning, and emotional intelligence.

Ultimately, the race to harness artificial intelligence will not be won by the companies that adopt it the fastest, but by those that adopt it the wisest. The evidence suggests that true, sustainable productivity gains will come from thoughtfully pairing machine efficiency with irreplaceable human judgment. For the C-suite, the central task is now to look beyond the promise of automation and build an organizational culture that knows not just how to use these powerful new tools, but also when not to.