Two Randomized Studies Find Google’s AI Search Cuts Clicks Without Pleasing Users

For two years, the fight over what Google’s AI search features do to publisher traffic ran mostly on before-and-after charts that Google could wave away. Two randomized experiments that surfaced this summer are harder to dismiss. Both found that the AI features send fewer people to websites, and neither found that searchers liked Google any better for it.

Hiding AI Overviews brought the clicks back

The first, by Saharsh Agarwal of the Indian School of Business and Ananya Sen of Carnegie Mellon University’s Heinz College, used a browser extension to randomly hide AI Overviews from some desktop Chrome users while others saw Google as usual. Across 1,065 U.S. participants, an overview cut outbound organic clicks by 39.8% and raised zero-click searches by 34.5% on the queries where it appeared, PPC Land reported. Removing it lifted organic clicks from 0.37 to 0.62 per search.

Those extra clicks weren’t low-value ones. Bounce rates, back-button returns and time on page didn’t differ between the groups, and an end-of-study survey found no difference in satisfaction, perceived information quality or ease of finding information. A smaller arm that sent users into Google’s conversational AI Mode was treated as exploratory because many participants dropped out, but those who stayed rated their experience 1.14 points lower and clicked out 0.36 times per search, against 0.53 in the control group.

A second study, the same direction

Researchers from the University of Pennsylvania and Northeastern University reached a similar result in a draft paper released in August, titled “AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence.” They tracked the searches of 1,100 volunteers over seven days, randomly assigning them to regular Google with AI features stripped out, Google as normal, or searches forced into AI Mode, Search Engine Journal reported.

AI Mode kept people inside Google longer, sent them to websites less often and made them less satisfied with Google Search, and some of those pushed into it tried other search engines. Removing AI Overviews increased clicks to websites with no detectable drop in how users rated their search experience. The study had its own hiccup: a change to Google’s AI Overview code midway through broke the extension’s hiding logic, so only 51.1% of overviews in that group were actually hidden.

Google’s position

Google executives have said users like AI Mode, and CEO Sundar Pichai has said AI Mode is sending more people to the open web than ever. Liz Reid, Google’s vice president of product for Search, said last year that clicks from AI Overviews are higher quality, with more purchase intent. The Agarwal and Sen study measured exactly that and found the extra clicks from hidden overviews were just as engaged.

They also fit earlier observational work. Pew Research Center found that people clicked a traditional result on 8% of visits when an AI summary appeared, compared with 15% without one, and clicked a link inside the summary just 1% of the time, Ars Technica reported last year. Ahrefs, which in early 2025 put the click loss for top-ranking pages at 34.5%, raised that estimate to 58% in February.