The First Randomized Test of Google AI Mode: Fewer Clicks, Lower Trust, More Bing

Short answer: In the first randomized field experiment on Google's AI features, people forced into AI Mode for a week clicked through to websites 18.8 percentage points less often, ran fewer searches, and trusted Google's results less. Turning AI Overviews off raised clicks by 8.8 points among the users for whom it worked. Across everyone assigned, that effect was small and not statistically significant.

Publishers have argued for two years that Google's AI answers take their traffic. Google has argued the opposite. Until now, both sides relied on observational data: traffic charts, panel studies and Google's own aggregate figures. None of these can separate cause from coincidence.

A preregistered experiment from researchers at the University of Pennsylvania and Northeastern University, published on arXiv in August 2026, can. It randomly assigned real Google users to three versions of Search and measured what they did. The headline result is clear. The fine print says just as much, about how big the effect really is and about what users actually want.

Key takeaways

• AI Mode cut click-through to websites by 18.8 percentage points versus normal Google. If a whole AI Mode conversation counts as one search instead of each prompt, the drop is still 14.9 points.
• AI Mode users searched Google less, not more: 0.92 fewer search sessions per day, against a baseline of 4.0. That is the opposite of what the researchers predicted from Google's own claims.
• Clicks fell hardest for Reddit (−21.2 pts), news sites (−12.5 pts) and Wikipedia (−9.9 pts). These figures are the share of users who clicked through to each type of site.
• Use of competing search engines rose 11.2 points among AI Mode users, and their stated intention to switch to Bing rose 1.25 points on a 7-point scale.
• Hiding AI Overviews was the weaker result. It raised clicks by 8.8 points among users where the hiding actually worked. Measured across everyone assigned, the lift was 2.7 points and not statistically significant.

What did the experiment actually test?

The study recruited US adults who used Chrome and Google Search as their defaults, in waves between March 17 and 19, 2026. A browser extension logged every Google search, every click out of Google, and whether an AI Overview appeared. After a three-day baseline, each participant spent seven days in one of three conditions:

Of 1,444 people enrolled, 1,100 made at least one search during the treatment week, and 956 completed the final survey. At baseline, participants averaged 9.2 searches, 4.0 search sessions and 4.1 outbound clicks per day. AI Overviews appeared on 36% of their searches. They used AI Mode for only 0.6% of them.

That last number matters, and we come back to it below.

How much did AI Mode reduce clicks to websites?

The paper defines click-through rate as external clicks from Google divided by Google searches, calculated per user. By that measure, AI Mode reduced it by 18.8 percentage points (95% confidence interval: −22.2 to −15.3).

There is a legitimate question about how to count. In AI Mode, each follow-up prompt counts as a new search, so a five-turn conversation that ends in one click looks worse than a single query with one click. The authors tested that as a preregistered robustness check: when a whole conversation counts as one search, the reduction is 14.9 points. Smaller, but still large and still highly significant.

The losses were not spread evenly across the web:

*The ad figure mostly reflects the fact that AI Mode showed no ads during the study. It says little about how AI Mode ads would perform once they exist.

The Reddit number is worth pausing on. Reddit has become one of the most-cited domains in AI answers, and a common assumption is that AI search funnels traffic toward it. When the answer is written for the user, though, even Reddit loses the click.

Does removing AI Overviews bring the clicks back?

Partly, and less conclusively than the headlines suggest.

Hiding AI Overviews depended on the extension recognizing Google's page markup. The paper reports that about 90% of AI Overviews were hidden on the first day of the treatment. After a Google interface change, the success rate fell to zero, and overall only 51.1% of AI Overviews were hidden.

Because of that, the researchers report two estimates:

The 8.8-point figure is the one being quoted, and it is a fair estimate of what removing AI Overviews does when it actually happens. But it rests on a smaller, self-selected slice of the sample. The data support saying that AI Overviews very likely suppress clicks. They do not support a precise estimate of how much.

For comparison, a 2025 Pew Research Center analysis of 900 US adults' browsing found users clicked a traditional result on 8% of visits with an AI summary, versus 15% without one. They clicked a link inside the summary on just 1% of visits. Pew's data is observational: searches that trigger AI summaries may differ from searches that don't. The new experiment is the causal complement, and it points the same way.

One more data point from the two studies: AI summaries appeared on about 18% of searches in Pew's March 2025 data, and AI Overviews appeared on 36% of searches in this study's March 2026 baseline. The two panels are different, so this is not a clean trend line. But it is consistent with AI answers covering far more of search than they did a year earlier.

Did users prefer AI Mode?

No. On every perception measure the study took, AI Mode users came out worse than those on normal Google:

The relevance result ran directly against the preregistered hypothesis, which predicted that AI Mode would feel more personalized, based on Google's own claims about AI personalization.

The open-ended feedback shows why. In participants' own descriptions, the recurring complaints were losing control (17.6% of responses), difficulty getting to a specific website (15.3%), and too few links or sources (13.4%). One participant said that when they typed in a site they wanted to visit, AI Mode told them about the website instead of taking them there.

That is a problem for brands, not just publishers. Navigational searches ("brand name + login", "brand name pricing") are a company's most valuable traffic. An interface that summarizes the destination instead of linking to it takes that traffic too.

Where does this leave Google's claims?

In August 2025, Google said on its official Search blog that "total organic click volume from Google Search to websites has been relatively stable year-over-year". It also said it was sending "slightly more quality clicks", defined as clicks where users don't quickly return to the results page. Google acknowledged that traffic was shifting between sites, with some losing and some gaining.

The experiment doesn't directly contradict the aggregate claim. It measured a forced, full-AI-Mode scenario over seven days, not Google's real traffic mix. It did, however, test the mechanism behind the claim. The idea that AI features make people search more, generating more chances to be clicked, was the basis for the researchers' hypothesis that AI Mode would increase search sessions. It did the opposite.

Heavy users drove this: people who averaged more than four Google sessions a day cut their sessions by about 2.0 per day under AI Mode. These are Google's most engaged users. They also looked elsewhere: an 11.2-point rise in the share of users searching Bing, DuckDuckGo or Yahoo.

What should you not read into this study?

Here are the caveats, all stated by the authors.

• AI Mode was forced on users. Participants used it for only 0.6% of searches before the study. Being pushed into it for every query is a full-adoption shock, not the gradual, opt-in uptake most people will actually go through. Some of the dislike may be resistance to change.
• Seven days is short. Users may adapt, and the effects could fade, or new ones could appear.
• The sample skews young, educated and left-leaning: 75% under 45, 87% with at least some college, 58% Democrat versus 20% Republican. Gender and race roughly matched the US Census.
• Not everyone disliked it. In the open-ended feedback, 33.6% of AI Mode users were negative, but 29.3% were positive and 13.7% mixed. A substantial minority liked the product.
• News preference was a wash. When participants were shown AI Mode and regular results side by side for news queries, the study found no significant effect on which they preferred or how much they trusted AI Mode for news. That result is about stated preference; the clicks to news sites did fall, by 12.5 points.

None of this undermines the core click result, which is large, preregistered and robust across definitions. It does mean "AI Mode is a poor user experience" is best read as "AI Mode, imposed abruptly, was rated worse by most users in its first week".

What should site owners do about it?

The practical conclusion is not to wait for Google to retreat. A 15–19 point drop in click-through under AI Mode means that as AI Mode's share of searches grows, a page that ranks the same will be clicked less. The response is to compete on the part of the result that still reaches users: being named and cited inside the AI answer.

1. Make sure AI systems can read you at all. If robots.txt or a CDN rule blocks AI crawlers, you are absent from the answers that are replacing clicks. Check with our free AI crawler checker.
2. Track citations and mentions, not just sessions. When click-through falls while visibility holds, analytics only shows the loss. Our AI search KPI methodology explains how to measure what the click no longer captures.
3. Protect navigational demand. Participants struggled to reach specific websites through AI Mode. Clear entity signals (consistent brand naming, Organization schema, an obvious homepage and product URLs) make it more likely an AI answer links to you rather than describes you.
4. Write sections that can be quoted. AI answers extract passages, not pages. Direct answers under question-shaped headings, with sourced numbers, are what gets cited. Our guide on how to get cited by AI covers this in detail.

Frequently asked questions

How much does Google AI Mode reduce clicks to websites?

In a randomized experiment with 1,100 US Google users, AI Mode reduced click-through to external websites by 18.8 percentage points compared with normal Google Search. Counting a whole AI Mode conversation as one search instead of each prompt, the reduction was 14.9 points. Both results were statistically significant.

Does turning off AI Overviews increase website traffic?

Probably, but the evidence is weaker. Among users whose AI Overviews were actually hidden, click-through rose 8.8 percentage points. Hiding them only worked about half the time after a Google interface change, and across everyone assigned to that group the increase was 2.7 points and not statistically significant.

Did users like Google AI Mode?

On average, no. AI Mode lowered trust in information on Google and reduced satisfaction, usefulness, sense of agency and perceived relevance. Still, 29.3% of AI Mode users described it positively, compared with 33.6% who were negative, so opinion was divided rather than uniformly hostile.

Did AI Mode make people search more?

No. AI Mode users had 0.92 fewer Google search sessions per day than the control group, against a baseline of about 4.0. Heavy users cut about 2.0 sessions a day. The share using a competing search engine rose by 11.2 percentage points.

How is this study different from earlier research on AI Overviews?

Earlier work, including Pew Research Center's 2025 analysis, was observational: it compared searches that happened to show AI summaries with searches that didn't. This study randomly assigned users to different versions of Google, so it can attribute the differences in clicks and perceptions to the AI features themselves.

What are the study's main limitations?

AI Mode was forced on users who had previously used it for only 0.6% of searches. The treatment lasted seven days. The sample skewed younger, more educated and more left-leaning than the US population. The effects of gradual, voluntary AI Mode adoption over months may differ.

Measure what AI search is doing to your traffic

Falling click-through does not always mean falling visibility. It often means visibility has moved to a place your analytics can't see. Start with the free AI crawler checker to confirm AI systems can reach your pages, then see why clicks are falling while brand demand isn't for how to read the gap.