Research shows Google AI Overviews reduce website clicks by almost half

I have a couple of thoughts here.

Thing the first... A couple of weeks ago I was having a problem with 1Password so I searched and Google's summary AI responded with this:

The issue of 1Password not working with specific IP addresses, particularly during lunch, suggests a potential problem with how the application is handling or accessing network resources...

The summary then went on with several paragraphs of either obviously wrong or useless text.

Then last week (I think?) I was looking for something I could link to to provide more in-depth information for a post I was making here on a subject that I am quite knowledgeable about. The AI summary was just objectively wrong nonsense.

Thing the second... Ignoring the , I think serious, link clickthrough problem, "regular people" are relying on AI answers and they are not turning AI results off. Readers of this forum may be, but lots and lots or ordinary people just "ask ChatGPT" (by which they mean any LLM, not necessarily actual ChatGPT) then just stop there. I am hearing it more and more from people in real life. Just two weeks ago I overheard someone where I was getting my hair cut talking about some objectively, factually wrong MAGA nonsense and they said "so I asked ChatGPT and it said I was right". And I've heard this kind of thing in lines at the grocery store and other places. This is happening. A lot.

Thing the third... the link clickthrough problem (in that link clicking is not happening).

Putting these things together, we seem to be headed full speed into a world where people are searching, sometimes for things that actually matter, reading LLM nonsense, moving on, and then using that nonsense to support whatever they want. In the real world.
 
Last edited:
Upvote
49 (50 / -1)

Wheels Of Confusion

Ars Legatus Legionis
76,062
Subscriptor
Putting these things together, we seem to be headed full speed into a world where people are searching, sometimes for things that actually matter, reading LLM nonsense, moving on, and then using that nonsense to support whatever they want. In the real world.
"But Gemini said it's got what plants crave! It's got electrolytes!"
 
Upvote
31 (31 / 0)

jesse1

Ars Scholae Palatinae
974
This might be a stupid question - but how are Google monetising this right now? If I get my answer from an overview and don’t click an ad (or visit a site, where I might see an ad powered by google) - they get nothing right? It doesn’t seem that their current activity is directly related to their profits (it might even cost them money). I’m pretty certain it won’t be long before those summaries integrate ads - but it’s weird to see them disrupt their own business so aggressively (might as well do it themselves before OpenAI do it maybe?)
this is the trillion dollar question for Google and OpenAI
 
Upvote
16 (16 / 0)
I've really noticed this lately. There seems to be a significant number of AI generated sites with the same installation guides. I was looking for steps on how to install a piece of software, and there are half a dozen sites with almost identical directions on how to install the software, and all of those are wrong because the command points to something that doesn't exist. The bigger problem is you could look a the article date and know it was for an older version and didn't work, now the article was published 1 month ago and yet the directions would have never worked at the time of publishing them.
Chiming in to say the same; the first set of results on any "how-to" seems to now be populated by slop. Example: I was trying to figure out how to load the Good Old Games launcher onto my Steam Deck. Literally none of the written guides I could find were of any help, and they all had that same-y overly verbose AI flavour. (In the end the saviour was a YT video by, like, some guy who figured it out.)

As someone said up-thread, it's just a matter of time before someone tries a task more serious than "install video games" and is hurt by trusting AI instructions.
 
Upvote
14 (15 / -1)
This is evaluating a fairly useless intermediate result. What Pew should be studying is whether the search users got their answer, got the wrong answer, or gave up. At least from Google's perspective their mission has long been to help the user achieve their goal, not specifically to direct them to sites.

As an example one of my searches from today is "Is Orinda incorporated" which has a conclusive AI summary answer, and has snippets for the AI summary that support the answer. Then in the search results there is some autogenerated spam from a VC-funded real estate marketing site, which I categorically do not want to visit. So I consider this a success.

SEO spammers do not own facts.
 
Upvote
-8 (7 / -15)
This is evaluating a fairly useless intermediate result. What Pew should be studying is whether the search users got their answer, got the wrong answer, or gave up. At least from Google's perspective their mission has long been to help the user achieve their goal, not specifically to direct them to sites.

As an example one of my searches from today is "Is Orinda incorporated" which has a conclusive AI summary answer, and has snippets for the AI summary that support the answer. Then in the search results there is some autogenerated spam from a VC-funded real estate marketing site, which I categorically do not want to visit. So I consider this a success.

SEO spammers do not own facts.

So the site you are posting this on no doubt relies to some extent on click-through from search engines for revenue. I mean, it's how I originally came across the site. If people aren't doing enough of that and Ars has to cut back, where are you going to make your posts in support of not clicking through?
 
Upvote
19 (19 / 0)

FinallyAnAccount

Ars Tribunus Militum
1,568
Subscriptor
Begrudgingly, a counter-anecdote.

The other morning I woke up with a cramp in a part of my leg that I hadn't experienced before. I googled on my phone looking for the combination of flex/extension that would give me relief.

The search results were awful... the keywords were there, but not in a way that described my question "what position do i need to use to relieve a cramp in the back of my upper calf (might have been lower thigh)". Search results were more like 'get more potassium and magnesium".

In desperation, I hit the downward chevron on the AI results and just tried the first suggestion (straighten knee, flex ankle, essentially) and got blessed relief from the pain. In kind of low stakes things where you just want a quick and dirty answer and don't care too much if it's wrong (I did try random things before googling, just eased off when it made the cramp worse).

I would still rather relevant search results, though! If google can take my question and answer it, why can't it display the websites that actually have those answers at the top of the search results? That might be an actual good use of AI in search!
 
Upvote
9 (11 / -2)
Honestly don't know if they're thinking that far. It feels like a mix of FOMO and desperately trying to justify the billions upon billions poured into it. Gesticulating in panic in hope that something sticks.
Remember Virtual Reality headsets and Oculus Rift?

Yeah, me neither.

Remember NFTs?

If you don't, that should be telling.

Consider that younger individuals are getting off social media and trending away from advertising every stupid thought about their personal lives even if they're permanently online.

If it didn't have a profitable use case out of the box to justify sunken costs it's likely it will be abandoned by the time or right after it's use hits critical mass, with no good way to monetize it for high quality procedural generation.

Just like VR.
 
Upvote
1 (6 / -5)

EBone

Ars Tribunus Militum
1,541
Subscriptor
Google reached its zenith by actually becoming a verb for searching for something. And they made massive revenue by monetizing the clicks in the search results.

My prediction: this A.I. search result slop is going to kill Google as a search engine. Someone else is going to build a better mousetrap for searching that doesn't rely on A.I. And "googling" something is going to go into the lexicon dustbin the way "tweet" is going.
 
Upvote
1 (5 / -4)
So the site you are posting this on no doubt relies to some extent on click-through from search engines for revenue. I mean, it's how I originally came across the site. If people aren't doing enough of that and Ars has to cut back, where are you going to make your posts in support of not clicking through?
Are you able to think of an example search that puts Ars in the top spot in the results, and generates an AI summary?
 
Upvote
0 (2 / -2)

GKH

Ars Scholae Palatinae
1,159
Multiple times in the past week I've scrolled past Google's AI summaries, clicked one of the first few links, and realized the "AI" answer was almost word-for-word the text of the linked website. Google's web search is already often completely useless, and on their current trajectory it won't be long before it's just an ouroboros of SEO'd AI slop.

But at least Google will make a ton of money off the backs of content producers with their stochastic bullshit and plagiarism machine as we skip deeper into the information apocalypse.
 
Upvote
11 (11 / 0)

Waco

Ars Tribunus Militum
2,256
Subscriptor
Thing the second... Ignoring the , I think serious, link clickthrough problem, "regular people" are relying on AI answers and they are not turning AI results off. Readers of this forum may be, but lots and lots or ordinary people just "ask ChatGPT" (by which they mean any LLM, not necessarily actual ChatGPT) then just stop there. I am hearing it more and more from people in real life. Just two weeks ago I overheard someone where I was getting my hair cut talking about some objectively, factually wrong MAGA nonsense and they said "so I asked ChatGPT and it said I was right". And I've heard this kind of thing in lines at the grocery store and other places. This is happening. A lot.
This is the real issue.

I've had conversations with FAR too many otherwise smart people that just...trust the slop that comes out of an AI summary / LLM of choice. Your average person using an LLM doesn't distrust it like they should and treat is as "the computer knows" far too often. :(
 
Upvote
22 (22 / 0)

JoHBE

Ars Praefectus
4,425
Subscriptor++
The worst part of this, imo, is that the "search overview" ai is ridiculously bad and people trust it because they've grown to trust google. Like, all ai is prone to errors, misinformation, etc, etc... but whatever version of Gemini super-small flash fast lobotomized minimized 2.5 they're using is legitimately unsuitable for this purpose and driving even worse engagement with misinformation online, further eroding our social mental health. It's blatant to the point it should be genuinely illegal or at least lead to legal consequences for Google when they eventually deliver misinformation that gets someone killed or maimed.
Trump loves it. The day they move to "curate" hallucinations, his administration will be there to claim bias and Freeze Speech violations and corrupted town square.
 
Upvote
3 (4 / -1)

Steve austin

Ars Scholae Palatinae
1,796
Subscriptor
Ignoring whether the summaries are good or bad, or whether they hurt the underlying sites, clearly Google’s intent is to provide a summary that answers the searchers’s question, which would directly replace the need to access the underlying information sources. And Google’s revenue is mostly driven by click-through ads - display ads are a small piece. So regardless of whether this is currently impacting Google’s income significantly, I don’t see the endgame - reducing click-through by making it redundant seems like it will eventually have a major (negative) revenue impact. How will they even in theory replace the click-through revenue? Or is this a “Phase 1 - collect underpants, Phase 2 - ?, Phase 3 - profit” situation, where they feel a need to do this to fight OpenAI and Microsoft, without an idea how to make money on it?
 
Upvote
8 (8 / 0)
Ignoring whether the summaries are good or bad, or whether they hurt the underlying sites, clearly Google’s intent is to provide a summary that answers the searchers’s question, which would directly replace the need to access the underlying information sources. And Google’s revenue is mostly driven by click-through ads - display ads are a small piece. So regardless of whether this is currently impacting Google’s income significantly, I don’t see the endgame - reducing click-through by making it redundant seems like it will eventually have a major (negative) revenue impact. How will they even in theory replace the click-through revenue? Or is this a “Phase 1 - collect underpants, Phase 2 - ?, Phase 3 - profit” situation, where they feel a need to do this to fight OpenAI and Microsoft, without an idea how to make money on it?
This has always, as far as I know, been Google's intent. Whether it amounts to a business strategy, I don't know. But when I worked there Larry and Sergey would say at TGIF that the whole point of everything was to help the user and it wasn't relevant whether that superficially seemed good or bad for revenue. The revenue would just materialize, if Google is useful, according to their thinking back then.
 
Upvote
3 (3 / 0)
Individual websites were already getting individually kneecapped by SEO bullshit, now this? Disgusting, and LLM "search results" aren't even reliably accurate.
And a single scrape is all Google or Bing needs. Once they have your website's data, they don't ever need to access it again because they can feed that scraped data into their LLMs. They throw you a bone through citation links but face it, who clicks those?

Public good, private profits.
 
Upvote
5 (5 / 0)
sorry, maybe I dont get it.

Google is making these AI overview that get people reading its search page and quitting as they dont need anything more.

Google gets its money from being the middleman between advertisers and web page owners and using its own search page to deliver people to said web page owners (which are buying advertising space and tech from google) so they can get more advertising money.

isnt making so people wont get to those web pages self defeating for google?
 
Upvote
1 (1 / 0)
Post content hidden for low score. Show…

silverboy

Ars Tribunus Militum
2,117
Subscriptor++
It always remind me of Wall-e. Everything consolidates into ever larger companies that permeate every aspect of human lives killing cultural diversity and making thought monolithic while utterly destroying the planet and making it uninhabitable. Sounds familiar?
Everything from Big Tech in the last 15 years has been hastening either Wall-E or Terminator. So you're right, but it's even worse than that.

What a fucked-up world we live in now.
 
Upvote
5 (5 / 0)

sigmasirrus

Ars Scholae Palatinae
1,267
Because, IMO, classic search is dead. It has become nearly impossible for a tool to differentiate between a quality site, and one that has been SEO'd effectively.
"Google search is garbage." was a common refrain for a number of years prior to modern AI. They probably see this as a lifeline.

Surely they could solve it with less expenditure than it takes to run all those stupid GPUs. They could even use machine learning in a good way to help score a site as low effort SEO farm. But no, let’s make the whole situation worse, even for Google!
 
Upvote
2 (2 / 0)

jdale

Ars Legatus Legionis
18,415
Subscriptor
PSA: DDG is just a white-label Bing.
Anonymized white-label Bing, which is a significant improvement.


That said, to be honest I find DDG intolerable because it returns many, many links that don't contain the things I searched for. The words simply aren't there. Google does that too, but it at least labels which links don't contain the search terms.
 
Upvote
7 (7 / 0)

karadoc

Ars Scholae Palatinae
947
The problem is exacerbated by the fact that many of the site listed in the search results are now AI generated slop.

It use to be that you could type in a search, and get a section of relevant genuine websites. But now the search gives an AI guess answer followed by a bunch of websites with tantalisingly relevant summaries - but which turn out to be just a long-form AI slop, no more helpful than the original answer on the search page. ... So yeah, it makes sense that people don't click through on those kind of search as often.
 
Upvote
8 (8 / 0)

edrowland

Ars Scholae Palatinae
646
The worst part of this, imo, is that the "search overview" ai is ridiculously bad and people trust it because they've grown to trust google. Like, all ai is prone to errors, misinformation, etc, etc... but whatever version of Gemini super-small flash fast lobotomized minimized 2.5 they're using is legitimately unsuitable for this purpose and driving even worse engagement with misinformation online, further eroding our social mental health. It's blatant to the point it should be genuinely illegal or at least lead to legal consequences for Google when they eventually deliver misinformation that gets someone killed or maimed.
I think you have it backward. People have not yet learned to distrust AI search as much as they distrust Google non-AI search. Like seriously! Who trusts the first link on a Google non-AI Search page?

And it's kind of silly to say that it isn't fit for purpose, when it clearly is eminently more fit for purpose than non-AI search.
 
Upvote
-1 (0 / -1)