Google: "There are bound to be some oddities and errors" in system that told people to eat rocks.
See full article...
See full article...
Google’s AI Overview is flawed by design, and a new company blog post hints at why
- Thread starter JournalBot
- Start date
Podginator
Ars Centurion
It’s unnecessary because the company already had a version of something like this that presented the information as it was presented in the original.
The new version just shits out a reworded version it, but also hallucinates a bunch of new stuff that was never intended.
The new version just shits out a reworded version it, but also hallucinates a bunch of new stuff that was never intended.
Last edited:
Upvote
188
(193
/
-5)
"There are bound to be some oddities and errors"
Well, then, perhaps don't put it at the top of your website that people use to get accurate information?
I'm also a little bemused at Google being like 'hey we did not tell pregnant women to smoke, we just told you to eat a rock a day, stop being all mad and stuff, fake news.'
Well, then, perhaps don't put it at the top of your website that people use to get accurate information?
I'm also a little bemused at Google being like 'hey we did not tell pregnant women to smoke, we just told you to eat a rock a day, stop being all mad and stuff, fake news.'
Upvote
246
(250
/
-4)
There's just not value in this. This feature cannot know which information is true or if, after it assembles sentences in a statistically likely way, it is repeating true information. Worse, it is likely to create disinformation as websites that try to create accurate information wither and die and the AIs are asking other AIs for advice.
"Here are some articles that may answer your question" is still the best way to answer the questions.
"Here are some articles that may answer your question" is still the best way to answer the questions.
Upvote
167
(169
/
-2)
- Add bookmark
- Featured
- #8
Usually, when querying Google, section 230 kicks in on the actual content making the hosting party of the content not liable. How does that work when Google generates a summary? Is it creating content or is it more like a translation from a liability point of view?
Upvote
233
(233
/
0)
Okay, so it’s not that Google’s AI that sucks, it’s Google’s Search that’s broken and the AI is just lipstick on the proverbial pig.
Weird flex, but… OK, Google.
Upvote
283
(283
/
0)
So Google literally used the "no, it's the children who are wrong" meme as a defense? Non-satirically?
Upvote
123
(126
/
-3)
When future historians write about the AI mania, "flawed by design" would be a rather pithy summary of these times.
Upvote
104
(105
/
-1)
Upvote
73
(75
/
-2)
"AI" is just repeating "My source is that I made it the fuck up!" for ever and ever. Who knew?
Upvote
54
(56
/
-2)
I've recently seen two AI summary products that are superficially the same, and yet totally different in value.
The first was an AI summary of reviews for hiking trails. In this case, it summarized the reviews and had been tweaked to understand that recency and seasonality mattered, so it could say things like "In January 2024, a reviewer posted that a downed tree was blocking the trail" or that "Reviewers report that this trail is muddy and difficult in winter." No official human was generating an authoritative summary of each trail, so this is helpful in getting the user to information that would otherwise be hard to find.
By contrast, Amazon has started adding a review summary for fiction books to its listings. In this case, there is always at least one authoritative human-written summary of the book which is more useful than the AI. If you want a human review, even reading a single highly rated review gives you better information than the AI summary did. "Customers report this book is funny and engaging" doesn't do anything - you already knew that, and the AI is just getting in your way.
The first was an AI summary of reviews for hiking trails. In this case, it summarized the reviews and had been tweaked to understand that recency and seasonality mattered, so it could say things like "In January 2024, a reviewer posted that a downed tree was blocking the trail" or that "Reviewers report that this trail is muddy and difficult in winter." No official human was generating an authoritative summary of each trail, so this is helpful in getting the user to information that would otherwise be hard to find.
By contrast, Amazon has started adding a review summary for fiction books to its listings. In this case, there is always at least one authoritative human-written summary of the book which is more useful than the AI. If you want a human review, even reading a single highly rated review gives you better information than the AI summary did. "Customers report this book is funny and engaging" doesn't do anything - you already knew that, and the AI is just getting in your way.
Upvote
109
(110
/
-1)
Upvote
44
(46
/
-2)
What a very Muskian response. Is this Reid person new to the Internet?
Just own up to your failures rather than doubling down and trying to blame users when your Artificial Idiocy program falls flat on its proverbial face.
Upvote
100
(103
/
-3)
(bold italics added for emphasis)
I think they forgot to say "until the next squirrel runs by" there.
Upvote
41
(42
/
-1)
I see the old "it's a feature, not a bug" defense is alive and well.
It's clearly working since humorous results certainly can be top results in their context. But it's the lack of context awareness that trips up Google's model. And I have absolutely zero confidence it will work differently with any other information should somebody try to game the system. And SEO is already a thing.
It's clearly working since humorous results certainly can be top results in their context. But it's the lack of context awareness that trips up Google's model. And I have absolutely zero confidence it will work differently with any other information should somebody try to game the system. And SEO is already a thing.
Upvote
57
(59
/
-2)
So, given the example in the article, why can't UC Berkely geologists, or whatever party is being libeled, sue Google?
Upvote
40
(42
/
-2)
Even worse. The LLMs are now being trained on their own garbage output being put on the web through these SEO spam sites. Thus further reinforcing the cycle of shit.
Upvote
97
(98
/
-1)
Still better that the mayonnaise pizza some people eat.
As inaccurate as the results in AI Assistant are, I guarantee that Google's focus is and will continue to be on the accuracy of the ads it delivers and the data it collects from the users. There'll be few errors there, I'm certain of that.
Upvote
42
(43
/
-1)
What evidence do you have that this isn't intentional?
It seems to be functioning exactly per Google's design if this Reid person is to be trusted.
Upvote
15
(24
/
-9)
Don't be ridiculous. No one would get a pizza half covered with rocks.
You are only supposed to eat one rock per day.
Upvote
77
(80
/
-3)
Google is the living embodiment of enshitification; that's what happens when you've achieved a de-facto monopoly on search. The only fix is real competition.
Upvote
87
(90
/
-3)
I suspect the longer-term vision is to evolve this into a voice assistant kind of product - Alexa-like - where you can just ask your phone, or watch, or car, or robot dog a question about anything, and have it speak the short and authoritative answer back at you.
Upvote
-18
(9
/
-27)
Podginator
Ars Centurion
I feel like Amazon are part of the make it work at all cost crowd that, ironically, aren’t actually trying to show this technology out in the best light.
Same with Google here. Is this making anyone think AI is the next big thing? It’s comically lying to you. It’s literally telling you to eat rocks.
They’re burning through a lot of trust fast.
Rabbit, all that advertising for an orange bricked android app that doesn’t work.
Humane, whatever the hell that is.
AI overview - spitting out complete bullshit.
Upvote
63
(63
/
0)
Podginator
Ars Centurion
Surely like those one glass of wine things - one is for health, the rest is just for me.
Upvote
24
(26
/
-2)
Upvote
28
(30
/
-2)
i love that the whole reason for an AI summary is because they know their search is FULL OF SEO DOG SHIT that they encourage.
rather than fix that problem just throw "ai" at it
rather than fix that problem just throw "ai" at it
Upvote
51
(54
/
-3)
- Add bookmark
- Featured
- #35
I think there's another reason it comes out with these hallucinations: it's not good at understanding what it's reading--indeed, it probably doesn't understand at all. But it's extremely good at writing decent English prose, which makes it look as if it's understanding.
Where does that prose come from, if it's not understanding what it's writing about? It's doing text summarization, which in this case seems to be driven by two things: (1) the words in the texts it's summarizing, and (2) the statistical(-like) models of what words typically follow what other words in English. The result is that it pulls some words out of the texts it's summarizing, and then glues them together in a way that makes good sensible English. The problem is that it misses a lot when it summarizes.
The screenshot of the video console summary illustrates this (although of course it's impossible to know whether these were the exact input that generated the output). It picked up the fact that the Atari Jaguar was released in 1993, but when it generated the summary, it didn't understand that the other two consoles came later.
Where does that prose come from, if it's not understanding what it's writing about? It's doing text summarization, which in this case seems to be driven by two things: (1) the words in the texts it's summarizing, and (2) the statistical(-like) models of what words typically follow what other words in English. The result is that it pulls some words out of the texts it's summarizing, and then glues them together in a way that makes good sensible English. The problem is that it misses a lot when it summarizes.
The screenshot of the video console summary illustrates this (although of course it's impossible to know whether these were the exact input that generated the output). It picked up the fact that the Atari Jaguar was released in 1993, but when it generated the summary, it didn't understand that the other two consoles came later.
Upvote
74
(74
/
0)
At this point, it would be cheaper and easier to build an encyclopedia staff for answers, like Amazon's mechanical turk solution to their "automated" grocery store. They're breaking things that didn't need fixing, to impress a bunch of ignorant shareholders, who are unhappy with a historically profitable business.
Upvote
45
(47
/
-2)
Very weird.
Reminds me of datastores.
There's a reason you use a search engine for search, a time series DB for the right data ... document DB and so on. Yeah you can cram the wrong use case in one or the other sometimes, but it won't work well long term.
People doing AI chats generally aren't just searching...
Reminds me of datastores.
There's a reason you use a search engine for search, a time series DB for the right data ... document DB and so on. Yeah you can cram the wrong use case in one or the other sometimes, but it won't work well long term.
People doing AI chats generally aren't just searching...
Upvote
12
(13
/
-1)
It never was. The people providing the information were a reliable source of information. Too bad the money was in spamming SEO results and plausible misinformation dissemination and not in accuracy.
Upvote
73
(74
/
-1)
The entire AI push over the last year makes a lot more sense when you realize no one really knows what AI is actually useful for, or how to make money off it (or even how it works, but that's not really all that important: most people don't know how the Internet itself works after all). All pretty much everyone knows is it's very popular, and might one day be extremely powerful and useful. So companies are throwing AI everywhere they can, because they're hoping they'll create the next Facebook or Netflix in the process. It's not actually irrational, once you realize management has absolutely no idea how and why some companies succeed, and others fail, they just know they need to be seen doing something, because doing nothing (even if that is the successfull move) looks like a failure of vision.
Upvote
89
(91
/
-2)
This dovetails nicely with yesterday's "Misinformation Superspreader" story on Ars.
Sounds like Google's AI Overview is a lot like everyone else's MiL, retweeting and forwarding everything they see online as factual information.
Without the capability of discerning content like satire, or just sometimes obvious BS, simply wrapping whatever you find on the internet with an AI Overview is just fundamentally a bad idea.
Sounds like Google's AI Overview is a lot like everyone else's MiL, retweeting and forwarding everything they see online as factual information.
Without the capability of discerning content like satire, or just sometimes obvious BS, simply wrapping whatever you find on the internet with an AI Overview is just fundamentally a bad idea.
Upvote
48
(49
/
-1)