ChatGPT-style search represents a 10x cost increase for Google, Microsoft

Google hints that an AI chatbot search engine will really cut into its profits.

See full article...

 

[NomadUK]

Really pretty much the only good news about ChatGPT I’ve heard so far.

 

[fidget42]

How much will it impact profits to use AI to reduce the amount of garbage returned in a search?

 

[Nowicki]

Doesnt matter how they push it, a GPT style search would cut into their current model for ad revenue the same way voice search does.

With a voice search you cant put up 12 promoted things in front of the result they are looking for, and generate as much data about it. With LLMs being a black box, and costly to reign in for product use its just not as profitable.

If someone else does it, google gets eclipsed potentially so they cant really avoid it.

 

[NicoTexas]

From my perspective, it would increase the time spent by users on each search engine as it becomes more interactive and you can get the answers without leaving Google or Bing.
So, yes more compute, but I would expect a higher user engagement and time spent on the platform, so potentially more revenue.

 

[olafgarten]

I've always thought this might be an issue.

I think 10 times more is a pretty conservative estimate as well, both ChatGPT and Bard are huge models and even after quantisation and other optimisations it's going to cost a lot to run.

That doesn't even factor in the cost of continually training it with new data to make sure it's information stays relevant.

 

[S-T-R]

Doesnt matter how they push it, a GPT style search would cut into their current model for ad revenue the same way voice search does.

With a voice search you cant put up 12 promoted things in front of the result they are looking for, and generate as much data about it. With LLMs being a black box, and costly to reign in for product use its just not as profitable.

If someone else does it, google gets eclipsed potentially so they cant really avoid it.

Text chats do have the advantage in that you can design them to display multiple profit-generating links at once in the result. Voice chat is limited by the medium in which it exists to present you with one result at a time.

Still, once the novelty wears off, are people really going to be that much more enthusiastic about chatting at a computer than speaking to one? I'm not sure. I think the headline grabbing tech is not going to be at the forefront of things, but relegated to the background. That is, improving things (e.g. customer service bots, code completion tools) that already exist and have narrower and more easily defined use cases. I will (again) warn that one of the best use cases for this tech as it exists today is propaganda, which requires content that is merely plausible sounding, a lower bar than all the other use cases that require results that are also true.

Having said that, the image bots seem to be very useful as a concept-phase tool as-is too. This is all going to be very complicated and will affect different industries at different rates.

 

[nathematics]

This also probably means greater energy use and more CO2 emissions.

 

[jdvorak]

This reminds of when Google decided to introduce free office software. Not because Google cared to make it a viable business, but because it wanted drive home the idea to consumers and businesses that office software should be free and in doing so hurt Microsoft’s steady cash cow with Office. The plan was that this would deprive MS of the financial resources to make inroads into Google’s domain.

Fast forward to now, Office still reigns supreme because MS pivoted to the subscription model. And it’s now MS forcing Google to spends tons of money and resources for a project that MS itself may not even be serious about: search. But it’s forcing Google to respond and distracting Google from other priorities. Gotta love corporate strategy.

 

[Dark Pumpkin]

I don't understand how monetizing it would be a problem. Can't they just devote a section of the page to ads relevant to the search/chat like they do with current search results?

I suppose I don't understand enough about marketing or I'd understand why such a simple solution wouldn't work well.

 

[Hunter13]

How much will it impact profits to use AI to reduce the amount of garbage returned in a search?

AMEN. And that's the Real Issue. They can't give a first page of garbage!
They might have to actually do a Deep Search like it's 1999.

 

[Hunter13]

From my perspective, it would increase the time spent by users on each search engine as it becomes more interactive and you can get the answers without leaving Google or Bing.
So, yes more compute, but I would expect a higher user engagement and time spent on the platform, so potentially more revenue.

But, right now ChatGPT is PURE Search.
Pure Interactive Search in search of a good answer to your question.

That's not Google Search.
That's where they're losing money. How do you inject wasteful bull* content to take advertiser money, and continue to destroy effectiveness of search?
That's Google's real problem.

 

[drachasor]

I don't understand how monetizing it would be a problem. Can't they just devote a section of the page to ads relevant to the search/chat like they do with current search results?

I suppose I don't understand enough about marketing or I'd understand why such a simple solution wouldn't work well.

So, you think maybe 20 adds per chat result is going to go over well? If it's 10 times the cost, that means 10 times the ads.

This makes it really seem like a huge waste of money, tbh.

 

[seelive]

Bing Search actually seems to do a pretty good job of embedding links to sources in its answers. If Google can do something similar with Bard, it could embed, e.g. sponsored links to its answer, i'm sure advertisers would gladly pay for that.

 

[Fred Duck]

You could imagine a future where instantly getting a good answer would result in less time on Google compared to having to dig through a list of 10 blue links. If that's true, then none of the money math on these new search engines really looks good.

I read that BingChat gives you an answer and a source link.

The hit search engine DuckDuckGo already does that for certain types of searches. eg:

director fern gully
returns "Bill Kroyer" at the top above all results.

Unfortunately, if they're really keen on pushing adverts, then they'll likely move to interstitials which you must look at before getting your results.

Or are they envisioning this scenario?

Fred > Who directed FernGully?
gchat > That is a good question, which reminds me. Did you know there are bargains to be had on smart home tech? Oh, to answer your question, Bill Moyers.
Fred > Thanks.
gchat > Anytime, meat bag.

Maybe there will be two interfaces, so you can chat with gchat or perform a traditional search.

 

[Ninja Puffin]

This also probably means greater energy use and more CO2 emissions.

The crypto analogies don't end with the excessive hype.

 

[abie]

I imagine advertising built into the chatbot's responses will be like something out of The Truman Show:

View: https://www.youtube.com/watch?v=5Zmh5hbgHFI

 

[Apple fanboy 101]

So that’s why Google has been stagnating.
I realized that Google was going nowhere before but this ChatGPT saga made me realize even more how Google hasn’t added a single innovation in a decade or more. What are those tens of thousands engineers even doing? Tweaking the algorithms and increasing ad revenue shouldn’t need that many people, no?

 

[S-T-R]

But, right now ChatGPT is PURE Search.
Pure Interactive Search in search of a good answer to your question.

That's not Google Search.
That's where they're losing money. How do you inject wasteful bull* content to take advertiser money, and continue to destroy effectiveness of search?
That's Google's real problem.

The chat bots are pure chat. They inherently drive towards recreating the conversations the model trained on. Like IRL conversations, that sometimes includes searching for facts, and sometimes (but not all the time) that results in facts that are true.

There is no reason you can't inject biases that point people towards certain products. The bots already ingest biases from their datasets. The only reason why BingBot isn't trying to sell you shit with every response is because they haven't got to building that part of the infrastructure yet. Rest assured they are getting to that and the "right now" portion of your comment will not last long. This is still the internet and selling shit is still the #1 prerogative.

 

[jdale]

Another Reuters report says that Microsoft has already met with advertisers to detail its plan of "inserting [ads] into responses generated by the Bing chatbot," but it's unclear how awkward that would be or if consumers will react when a chatbot suddenly kicks over into an ad break.

The problem with chat-interface search is going to be trusting the results. That's two-fold, one side is that some people won't trust the results, a problem which will be magnified by including ads which indicate the answer is biased. The other side is that some people will trust the results even when they shouldn't, and presenting ads as facts could easily drift over towards fraud -- especially if the ads are driven through an automated system where no human is checking their appropriateness.

 

[jdale]

They'll probably solve this the same way they solved the YouTube problem, with bespoke hardware. Neurally-designed silicon is a thing and Google can afford to spin up a fab.

That reduces but does not eliminate the extra cost. It still requires unique processing per user. And in the case of YouTube, the extra costs are handled by adding extra advertising, something which it is not clear how it will work.

 

[Sajuuk]

So that’s why Google has been stagnating.
I realized that Google was going nowhere before but this ChatGPT saga made me realize even more how Google hasn’t added a single innovation in a decade or more. What are those tens of thousands engineers even doing? Tweaking the algorithms and increasing ad revenue shouldn’t need that many people, no?

[...] Google hasn’t added a single innovation in a decade or more. What are those tens of thousands engineers even doing?

Well, pioneering the language models behind stuff like ChatGPT, for one.

 

[seelive]

Text chats do have the advantage in that you can design them to display multiple profit-generating links at once in the result. Voice chat is limited by the medium in which it exists to present you with one result at a time.

Still, once the novelty wears off, are people really going to be that much more enthusiastic about chatting at a computer than speaking to one? I'm not sure. I think the headline grabbing tech is not going to be at the forefront of things, but relegated to the background. That is, improving things (e.g. customer service bots, code completion tools) that already exist and have narrower and more easily defined use cases. I will (again) warn that one of the best use cases for this tech as it exists today is propaganda, which requires content that is merely plausible sounding, a lower bar than all the other use cases that require results that are also true.

Having said that, the image bots seem to be very useful as a concept-phase tool as-is too. This is all going to be very complicated and will affect different industries at different rates.

I think search is one of the least interesting use cases for LLM. Because the input/output via chat has higher throughput than via voice, it opens of whole new classes of use cases that weren't possible before. Like, i told chatgpt to generate 30 random names, with names, life stage, and to group them. yes, I could have done it via a script, but it was so much faster, as well as being accessible for those w/o a coding background. the next step is adding more APIs, i think.

 

[Ninja Puffin]

Oh it's too expensive google?

Then just open source the model, so people can run them locally. We'll save you a ton of money, you can thank us later ;-)

Unless Bard is very different from ChatGPT, running it on a consumer grade machine, even a high-end gaming PC does not sound practical.

According to Wikipedia, GPT-3's parameters take up 800 gigs.
https://en.wikipedia.org/wiki/GPT-3

 

[jdale]

I think search is one of the least interesting use cases for LLM. Because the input/output via chat has higher throughput than via voice, it opens of whole new classes of use cases that weren't possible before. Like, i told chatgpt to generate 30 random names, with names, life stage, and to group them. yes, I could have done it via a script, but it was so much faster, as well as being accessible for those w/o a coding background. the next step is adding more APIs, i think.

Chat isn't a very good match for search, especially because getting it right matters. But search is something that a very large number of people do. Generating a list of random names etc is a perfect use, but it's also a niche case. Are there really enough GMs and writers to make that a business case for something as expensive as the training for these systems? I doubt it.

 

[msadesign]

How much will it impact profits to use AI to reduce the amount of garbage returned in a search?

It’s hard to see how one could ever trust a AI to discern garbage from useful, accurate information.

It is already difficult enough to figure out if individual websites are reliable. Trusting an AI to aggregate website information into a trustworthy product?

 

[pusher robot]

Microsoft already has another obvious path to profitability with this, by adding capabilities to Office 365. Would companies be willing to pay a few bucks more per user per month to have access to an AI assistant that knows everything contained in your company's Sharepoint, your OneDrive, and your mailbox, and can provide you with information on demand or compose emails and letters for you with minimal input? I'm guessing that would be an easy sell for most. Forget New Bing, New Clippy could be huge.

 

[MrMalthus]

I think search is one of the least interesting use cases for LLM. Because the input/output via chat has higher throughput than via voice, it opens of whole new classes of use cases that weren't possible before. Like, i told chatgpt to generate 30 random names, with names, life stage, and to group them. yes, I could have done it via a script, but it was so much faster, as well as being accessible for those w/o a coding background. the next step is adding more APIs, i think.

Yeah, it's weird that the headline has a "Does anyone think this is a good idea?" above it, but the article never mentions the elephant in the room that is Bing's...emotional issues. You got some of the best of ChatGPT married to a search engine that sometimes lies or berates you (and after the recent neutering, it has the best of neither).

Maybe the real answer to costs is that ChatGPT-style search isn't a good idea yet and it's a lot better suited to a Clippy in Office/Docs, where you also happen to have a large number of users already paying for subscriptions.

 

[S-T-R]

Chat isn't a very good match for search, especially because getting it right matters. But search is something that a very large number of people do. Generating a list of random names etc is a perfect use, but it's also a niche case. Are there really enough GMs and writers to make that a business case for something as expensive as the training for these systems? I doubt it.

I couldn't have said this better myself. None of this stuff is cheap. Not the R&D, the training, or the ongoing server costs. That has to be recouped and the easiest way is with a broad user base. The second base way is B2B sales where high margins can justify narrower uses. Still, you have to bring a value add to the table if you want this to move out of academia.

 

[aposm]

This is a huge and very strange assumption to make right off the bat:

Is a ChatGPT-style search engine a good idea? The stock market certainly seems to think so, with it erasing $100 billion from Google's market value after the company's poor showing at its recent AI search event.

You're implying that they lost $100B of market cap purely because they executed a demo poorly - and not the much more reasonable assumption that the stock market thinks the whole thing (large language model based chat search) is a bad idea. Google has spent 20 years honing their search model, figuring out how to execute their core product well. It makes much more sense that investors want them to stick with what they're good at (even if being good at running a good business around search is independent from making the product good for the user).

 

[jwbaker]

Imagine thinking of Hennessy as "Alphabet Chairman" instead of as legendary giant of computer science.

 

[S-T-R]

This is a huge and very strange assumption to make right off the bat:

You're implying that they lost $100B of market cap purely because they executed a demo poorly - and not the much more reasonable assumption that the stock market thinks the whole thing (large language model based chat search) is a bad idea. Google has spent 20 years honing their search model, figuring out how to execute their core product well. It makes much more sense that investors want them to stick with what they're good at (even if being good at running a good business around search is independent from making the product good for the user).

I'd take this a step further and tell people to not assume any day-to-day movement in the market has a rational basis. Financial reporting tends to ascribe a reason to every market movement, but it's largely baloney.

 

[BevansDesign]

How much will it impact profits to use AI to reduce the amount of garbage returned in a search?

Exactly what I was thinking. AI has the potential to destroy SEO as we know it and show us the information that we're actually looking for, and I can see that having a very negative impact on Google's profits, especially since sites pay Google for preferential treatment.

Bard may be a major step forward for customer usability, but Bard2 will be a shameless cash-whore.

 

[Digital Dud]

I'm guessing we will see some robots.txt equivalent for bots scraping data for chat models, and everyone will horde their own data silos for training LLMs.

 

[Digital Dud]

Exactly what I was thinking. AI has the potential to destroy SEO as we know it and show us the information that we're actually looking for, and I can see that having a very negative impact on Google's profits, especially since sites pay Google for preferential treatment.

Bard may be a major step forward for customer usability, but Bard2 will be a shameless cash-whore.

Yeah until people figure out a whole new set of tricks to get higher placement inside the chat bots, like write really convincing English prose I guess.

 

[Demosthenes642]

A significant problem that's cropped up with Google's search business model is that driving ad revenue from search results actively makes search worse. Too many ads and people don't trust the search results because they can't find what they're actually after. At least you can see what's an ad and what's not more or less. With a chat AI search, integrating advertising makes the trust problem much much worse. Skimming 20 search results to jump past the ads is relatively quick versus engaging with a chat AI and then trying to discern what's an ad and what isn't. Even worse would be something like SEO targeted at chat AI.

I don't really have a solution for google here but maybe chat AI, especially if it's really really good, is their loss leader and they lean on their ad placement on target pages for revenue. Good functionality and mindshare got google into the collective consciousness in the first place, it might be time to look a bit to the past for strategy.

 

[S-T-R]

Exactly what I was thinking. AI has the potential to destroy SEO as we know it and show us the information that we're actually looking for, and I can see that having a very negative impact on Google's profits, especially since sites pay Google for preferential treatment.

Bard may be a major step forward for customer usability, but Bard2 will be a shameless cash-whore.

I think the safer assumption is that LLM's may trigger a shift in SEO. There is an inherent arms race between search tools and tools that clog results with bullshit. There are constant financial incentives for each side to improve their techniques. New tech doesn't make that go away. It just further incentivizes the other side to up their game.