ChatGPT-style search represents a 10x cost increase for Google, Microsoft

Google hints that an AI chatbot search engine will really cut into its profits.

See full article...

 

[doubleyewdee]

AMEN. And that's the Real Issue. They can't give a first page of garbage!
They might have to actually do a Deep Search like it's 1999.

[I work at Microsoft, formerly Bing, now Azure Machine Learning -- speaking on my own behalf, I don't have special knowledge about anything here]

As usual I'm late to the comments party but there's a variety of interesting factors at play here. The per-interaction serving cost of LLMs is, compared to the typical cost of serving results from an inverted index lookup (grossly oversimplifying web search), astronomical. However, the state of the art in "internet scale generation + serving of a distributed information index" has had about two decades to mature. In contrast, AI model serving is relatively nascent, and internet-scale serving is effectively a toddler. Expect advances here as profit motives incentivize further R&D. Lots of people are working very hard to make this cost less.

However, if you use Bing Chat today, what it's doing is pretty obvious: generating, executing, and synthesizing results from one or more traditional search queries. So what I find really interesting, as a now outside observer of this new scenario, is that all the extant costs of having that massive knowledge index and search engine infrastructure, are being piled on top of the new costs of hosting an LLM to synthesize the data. If I'm a search engine, I'm not necessarily ecstatic about that. If I'm a search engine with 97% worldwide share, I'm certainly not excited about scaling that out immediately in response to a competitor's threat. Now I have to do all the work of being a search engine in addition to this new thing. However, I'm confident Google can figure it out. They got YouTube working for them when, at one time, the costs of owning and operating the platform seemed likely to dwarf any conceivable profit they could wring from it.

I think, also, it might be a little bit early to declare that "10x cost" is a reasonable factor. I suspect the real number will end up being lower, because the incentives are just so good to make that the case. Not to mention, it may force a pivot in fundamental assumptions about search engines, knowledge indexing, etc. As AI-generated content proliferates, it seems likely to me that our ability to sift out genuine, human-created content will need to improve. That genuine human content, particularly the quality stuff, is a tiny fraction of the garbage that ends up in search indices today. If the profit motives shift to fewer, higher-quality data sources, it seems plausible that all that godawful content farm spam that has made the traditional/current search experience so increasingly awful over the last decade, may dry up, in which case the cost of the existing service naturally drops.

There's also the real possibility that these additive AI assistant style data synthesizers will, in fact, end up too expensive or too impractical to give away in an ad-supported manner, and will end up being bundled with some sort of subscription service or pay-per-use model, further upending the ad-supported internet search-driven mechanism generating so much traffic today. I can't say, as a user, that I would be sad to see the current advertisement-based revenue model of most of the internet be severely disrupted in favor of a pay-for-use mechanism.

 

[doubleyewdee]

If they were to open source the model, we can prune it and it'd be able to run just fine, even if it's not fully featured.

This is why AI will be so disruptive to Google's business model.
We'll have models that are very capable, that can run locally from a 200MB file.

That's not how any of this works.

ETA: we're (correctly) criticizing state-of-the-art models with billions of parameters as being inadequate/unfit for task. Pruning and otherwise downsizing is the opposite direction anybody in the industry is headed. There are great applications for narrow-purpose smaller/lower parameter AI models, but "synthetic human language" doesn't fit in that wheelhouse even a little, and that is unlikely to change.

Sorry for the initial snippy response.

 

[doubleyewdee]

Entries don't get "ranked and categorized" when you search. That happens BEFORE you search and it certainly doesn't take less than a second.

That’s not entirely accurate. There is static ranking that occurs during the generation of the indices, and is effectively baked in, but there is also query time “dynamic” ranking to attempt to bubble up the best 10 results out of the order of magnitude more the search pulls up. The static (page) rank is a big part of it, but at query time, and definitely in under a second, other factors are used to attempt to pull up the best results.

A sample of factors might be:

• user location vs page “location”. I live in Mount Vernon, WA. If I search for Mount Vernon I’m probably not as interested in George Washington’s home as someone from, say, Iowa might be.
• freshness. if my query appears to be looking for news or recent data (determined, at query time, by a model) then newer pages will get a boost
• user profile: ambiguous terms may be disambiguated and influence rank based on multiple factors

Google’s query-time benchmark number is the sum total of time to retrieve the index data (from a boatload of backend services, of which the web index is the primary), collate the various data, and finally rank it all to prioritize rendering.

Also there are some ads in there.