Google's AI infrastructure chief tells staff it needs thousandfold capacity increase in 5 years.
See full article...
See full article...
Google tells employees it must double capacity every 6 months to meet AI demand
- Thread starter JournalBot
- Start date
Missing Minute
Wise, Aged Ars Veteran
You do understand my confusion seeing as it has next to nothing to do with your example, right?
Can the conclusions about the costs of pre-training be generalized to other AI costs, namely inference?
Upvote
12
(13
/
-1)
Missing Minute
Wise, Aged Ars Veteran
That's fair, though it can be a bit of a self fulfilling prophecy. It's also a lot easier to reason with someone irl than it is online. Finally, it takes a lot of skill, kindness, patience, understanding, and respect.
Last edited:
Upvote
1
(2
/
-1)
There are maybe 13-16 billion searches conducted on Google per day. Just think how much less compute they would need if you had to deliberately click an AI button after each search instead of running it automatically.
For comparison, there are about 2 billion ChatGPT queries per day. So Google is running 8 times that just for searches, where some significant proportion of users aren't even interested in getting their AI summaries at all.
Upvote
24
(24
/
0)
Upvote
4
(4
/
0)
Upvote
10
(10
/
0)
On the other hand, a LLM query requires vastly more compute and dedicated hardware than a search query.
Upvote
7
(8
/
-1)
Yeah, that's where a huge breakthrough in quantum computing and its applications to neural network training/inference would come in handy.
Because, as we all know, that's exactly how discoveries work: you need them to sustain your investments, so they pop up right on schedule, right?
Upvote
19
(19
/
0)
Interesting points, thanks. This makes me less skeptical than when I started reading this article. Are you sure about the hardware efficiency, though? This sounds strongly like Moore's law, which stopped describing reality a while ago.
Regardless, GPU efficiency improvements and algorithm improvements are not the only requirements you need to increase capacity. You also need more RAM and HD capacity, for instance, and more bandwidth, both within the machine and between data centers. To the best of my understanding, there hasn't been any great improvement in any of these domains in the last 10 years. In particular, bandwidth needs tend to scale more than linearly, as things get more parallelized.
Let's place all of this in a worldwide situation that threatens supply chains and let's not forget that Google is not the only company attempting to achieve this, so there will be competition for upstream components.
I therefore remain rather skeptical.
Upvote
12
(16
/
-4)
CEOs who tell all c-levels to introduce AI into the business for "I don't know, something!" (true story)
Upvote
18
(19
/
-1)
Upvote
3
(3
/
0)
We can just throw an AI at the problem. That solves all current problems? Right? I totally asked chatgpt if we could violate the laws of nature and it said it was possible, so it must be true.
Last edited:
Upvote
4
(4
/
0)
ChatGPT isn't profitable with 800 million users.
Google wants to increase compute 1000x
So .... 800 billion users should do it.
Everyone needs to go out and start fucking everything they can find. OpenAI needs more (100x more than the total population) users.
Tell me again how this isn't a bubble.
Google wants to increase compute 1000x
So .... 800 billion users should do it.
Everyone needs to go out and start fucking everything they can find. OpenAI needs more (100x more than the total population) users.
Tell me again how this isn't a bubble.
Upvote
22
(23
/
-1)
these guys should know better, there are datacenters that were built 3 years ago that are waiting on grid connections because there isn't enough power. the industries that make all this equipment can't keep up with geometric scaling, its not possible. GE and Pratt won't be able to boost output of gas turbines by much because the aviation industry is already in a shortage of parts because of covid when a large amount of aero workers were let go, nuclear is just greenwashing none of them are serious about it, and renewables are dead in this country right now.
Upvote
19
(19
/
0)
The issue is that when it bursts, it will inevitably collapse the greater economy... just like the Dot Com, just like the Housing Crisis. And then, the American tax payer will be the one to pick up the bill and feel the pain. Those well-off enough can shield their savings by diversification, but the average dude out there with a 401K, just scraping by as it is is boned.
Upvote
9
(10
/
-1)
If they want to stop those data centers, shorting the AI companies and flooding the news with reasonable arguments that the accomplishments of LLMs don't mean Jack Shit if they can't scale and their accuracy doesn't improve to the 96% thousandth decimal point from where they are at now... with less effort and time than legal battles to enforce zoning laws.
It sucks to say but waking up the non-tech-broligarch class to the fact that LLMs are a scam in the white collar space to get them to drop the contracts, including ChatGPT subs is cheaper, faster and more effective than legal injunctions. AKA working to pop the market bubble faster by playing the market financial stock market game better than the broligarchs to no one's benefit.
And that sucks.
Upvote
10
(10
/
0)
I will definitely be on the lookout for fire-sale toys for the homelab. There's a tidal wave on the horizon.
Upvote
9
(9
/
0)
Bingo. The WaPo has a wildly optimistic bubble/no bubble article but one of the interesting bits is that they looked at the revenue side of the AI business.
It's not a pretty picture.
Basically, the AI industry need to generate an additional $650 billion in revenue every year just to get a 10% return on their current investments. OpenAI is estimated to be at about $20 billion. Alphabet/Google's total worldwide revenue in 2024 was just under $350 billion.
Now how many people think that the Silicon Valley elite are going to be happy with just a 10% return?
Upvote
26
(26
/
0)
Melon of Troy
Ars Centurion
It's amazing that so few "investors" care about the fundamentals of business...
Upvote
17
(17
/
0)
Grocery and housing prices are trending that way. They'll find a way.
Upvote
8
(8
/
0)
Unless Google is planning to fundamentally shift their entire business model, their investments in AI will cannibalise their existing search-based adserve revenue. What isn't going to increase 1000x is the amount of users you can serve ads to, or their collective buying power.
Upvote
3
(3
/
0)
Maybe if the bubble pops, they'll eventually get re-purposed for some kick-ass MMOs!
Upvote
1
(1
/
0)
Heh, maybe it will mean the God Machine doesn't worry so much about our alignment...
Upvote
0
(0
/
0)
What do you mean? My examples exemplified the study's findings.
I can do the pretraining of a gpt 3 level intelligence model (which would have cost millions for openai) for around $2k - from scratch.
Upvote
-18
(3
/
-21)
This is the part I don't understand at all. Used to be that paying advertisers got the top of Google's search results page all to themselves, followed by Google's generalized results. Now, the "AI Summary" appears in place of that top spot, and one of Google's main advertiser advantages - paying to have your product shoved in consumer's faces - is lost. The advertisers are the ones paying Google's bills, they're cutting advertisers off at the knees with the AI Summary, so...how does this work?
Upvote
23
(24
/
-1)
Upvote
26
(26
/
0)
They'll slowly find malicious ways to force us. Gmail no longer automatically sorts out "promotions", "social" and other categories unless you agree to full AI scanning of your email, drive and other Google services. What used to be a handy auto-sort filter feature is now locked behind an AI use agreement. They'll keep doing this until their services are miserable to use without agreeing to AI everywhere.
Upvote
16
(17
/
-1)
They're already sort of doing that by default for YT content creators, creating all the incentives that gradually turn people's eyes to slop content. It's just less viewer-facing.
Upvote
13
(13
/
0)
Your point of view explains OpenAI and Anthropic as well as the host of lesser companies. They are important players in the game to be sure.
But explaining the actions of Google and Microsoft (and Google is the focus of this article) takes more than that, because they and their shareholders are well past the gambling enthusiasm phase. They are blue chip stocks. The attitude is different and insisting on applying the wrong model for their choices will leave you wrong almost all the time. NVIDIA is also not a gambling company. At this point the big players in the game include leviathans that are acting like the upstarts because.. why? I believe it's fear.
Upvote
-7
(2
/
-9)
F!yeah can you imagine the Bitcoin one of those centres could mine?!
Especially if it comes with its own refurbished coal-fired power plant.
Upvote
0
(3
/
-3)
I think it's basically because they've hit the wall, and they know it. They're not going to design or innovate or invent their way around the simple fact that they've taken LLMs as far as they can go, and they will never be any better than what they are right now, because their current performance is a maximization of their fundamental underlying design. Management is trying to bluster their way past this dead end, trying to cover that their incessant demand for more and more money and resources is a shell game without a pea.
Upvote
14
(16
/
-2)
I'm with you on that lunch, I it might make turkey taste ok. So all those millions of people just didn't KNOW they wanted it, which is what Sam Altman would say too
Upvote
0
(0
/
0)
And the climate ?
And the need to fund public health ?
And the war against Russia ? (Or with Russia and so one day against UE. After all you are amurikans, here)
And the need to fund public health ?
And the war against Russia ? (Or with Russia and so one day against UE. After all you are amurikans, here)
Upvote
7
(8
/
-1)
I think you misunderstood. I said "not just". My point was in addition to yours. CEOs of different companies have different motivations than other companies. I wasn't talking about OpenAI or Anthropic. CEOs of companies that don't actually create their own AIs are the ones being driven by investors hell bent on getting those big returns when the company they are invested in uses AI to somehow magically replace its workforce or give them new capabilities to increase their growth/sales.
And the AI companies need them to be afraid of being left in the dust so they can sell their snake oil to them. Companies like Johnson & Johnson, Boeing, News Corp, etc. Big companies like that, but also much smaller companies as well.
Upvote
7
(7
/
0)
So RAM is going to be really really expensive for a little while here, and then used RAM is going to be really really cheap for a good long while (as well as servers, server racks, firewalls, etc etc).
Upvote
5
(5
/
0)
Easy! You now have to watch 16 adverts before you YouTube video plays....
Upvote
13
(13
/
0)