Google unveils Ironwood, its most powerful AI processor yet

Ironwood will be available in configurations of up to 9,216 liquid cooled chips.

See full article...

 

[entropy_wins]

So that TFLOPS is really TFLOPS(FP8)?

 

[afidel]

Google says the higher inference speed and efficiency of Ironwood sets the stage for more breakthroughs in the coming year.

It does? Doesn't it just increase the rate of training which will lead to incremental improvements in existing methods? Breakthroughs happen because someone does new research that leads to a new approach or uses an existing approach in a novel way, simply increasing the amount of CPU power does little to fundamentally change the way things work.

 

[ProdigySim]

It sounds like maybe these chips are inference focused? Inference is a really really basic operation compared to training, so it makes sense there's a lot of room for optimization. Hopefully it helps with energy consumption as well

 

[wildsman]

Google says the higher inference speed and efficiency of Ironwood sets the stage for more breakthroughs in the coming year.

It does? Doesn't it just increase the rate of training which will lead to incremental improvements in existing methods? Breakthroughs happen because someone does new research that leads to a new approach or uses an existing approach in a novel way, simply increasing the amount of CPU power does little to fundamentally change the way things work.

As Demis Hassabis said, they are going to try both approaches. They're going to continue scaling and try to come up with architectural breakthroughs as well - the additional compute will help speed things up regardless of the approach.

 

[End_of_Eternity]

Idiocracy (2006)

But sadly, the greatest minds and resources were focused on conquering hair loss and prolonging erections.

 

[BigOlBlimp]

It sounds like maybe these chips are inference focused? Inference is a really really basic operation compared to training, so it makes sense there's a lot of room for optimization. Hopefully it helps with energy consumption as well

Inference is a nontrivial part of training. You can’t know how to optimize your model if you can’t measure how wrong it is against training data.

 

[Tom Terrific]

Who’s fabbing the various parts? What node(s)? Where?

 

[sarusa]

which Google prefers to call "thinking."

Spoiler: It is not actually thinking (and yeah, Ryan knows this). We're at the 'hit a big wall so oh boy you thought the marketing fluff was bad before?' phase.

 

[afidel]

Who’s fabbing the various parts? What node(s)? Where?

TSMC, probably N2 but some reports are saying an N3 variant, Taiwan.

 

[urbanCTO]

Human brains use about 20w. Roughly a bright LED.
These chips = how many watts again?

We have some ways to go.

 

[picklefactory]

very disappointed in Google for not including a column in their benchmark chart for 𝒕𝒉𝒆 𝒗𝒊𝒃𝒆𝒔

 

[tuna74]

It would be nice if there were more details on how you actually program these devices. At what level are you suppose to interact with them?

 

[ShatteredMirrorLogic]

Do I calculate correctly that ONE pod has about 1.7 Petabytes of RAM? (9216 * 192 GB)

 

[blank0]

Imagine what kind of wonders one can hallucinate with all this raw power !

 

[cacarr]

" ... like simulated reasoning ... "

Please provide a precise definition of real reasoning.

 

[Echohead2]

Human brains use about 20w. Roughly a bright LED.
These chips = how many watts again?

We have some ways to go.

How far have we came in 10 years? 20 years? 30 years? How far will it be in another 10, 20, 30?

 

[Kenjitsuka]

Thanks for the Google marketing copy. Some unsupported claims and puffing itself up.

 

[rwhitwam]

" ... like simulated reasoning ... "

Please provide a precise definition of real reasoning.

To be clear, we like to use the phrase "simulated reasoning" at Ars to delineate the sometimes confusing boundary between the the way brains reason and the kind of reasoning AI models do. As far as we can currently tell, these two processes are not the same.

 

[Lord Bayaz]

I heard Ironwood was the original name of Viagra.

Looks like OpenAi has some stiff competition

 

[GMcK de Cypress]

Blindingly missing are any comparisons with Nvidia products. Perhaps because Nvidia was claiming 4000 FP8 TFLOPS for the H100 back in 2022.

 

[graylshaped]

Whenever Google talks about the capabilities of a new Gemini version, it notes that the model's capabilities are tied not only to the code but to Google's infrastructure.

"Here I am, brain the size of a planet, and you want me to decide which ads to serve to a site featuring user-generated hentai haiku."

 

[staskaya]

" ... like simulated reasoning ... "

Please provide a precise definition of real reasoning.

If you can count the letter r in strawberries it's real reasoning, if you can't...

 

[jock2nerd]

From a quick review of Google's statements on this product, I can't tell if it is specifically dedicated to FP8 computations, which is the core computation unit for AI models, or whether it can be used for other purposes with either fixed point or larger floating point variables.

 

[SeanJW]

Google says the higher inference speed and efficiency of Ironwood sets the stage for more breakthroughs in the coming year.

It does? Doesn't it just increase the rate of training which will lead to incremental improvements in existing methods? Breakthroughs happen because someone does new research that leads to a new approach or uses an existing approach in a novel way, simply increasing the amount of CPU power does little to fundamentally change the way things work.

Faster inferencing meant things like search predictively happening in real time as you type could be created. The original TPUs (which were inferencing only) were created for exactly that reason - without them, Google was estimating that they'd need to double their online compute to do things like that.

It's not just training that matters, its actually running the models that matters more.