Microsoft declares its underwater data center test was a success

Underwater pods can reduce latency by moving cloud services closer to customers.

Read the whole story

 

[traumadog]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

Thing is, all of this heat would have gone into the atmosphere already.

At least using the ocean as a heat sink consumes way less energy than running on-shore AC for the facility, thus being a relative net gain overall.

While it's true that the heat would have gone into the environment anyway, the OP's point is likely that the posited central heating will have to be achieved be other means without the servers thus effectively doubling the heat load to the environment.

True, but in order to see a net overall benefit, the data center owners would have to be ok with their centers physically located in residential areas, directly piping the heat to homeowners over relatively short runs.

As I recall, 40% of a data center's electricity need comes from the need to cool the equipment. Despite that, there likely isn't enough heat to warm more than a few homes this way, while electricity use for cooling and transporting said heat would still remain high.

 

[Snarky Robot]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

Others have covered the fact that the reduced power load from lack of AC almost certainly outpaces this, but also, it's...really unrealistic given the size of the oceans. There are over 1.3 trillion liters of seawater in the world. It takes 4,200 joules to raise it by one degree. That's 5,586,000,000,000,000 joules.

You could put 100,000 of these in the ocean, each putting off 100,000 joules a year for 100,000 years, and it would have raised the temperature of the ocean by less than 2/10ths of a degree (celsius).

 

[traumadog]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

Thermodynamics fail.

Global warming is not caused by humans creating heat. There is nothing humans could do that would produce even the tiniest fraction of the thermal energy that reaches the Earth from the sun.

Global warming is caused by GHG which increase the percentage of the thermal energy from the sun which is trapped. The sun is so off the charts powerful compared to anything heat sources produced by humans that even the tiniest increase in the greenhouse effect can raise global temperatures.

On the flip side, since a lot of electricity is used to cool data centers, the production of said electricity to cool data centers does come at a GHG cost.

The beauty of the underwater data center is the (likely relatively large) lifetime reduction in overall electricity needs.

 

[dirtside]

Another factor is the potential for littering the oceans with vaguely toxic electronics debris. Say someone puts one of these pods down there, and then goes bankrupt. Whose responsibility is it now to remove it? Eventually (maybe decades, but eventually) the pressure hull will rust through.

I'm not saying this is a showstopper, but it's something that needs to be considered. Companies will swear up and down that they'll take responsibility for long-term cleanup, but we all know how often that's a lie.

 

[Jeff S]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

Others have covered the fact that the reduced power load from lack of AC almost certainly outpaces this, but also, it's...really unrealistic given the size of the oceans. There are over 1.3 trillion liters of seawater in the world. It takes 4,200 joules to raise it by one degree. That's 5,586,000,000,000,000 joules.

You could put 100,000 of these in the ocean, each putting off 100,000 joules a year for 100,000 years, and it would have raised the temperature of the ocean by less than 2/10ths of a degree (celsius).

I think there's something missing from your explanation, somewhere in the vicinity of "It takes 4,200 joules to raise it by one degree. That's 5,586,000,000,000,000 joules."

Do you mean 4,200 joules per liter?

 

[Deleted member 545801]

Welp. Microsoft has truly reached a new low. They're tanking. Totally underwater.

Seems fishy.

 

[afidel]

The thing I don't get is that MS was touting the superior reliability, but the numbers don't make sense to me. If the numbers they were quoted as saying are right then they have a 4% AFR (Annual Failure Rate) for their 'traditional' datacenters. The only thing I ran into with anywhere near that kind of AFR in a datacenter context was nearline HDDs, nothing else was above 1.5%, and from a system perspective we had well under 1% AFR. Now I know that MS runs provider scale datacenters, so perhaps they have optimized their designs WAY more for cost and efficiency than traditional datacenters, but that would just mean that they managed to achieve normal COTS levels of AFR with this bespoke design, hardly something worth crowing about IMHO.

 

[alansh42]

Although you don't need a datacenter for the servers themselves or A/C for them, you still need the usual power and data support: UPS, backup generators, and routers. And they have to be located close to the shore, with a right-of-way to the water.

 

[Snarky Robot]

I think there's something missing from your explanation, somewhere in the vicinity of "It takes 4,200 joules to raise it by one degree. That's 5,586,000,000,000,000 joules."

Do you mean 4,200 joules per liter?

I'm an attorney. Messing up numbers is, like, our way of being humble.

But yes, it takes ~4,200 joules to raise one liter of water one degree celsius. I didn't find a number for seawater from a quick search, so there may need to be some fiddling there, but overall, the general concept should hold true.

You'd have to have a stupidly high number to affect the heat of the ocean in any reasonable period of time.

 

[pkmnfrk]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

Others have covered the fact that the reduced power load from lack of AC almost certainly outpaces this, but also, it's...really unrealistic given the size of the oceans. There are over 1.3 trillion liters of seawater in the world. It takes 4,200 joules to raise it by one degree. That's 5,586,000,000,000,000 joules.

You could put 100,000 of these in the ocean, each putting off 100,000 joules a year for 100,000 years, and it would have raised the temperature of the ocean by less than 2/10ths of a degree (celsius).

Note that 100,000J/y is a very very small amount of energy, ~3.171 milliwatts. I don't know how much power these pods use, but I'm willing to bet it's at least a million times highter (thousands of watts)

Edit: Even I lowballed it. A single rack can use 7-20 kW, and there looks to be 12ish? So, let's say 120 kW for the whole pod. Wolfram Alpha gives me 3.784×10^12 J/yr, which is about 10 million times higher.

 

[yakinabe]

How does the cooling actually work? Is the pod itself essentially a heatsink for all the servers, or is there internal cooling/airflow that then routes to the edge of the pod to cool? Can they go as extreme as not even having fans on the servers?

Did you look at the photos? They appear to be conventional air-cooled racks inside the pod. So airflow carries heat from the servers to the pod surface, which is constantly cooled by seawater.

 

[Mark ASDF]

Did they mention whether they pressurized to the water pressure at that depth, or at sea level? Higher pressure would help prevent leaking and increase thermal transfer to the air (presumably the servers still had fans), but may cause issues if the hard drives weren't designed for the thicker air. And how about heat sink fins on the inside? Just to make the small space even more hazardous to move around in

And was there any battery backup built into the system? Again, presumably there was some sort of power supply conditioning. Was there even 110/220 volt power available or was it higher power with oil filled transformers on the outside?

So many questions...

 

[ColdWetDog]

Another factor is the potential for littering the oceans with vaguely toxic electronics debris. Say someone puts one of these pods down there, and then goes bankrupt. Whose responsibility is it now to remove it? Eventually (maybe decades, but eventually) the pressure hull will rust through.

I'm not saying this is a showstopper, but it's something that needs to be considered. Companies will swear up and down that they'll take responsibility for long-term cleanup, but we all know how often that's a lie.

While a non zero issue, it's completely dwarfed by *all of the other crap* (both literally and figuratively) that we dump into the oceans. Not even a rounding error.

 

[Jim Salter]

Disappointed that you didn't mention a "deep dive" into the topic anywhere in this piece.

I started to go off on a whole "surface" pun thread in the middle, but then decided that it was too confusing, and that I was no Beth Mole.

 

[afidel]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

Thermodynamics fail.

Global warming is not caused by humans creating heat. There is nothing humans could do that would produce even the tiniest fraction of the thermal energy that reaches the Earth from the sun.

Global warming is caused by GHG which increase the percentage of the thermal energy from the sun which is trapped. The sun is so off the charts powerful compared to anything heat sources produced by humans that even the tiniest increase in the greenhouse effect can raise global temperatures.

On the flip side, since a lot of electricity is used to cool data centers, the production of said electricity to cool data centers does come at a GHG cost.

The beauty of the underwater data center is the (likely relatively large) lifetime reduction in overall electricity needs.

Meh, Microsoft already proved that given the right location you can run a datacenter without supplemental cooling, their Dublin location went 100% freecool back in 2009.

 

[David Crowell]

Azure, like the deep sea...

 

[afidel]

Not a great idea, unlike Physical Land location, if something does go wrong it could take days to repair and worst-case scenario a complete and total loss of all data.

And? Google/Microsoft/Amazon run at the scale of RAID, redundant array of inexpensive datacenters. In order for thing to make sense for their scale they have to not care if not only a single server dies, but an entire datacenter goes offline since that will inevitably happen when you're running millions of nodes in dozens of locations.

 

[gmerrick]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

Unless the hull of the pod was specifically designed to be EM resistant and the input power was filtered and isolated, there's a good chance that a large em pulse would take it out anyway.

 

[Jim Salter]

you also gain security benefits (nobody around to try and break in)

That is not going to be a forever thing—and I, for one, was already excited for future movies featuring SCUBA-enabled underwater physical pentests while I was writing the article.

I mean. There could legitimately be GUARD SHARKS. With frickin' lasers! (Okay, maybe not so much the lasers. And dolphins would probably be a better choice, or maybe sealions. And... and... and I still wanna see that movie.)

 

[Jamjen831]

Couldn't they just seal datacenters and pressurize them with nitrogen on land? That seems like that'd get you basically all the benefits of this setup without having to, you know, actually submerge the thing in the ocean.

Except for all the water cooling. And not needing expensive, hard to develop land.

Don't forget the sizable infrastructure for security and maintenance. Not needing to employ 24/7 security, not having any hands touching anything. Compute node dies? So what, there's plenty others to take it up. Pull the whole thing after X years and refurbish.

 

[ColdWetDog]

you also gain security benefits (nobody around to try and break in)

That is not going to be a forever thing—and I, for one, was already excited for future movies featuring SCUBA-enabled underwater physical pentests while I was writing the article.

I mean. There could legitimately be GUARD SHARKS. With frickin' lasers! (Okay, maybe not so much the lasers. And dolphins would probably be a better choice, or maybe sealions. And... and... and I still wanna see that movie.)

Are there Sea Bass in the North Sea?

 

[alansh42]

Not a great idea, unlike Physical Land location, if something does go wrong it could take days to repair and worst-case scenario a complete and total loss of all data.

For modern large-scale data centers there are two sorts of failures: normal individual component failures which are left alone until the next scheduled R&R cycle, and catastrophic failures which take down most or all of the data center. The second one is always going to be days to repair, and potential total loss of data.

The idea is that you're locally redundant enough to take single failures, and geographically redundant to handle catastrophic failures. In case of an underwater pod failing it's no different than a fire at an above-ground data center. You take it offline and route its work somewhere else.

 

[Snarky Robot]

Note that 100,000J/y is a very very small amount of energy, ~3.171 milliwatts. I don't know how much power these pods use, but I'm willing to bet it's at least a million times highter (thousands of watts)

I think I may also have messed up on my conversion of KM^3 to liters. I think it should have been ~5.6^e24.

So, 56,000,000,000,000,000,000,000,000 liters.

Let's call it 100,000 of these again. That's 56,000,000,000,000,000,000 joules you'd need, per container. Let's call it 100 billion joules per container per year. That's 56,000,000 years it would take to raise it one degree. Another way of expressing it is if you wanted to raise the ocean 1/10th of a degree, over the course of 100 years, it would take 5,600,000,000,000,000,000,000 joules per year to do so. If each unit puts out 100,000,000,000 joules per year, you would need 5,600,000,000 of these things. If each one is 10 meters squared space, you'd need 56,000,000 km^2 of space. Or approximately an area 10 times the size of Europe.

This is all if my calculations are correct. Which, again, they almost certainly are not.
In fact, my guess is that by the time I've finished editing this, they've already been proven wrong.

Edited words to make units more clear
Edited again to try fancy maths regarding number of units needed

 

[alansh42]

you also gain security benefits (nobody around to try and break in)

That is not going to be a forever thing—and I, for one, was already excited for future movies featuring SCUBA-enabled underwater physical pentests while I was writing the article.

I mean. There could legitimately be GUARD SHARKS. With frickin' lasers! (Okay, maybe not so much the lasers. And dolphins would probably be a better choice, or maybe sealions. And... and... and I still wanna see that movie.)

Sounds like scene from a Bond movie -- 007 is trying to tap into a data pod for SPECTRE's underwater lair and gets attacked by sharks!

 

[Albino_Boo]

Another factor is the potential for littering the oceans with vaguely toxic electronics debris. Say someone puts one of these pods down there, and then goes bankrupt. Whose responsibility is it now to remove it? Eventually (maybe decades, but eventually) the pressure hull will rust through.

I'm not saying this is a showstopper, but it's something that needs to be considered. Companies will swear up and down that they'll take responsibility for long-term cleanup, but we all know how often that's a lie.

The Germans sunk 21 million tons of shipping in the Atlantic. In the size of the oceans you could dump ever single server made in each year and still not equal the monthly tonnage sunks by the Germans.

 

[GottaSaySomething]

Another factor is the potential for littering the oceans with vaguely toxic electronics debris. Say someone puts one of these pods down there, and then goes bankrupt. Whose responsibility is it now to remove it? Eventually (maybe decades, but eventually) the pressure hull will rust through.

I'm not saying this is a showstopper, but it's something that needs to be considered. Companies will swear up and down that they'll take responsibility for long-term cleanup, but we all know how often that's a lie.

The Germans sunk 21 million tons of shipping in the Atlantic. In the size of the oceans you could dump ever single server made in each year and still not equal the monthly tonnage sunks by the Germans.

At some point it has to be water under the bridge, move on.

 

[QuantifiableQuoll]

you also gain security benefits (nobody around to try and break in)

That is not going to be a forever thing—and I, for one, was already excited for future movies featuring SCUBA-enabled underwater physical pentests while I was writing the article.

I mean. There could legitimately be GUARD SHARKS. With frickin' lasers! (Okay, maybe not so much the lasers. And dolphins would probably be a better choice, or maybe sealions. And... and... and I still wanna see that movie.)

It's already not a forever thing. I'd like to point out operation Ivy Bells.

Is this the kind of thing that's going to be an ongoing problem? Hell no, but it's possible. And awesome.

 

[wolfwood6]

you also gain security benefits (nobody around to try and break in)

That is not going to be a forever thing—and I, for one, was already excited for future movies featuring SCUBA-enabled underwater physical pentests while I was writing the article.

I mean. There could legitimately be GUARD SHARKS. With frickin' lasers! (Okay, maybe not so much the lasers. And dolphins would probably be a better choice, or maybe sealions. And... and... and I still wanna see that movie.)

It's already not a forever thing. I'd like to point out operation (Ivy Bells)[https://en.wikipedia.org/wiki/Operation_Ivy_Bells].

Is this the kind of thing that's going to be an ongoing problem? Hell no, but it's possible. And awesome.

You don't have to break into the room, just the cables. We can do that already.

 

[pkmnfrk]

Note that 100,000J/y is a very very small amount of energy, ~3.171 milliwatts. I don't know how much power these pods use, but I'm willing to bet it's at least a million times highter (thousands of watts)

I think I may also have messed up on my conversion of KM^3 to liters. I think it should have been ~5.6^e24.

So, 56,000,000,000,000,000,000,000,000 liters.

Let's call it 100,000 of these again. That's 56,000,000,000,000,000,000 joules you'd need, per container. Let's call it 100 billion joules per container per year. That's 56,000,000 years it would take to raise it one degree, if my calculations are correct.

Which, again, they almost certainly are not.

Edited words to make units more clear

That I think makes a lot more sense! Numbers are hard!

 

[Jim Salter]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

Unless the hull of the pod was specifically designed to be EM resistant and the input power was filtered and isolated, there's a good chance that a large em pulse would take it out anyway.

The power would be the only issue. 120 feet of water is one hell of an EMF attenuator. It takes about 5cm of water to halve the intensity of gamma rays below 200 keV.

For reference, 120 feet of water is roughly equivalent to six to twelve feet of solid lead as a radiation shield...

 

[Sukasa]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

Unless the hull of the pod was specifically designed to be EM resistant and the input power was filtered and isolated, there's a good chance that a large em pulse would take it out anyway.

The power would be the only issue. 120 feet of water is one hell of an EMF attenuator. It takes about 5cm of water to halve the intensity of gamma rays below 200 keV.

For reference, 120 feet of water is roughly equivalent to six to twelve feet of solid lead as a radiation shield...

Knowing almost nothing about EMF/EMPs, does the big power cable going to it (say, a hypothetical shore power cable to go with your hypothetical EMP) make any difference to that?

 

[NotYourUsername]

its grid is supplied 100 percent by wind, solar, and experimental green technologies

Not entirely, the wind doesn't always blow after all (even on the Orkney islands). Some googling shows there are currently two subsea cables to the mainland (PDF) that allow the islands to import electricity when generation from sun and wind is low and export excess renewable generation.

Currently this is limited to 40MW (previous link), but a new 220MW interconnector has recently been greenlit (conditionally), which will also pave the way for at least 135MW of new wind generation.

 

[GottaSaySomething]

Since my comment just above, I found: https://news.microsoft.com/innovation-s ... atacenter/

I'd have loved a PDF but it's very fluffy, ending with "how we helped fight Covid" - it's like any turing complete thing can compute (there's a film called "The Accountant" ends with a supposedly great computer "you kidding, she could hack the pentagon with that thing" - any computer would do)

This is just what they computed with it, that has nothing to do with all the prep work which I'm sure we'd all want to know about.

Like I want to know about cooling, surely the hull wasn't it (look at the picture with all the fans on the units....)

I'm not sure what my point is, I guess I'm a bit pissy because I didn't hold down ctrl, press 7-10 things ending with ".pdf" and have some reading about factors I'd never consider and how they built this.

 

[Albino_Boo]

Another factor is the potential for littering the oceans with vaguely toxic electronics debris. Say someone puts one of these pods down there, and then goes bankrupt. Whose responsibility is it now to remove it? Eventually (maybe decades, but eventually) the pressure hull will rust through.

I'm not saying this is a showstopper, but it's something that needs to be considered. Companies will swear up and down that they'll take responsibility for long-term cleanup, but we all know how often that's a lie.

The Germans sunk 21 million tons of shipping in the Atlantic. In the size of the oceans you could dump ever single server made in each year and still not equal the monthly tonnage sunks by the Germans.

At some point it has to be water under the bridge, move on.

What are talking about, that figures represents 100,000s tonnes of oil, asbestos, copper, phosphorus and other hazardous chemicals. There are 1000s of tonnes of chemical weapons in the Irish seas and the Baltic. So how about you work that facts matter.

 

[GottaSaySomething]

Note that 100,000J/y is a very very small amount of energy, ~3.171 milliwatts. I don't know how much power these pods use, but I'm willing to bet it's at least a million times highter (thousands of watts)

I think I may also have messed up on my conversion of KM^3 to liters. I think it should have been ~5.6^e24.

So, 56,000,000,000,000,000,000,000,000 liters.

Let's call it 100,000 of these again. That's 56,000,000,000,000,000,000 joules you'd need, per container. Let's call it 100 billion joules per container per year. That's 56,000,000 years it would take to raise it one degree, if my calculations are correct.

Which, again, they almost certainly are not.

Edited words to make units more clear

That I think makes a lot more sense! Numbers are hard!

1 meter cubed is 1000 litres, so it's 1000x1000x1000 of those, 10^12 L = 1km^3

 

[Jim Salter]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

Unless the hull of the pod was specifically designed to be EM resistant and the input power was filtered and isolated, there's a good chance that a large em pulse would take it out anyway.

The power would be the only issue. 120 feet of water is one hell of an EMF attenuator. It takes about 5cm of water to halve the intensity of gamma rays below 200 keV.

For reference, 120 feet of water is roughly equivalent to six to twelve feet of solid lead as a radiation shield...

Knowing almost nothing about EMF/EMPs, does the big power cable going to it (say, a hypothetical shore power cable to go with your hypothetical EMP) make any difference to that?

Not directly. The cable's under all that water also. Your real concern is whether the EMP causes enough of a surge in any above-ground portion of the grid to propagate down that shore power line, and whether or not you've got breakers sufficiently fast and capable to interrupt the surge before it hits the expensive stuff.

Along, of course, with whether or not you've got some other way of powering the goods once the power goes out. And/or whether the servers are able to power themselves back on if power is ever interrupted to the entire pod.

The Marine Energy Research Centre is specifically mucking about with tidal current and wave motion electrical generation, which would in theory remove that avenue for failure also. If you end up not needing shore power at all and the only thing tethering the pod to the surface is fiber, you'd no longer have a route for an EMP to screw up your pod at all. But that's an if.

 

[ARS_dabbler]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

The heat is going into the environment one way or the other anyway. Even if the heat is redirected to heating a building, the heat then goes from the building into the environment.

Also, the amount of heat the oceans absorb from these would be negligible compared to what they absorb from solar radiation every single day. You might see some minor local heating in the immediate environment, but that's not really a concern compared to the reduced energy usage (and therefore reduced CO2 emissions) needed for the cooling system.

My point about “recycling” the heat is that you don’t need to produce two units of heat (1 as a byproduct from the data center, the other for keeping buildings at liveable temperature). Byproduct heat of data center is subtracted in part from energy required to heat a building; this leading ultimately to less overall heat dumped in the atmosphere. What am I missing?

What you are missing is the concept of exergy. The heat discharged from server system is low grade heat; it's temperature is only a little above ambient. Moving it to where it would provide heating to residents of a building would consume more capital and energy than its worth.

Then again in the summer months you are back to discharging the heat into the environment.

Large structures used to be heated by their lighting systems. This is using heat in a similar manner to what you are suggesting. But the lighting systems are already distributed about the building and thus the heat is also distributed. Then again in the summer they had to run AC to compensate for the lighting. They have now migrated to LED lighting reducing the AC in the summer but requiring some heating in the winter. Overall still a net energy reduction.

 

[Jeff S]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

Unless the hull of the pod was specifically designed to be EM resistant and the input power was filtered and isolated, there's a good chance that a large em pulse would take it out anyway.

Isn't a metal cylinder basically a Faraday Cage? Also, isn't 20+ feet of water an excellent shield from radiation of all types? I remember reading the submarines have special very low bandwidth, extremely low frequency radios, because that's all that they can get to really penetrate the ocean (they also, if I understand right have more normal radios, but I think have to basically surface or get to shallow depths of water, to use them).