Microsoft declares its underwater data center test was a success

Underwater pods can reduce latency by moving cloud services closer to customers.

Read the whole story

 

[Nightdweller]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

I think (from the photos) it’s in a metal container so it might function as a faraday cage...

 

[C64 raids Bungling Bay]

you also gain security benefits (nobody around to try and break in)

That is not going to be a forever thing—and I, for one, was already excited for future movies featuring SCUBA-enabled underwater physical pentests while I was writing the article.

I mean. There could legitimately be GUARD SHARKS. With frickin' lasers! (Okay, maybe not so much the lasers. And dolphins would probably be a better choice, or maybe sealions. And... and... and I still wanna see that movie.)

Green lasers, they work underwater without terrible redlight attenuation. As demonstrated on Shark Week this year.

 

[williamyf]

Couldn't they just seal datacenters and pressurize them with nitrogen on land? That seems like that'd get you basically all the benefits of this setup without having to, you know, actually submerge the thing in the ocean.

I wonder ... MS (and others) has some very large data centers in Quincy, Washington state, which is also a farming region with high grade top soil about 20 feet thick. Just up the street is a chain of hydro dams churning out some of the cheapest power in the country. Just up a little further is Wenachee, the apple capitol of the World. There, the fruit growers use enormous warehouses filled with nitrogen to keep the fruit prior to shipping. So, you have the infrastructure and knowledge in place in Eastern Washington to do exactly what you thought, just with bits not pits (stone fruit). Ever wonder how you get a "fresh" apple in April?

Aside: these data centers also resulted in fiber to the farm in Grant county spliced off the cable that runs to Seattle and who knows where else along the Burlington Northern tracks. Friends and family there had fiber to their houses years before I got mine in Northern California.

submerge the pods in the Dam, best of both worlds. You get the knowledge, the hydro, the fiber and the free water cooling.

 

[williamyf]

you also gain security benefits (nobody around to try and break in)

That is not going to be a forever thing—and I, for one, was already excited for future movies featuring SCUBA-enabled underwater physical pentests while I was writing the article.

I mean. There could legitimately be GUARD SHARKS. With frickin' lasers! (Okay, maybe not so much the lasers. And dolphins would probably be a better choice, or maybe sealions. And... and... and I still wanna see that movie.)

Green lasers, they work underwater without terrible redlight attenuation. As demonstrated on Shark Week this year.

Blue/UV lasers would be even better. Any scuba diver worth his/her salt(er) (pun intended) would tell you that. You shoud see what a 40mts dive does to a color chart. I still keep mine from the AOW course

Edit: Added a pun as a joke.

 

[tigerhawkvok]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

The heat is going into the environment one way or the other anyway. Even if the heat is redirected to heating a building, the heat then goes from the building into the environment.

Also, the amount of heat the oceans absorb from these would be negligible compared to what they absorb from solar radiation every single day. You might see some minor local heating in the immediate environment, but that's not really a concern compared to the reduced energy usage (and therefore reduced CO2 emissions) needed for the cooling system.

The total heat released is almost certainly *less*, to boot. In a datacenter, you need to run cooling machines to dump the heat, and there's an efficiency penalty there.

Submerged in 4C water just convection probably does the job, removing at least one system of efficiency loss.

 

[quijoticmoose]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

Unless the hull of the pod was specifically designed to be EM resistant and the input power was filtered and isolated, there's a good chance that a large em pulse would take it out anyway.

The power would be the only issue. 120 feet of water is one hell of an EMF attenuator. It takes about 5cm of water to halve the intensity of gamma rays below 200 keV.

For reference, 120 feet of water is roughly equivalent to six to twelve feet of solid lead as a radiation shield...

Jim,

You've had the misfortune to make a statement on a topic I'm giving a lecture on tomorrow. Prepare to be...commented...

Your figure--5 cm of water halving the intensity of gamma rays/x-rays below 200 keV is assuming that the x-rays are interacting via the Compton effect.

For a nice oversimplified picture, in this case, the photons have a short enough wavelength that they interact with individual electrons in a pretty particle-like manner. Their attenuation is really just driven by the sheer amount of stuff.

EMP pulses, based off of my fuzzy memory and backed up with a quick Google search, have lots of lower-frequency (longer wavelength) components. I'd be fearful of using Compton effect HVLs on them; RF energies act much more like waves and interact differently.

 

[Jim Salter]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

Unless the hull of the pod was specifically designed to be EM resistant and the input power was filtered and isolated, there's a good chance that a large em pulse would take it out anyway.

The power would be the only issue. 120 feet of water is one hell of an EMF attenuator. It takes about 5cm of water to halve the intensity of gamma rays below 200 keV.

For reference, 120 feet of water is roughly equivalent to six to twelve feet of solid lead as a radiation shield...

Jim,

You've had the misfortune to make a statement on a topic I'm giving a lecture on tomorrow. Prepare to be...commented...

Your figure--5 cm of water halving the intensity of gamma rays/x-rays below 200 keV is assuming that the x-rays are interacting via the Compton effect.

For a nice oversimplified picture, in this case, the photons have a short enough wavelength that they interact with individual electrons in a pretty particle-like manner. Their attenuation is really just driven by the sheer amount of stuff.

EMP pulses, based off of my fuzzy memory and backed up with a quick Google search, have lots of lower-frequency (longer wavelength) components. I'd be fearful of using Compton effect HVLs on them; RF energies act much more like waves and interact differently.

5cm is an example, though. There's one hell of a fudge factor at play, here, and I am politely interested in your very specific scenario that involves damaging levels of RF energy penetrating 120 feet of seawater. Be certain to double-check your work.

 

[Snarky Robot]

Seafloor-based pods don't require expensive commercial real estate, and they get nearly free cooling from the surrounding tons of seawater.

One test data center probably isn't a problem, but how much can you use the ocean as a heatsink before it starts fucking up the ocean? Is it fine if it's geographically spread out enough or does spreading it out stop mattering once you've started doing it enough?

If only multiple people had discussed the math and shown how ridiculous this argument is!

 

[quijoticmoose]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

Unless the hull of the pod was specifically designed to be EM resistant and the input power was filtered and isolated, there's a good chance that a large em pulse would take it out anyway.

The power would be the only issue. 120 feet of water is one hell of an EMF attenuator. It takes about 5cm of water to halve the intensity of gamma rays below 200 keV.

For reference, 120 feet of water is roughly equivalent to six to twelve feet of solid lead as a radiation shield...

Jim,

You've had the misfortune to make a statement on a topic I'm giving a lecture on tomorrow. Prepare to be...commented...

Your figure--5 cm of water halving the intensity of gamma rays/x-rays below 200 keV is assuming that the x-rays are interacting via the Compton effect.

For a nice oversimplified picture, in this case, the photons have a short enough wavelength that they interact with individual electrons in a pretty particle-like manner. Their attenuation is really just driven by the sheer amount of stuff.

EMP pulses, based off of my fuzzy memory and backed up with a quick Google search, have lots of lower-frequency (longer wavelength) components. I'd be fearful of using Compton effect HVLs on them; RF energies act much more like waves and interact differently.

5cm is an example, though. There's one hell of a fudge factor at play, here, and I am politely interested in your very specific scenario that involves damaging levels of RF energy penetrating 120 feet of seawater. Be certain to double-check your work.

I don't need a specific scenario, I'm being pedantic!

Seriously--120 ft of water is a heck of a shield, and I'd be shocked if you could damage electronics through it. I was just taking issue with your specific figure, since it's based on a physical process that's negligible at lower photon energies.

 

[Nop666]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

Seawater is conductive, so yes, it should make for a pretty good Faraday cage. There'd likely still be some level of risk via the external cabling though.

 

[Nop666]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

The heat is going into the environment one way or the other anyway. Even if the heat is redirected to heating a building, the heat then goes from the building into the environment.

Also, the amount of heat the oceans absorb from these would be negligible compared to what they absorb from solar radiation every single day. You might see some minor local heating in the immediate environment, but that's not really a concern compared to the reduced energy usage (and therefore reduced CO2 emissions) needed for the cooling system.

My point about “recycling” the heat is that you don’t need to produce two units of heat (1 as a byproduct from the data center, the other for keeping buildings at liveable temperature). Byproduct heat of data center is subtracted in part from energy required to heat a building; this leading ultimately to less overall heat dumped in the atmosphere. What am I missing?

Moving heat from one place to another costs energy, producing additional heat in the process.

 

[domikai]

I wonder if being underwater would provide any protection from CMEs/geomagnetic storm type events?

So not only will the next solar storm wipe out the net, but when it comes back up, all we'll have is Bing.

 

[Frennzy]

It really is unfathomable.

I see what you did there.

 

[Bongle]

This, and all the other exotic data center locations like Iceland or the Arctic feel very sci-fi:
Problem: "We have so many tonnes of computronium packed so densely that both land availability and the computers melting is a problem"
Solution: "Throw them in the Ocean / below a glacier / at L2 in Earth's shadow"

Seems like a pretty reasonable project though. If MS is going to need the compute power anyway and this is actually cheaper in terms of energy/space/reliability, why not?

Plus you could potentially design the outside of the pods to act as an artificial reef for the couple years they're on the bottom.

Another potential upside is if the big tech companies suddenly start coveting vast swaths of oceans, they may inadvertantly become large nature reserves because they won't want their pods snagged by nets/anchors/etc.

 

[eggie]

Honestly this whole cloud thing is really misleading. Why not blimps!?

MS scuttled that plan after switching to SSDs instead of helium-filled HDDs.

 

[MJMullinII]

Just skimming through the article, my very first thought was how super-useful this would be in regard to cooling. I get the point of putting things closer to customers, **but** can you imagine how much tightly you would pack equipment if you knew you had an virtual infinite heatsink available without increasing your power budget?

 

[mikedelhoo]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

Global warming is not caused by humans creating heat. There is nothing humans could do that would produce even the tiniest fraction of the thermal energy that reaches the Earth from the sun.

<snip> & emphasis added.

Well, that's somewhat overstated; according to this document selected carefully by picking the first google hit, the Sun accounts for

more energy striking the earth’s surface in one and a half
hours (480 EJ)
than worldwide energy consumption in the year 2001 from all sources combined
(430 EJ)

If we wave our hands and round 2001's energy consumption up to 480 EJ, and assume the production was "somewhat efficient" in terms of waste heat and round it up further to, say 3 times that much, we have ballpark human production of heat equivalent to 4.5 hours of the annual total solar influx at the surface, which is to say 4.5 / (365.25 * 24) ~= 0.000513

I've seen tinier fractions ;-)

(Yes I'm quibbling, but you're Statistical after all.)

Edit: Darn, LesDawg beat me to it. I see I have to read all the comments before posting.

 

[NetworkGuy1912]

I know they aren’t putting these this far out, but if they DID put them in international waters, which country owns the data? Are these things “flagged” like ships?

It would take "pirate data center" to a whole new (deeper) level...

 

[Cognac]

While this is great for cooling, especially up there, I can't see any other meaningful benefits to doing this.

There's so many factors which could take the thing offline, stormy weather, shipping, fishermen, leaks.

Cool idea but just doesn't seem worth the hassle

This isn't some white paper by an undergrad just theorising on some abstract idea. It's the result of much planning, many small-scale tests, and now the completion of a i2 year in-situ study to test viability for exactly the reasons you just mentioned. :facepalm:

You can bet your bottom dollar that if Microsoft didn't think it might be worth the hassle they certainly wouldn't have sunk this much time and effort into it.

 

[h.m.]

If the big players in cryptocurrency mining decide to go underwater a lot of fish would quit the annual migration to warmer areas during the cold season and just bob around the data-centers in the nice toasty water.

 

[mjbvz]

Another factor is the potential for littering the oceans with vaguely toxic electronics debris. Say someone puts one of these pods down there, and then goes bankrupt. Whose responsibility is it now to remove it? Eventually (maybe decades, but eventually) the pressure hull will rust through.

I'm not saying this is a showstopper, but it's something that needs to be considered. Companies will swear up and down that they'll take responsibility for long-term cleanup, but we all know how often that's a lie.

Look: when things inevitably go to hell underwater, all you need to do is ask someone real kindly like and they will fly on down to take care of all your problems with a wrench.

(Source: my good Irish friend who has considerable firsthand experience working under the North Atlantic)

 

[SixDegrees]

Maintenance isn't the problem many are making it out to be. Server farms have enormous redundancy built in; it isn't necessary to rush in and replace a single faulty hard drive. More likely units like these would just be left down there until some significant percentage of failures occurred, then hauled up for repair and re-sunk.

The eight-times better lifetime of equipment found in these trials makes maintenance even less of a concern.

 

[Azethoth666]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

Thing is, all of this heat would have gone into the atmosphere already.

At least using the ocean as a heat sink consumes way less energy than running on-shore AC for the facility, thus being a relative net gain overall.

While it's true that the heat would have gone into the environment anyway, the OP's point is likely that the posited central heating will have to be achieved be other means without the servers thus effectively doubling the heat load to the environment.

True, but in order to see a net overall benefit, the data center owners would have to be ok with their centers physically located in residential areas, directly piping the heat to homeowners over relatively short runs.

As I recall, 40% of a data center's electricity need comes from the need to cool the equipment. Despite that, there likely isn't enough heat to warm more than a few homes this way, while electricity use for cooling and transporting said heat would still remain high.

It's even worse. You can actually design a "superinsulated" house with enough insulation so that it is heated merely by the body temperature of occupants and waste heat of appliances. Add an efficient heat exchanger to heat or cool the outside air you circulate in and there is no need to reuse data center air, which is full of outgassed stuff anyway. Ars had a nice article on this a few years back. Too far to easily Google unfortunately.

 

[Entkoffeiniert]

It's worth pointing out that the Orkney Islands have two 33kV sub sea power connections to the mainland with the capacity of 40 MW. So the Orkney Islands is connected to the UK's power grid and ultimately to continental Europe. There is also a failsafe diesel generator.

So this would mean that 100% of their electricity isn't generated on the island, but that they generate as much as they use.

Might be seen as quibbling, but it is *the* main point of contention around going 100% renewable.

 

[Entkoffeiniert]

With enough data centres like these, all the excess heat will go straight into the sea, further contributing to climate change. The only place these data centres wouldn’t contribute further to climate change is if they were buried deep down in earth’s crust but then cooling would be an issue.

I think I prefer the idea of city center data centres where the excess heat is put to use for central heating systems

Thermodynamics fail.

Global warming is not caused by humans creating heat. There is nothing humans could do that would produce even the tiniest fraction of the thermal energy that reaches the Earth from the sun.

Global warming is caused by GHG which increase the percentage of the thermal energy from the sun which is trapped. The sun is so off the charts powerful compared to any heat sources produced by humans that even the tiniest increase in the greenhouse effect can raise global temperatures.

To wit: the Sun bombards the Earth with more energy in *ONE HOUR* than the entire human race uses in a year. One. Hour.

And there are fools who still thing solar power isn't viable....

Rain is a viable source of water too, but you don't get to chose when you get it

 

[Ananke]

Microsoft retrieved a 40-foot-long, 12-rack, self-contained underwater data center from its seafloor home offshore from the Orkney Islands earlier this summer.

...exploring the concept of deploying sealed server pods just offshore major population centers as a replacement for traditional onshore data centers.

This must be using some new and exciting definition of "major population centres" of which I was previously unaware

I don't blame them for chosing the Orkneys as a test bed, though. Company-paid trips to go off birdwatching in the northern isles would be just lovely!

 

[AdrianS]

Another factor is the potential for littering the oceans with vaguely toxic electronics debris. Say someone puts one of these pods down there, and then goes bankrupt. Whose responsibility is it now to remove it? Eventually (maybe decades, but eventually) the pressure hull will rust through.

I'm not saying this is a showstopper, but it's something that needs to be considered. Companies will swear up and down that they'll take responsibility for long-term cleanup, but we all know how often that's a lie.

The Germans sunk 21 million tons of shipping in the Atlantic. In the size of the oceans you could dump ever single server made in each year and still not equal the monthly tonnage sunks by the Germans.

For fairness, add in a fair few million tons of shipping on the bottom of the pacific, put there by the US (when they finally worked out how to make torpedoes that worked)

 

[jlredford]

Another factor is the potential for littering the oceans with vaguely toxic electronics debris. Say someone puts one of these pods down there, and then goes bankrupt. Whose responsibility is it now to remove it? Eventually (maybe decades, but eventually) the pressure hull will rust through.

I'm not saying this is a showstopper, but it's something that needs to be considered. Companies will swear up and down that they'll take responsibility for long-term cleanup, but we all know how often that's a lie.

The ocean floor already has 5000 years' worth of sunken ships on it. The more modern shipwrecks are now getting salvaged just for their steel. Battleships from WW II are disappearing, much to the dismay of the families of the sailors who went down on them. The underwater server racks would also be salvageable, and might even be worth stealing just for the gold in the chip bond wires.

 

[thedannywahl]

I, for one, am extremely excited about the convergence of physical and digital sea-based piracy.

 

[Kilbane]

It's a neat idea and has merit. One thing I didn't see it in the comments or article, and maybe it's a dumb question, but did they actually have this pod as an active node? Or were the servers just sitting there bouncing a screen saver around on a non-existent monitor? What percentage of usage did they get? Failure rates without that kind of information can be really misleading.

 

[pusher robot]

Another factor is the potential for littering the oceans with vaguely toxic electronics debris. Say someone puts one of these pods down there, and then goes bankrupt. Whose responsibility is it now to remove it? Eventually (maybe decades, but eventually) the pressure hull will rust through.

I'm not saying this is a showstopper, but it's something that needs to be considered. Companies will swear up and down that they'll take responsibility for long-term cleanup, but we all know how often that's a lie.

The Germans sunk 21 million tons of shipping in the Atlantic. In the size of the oceans you could dump ever single server made in each year and still not equal the monthly tonnage sunks by the Germans.

For fairness, add in a fair few million tons of shipping on the bottom of the pacific, put there by the US (when they finally worked out how to make torpedoes that worked)

Not to mention Million Dollar Point.

 

[PrionDX]

So Skynet or Shodan or wheover has already tricked us into testing computing nodes in remote locations and with no habitable environment within.

 

[jlredford]

Chips last a lot longer if they're kept cool - all the failure mechanisms go up non-linearly with temperature. Running cooler might also be contributing to the better reliability here. If these pods could be placed in even deeper water, they'll be cooler still.

But I wonder about the effect on sea life. You don't want to disrupt the local ecology and probably the fishing by putting these warm pods all over the sea floor. If this idea is as attractive as it sounds, there might be thousands of them off the coasts of Seattle, New York, and Shanghai. Maybe that just means they have to be spaced out more.

As chips get more efficient, they produce less heat per unit of computational work. But that also means that more of them can be packed into these pods. Maybe if Moore's Law beats the increasing demands on servers, the pods can run cooler, but it could just as well go the other way.

 

[Caelestian]

Undersea pods that could be placed near major undersea cables - Sounds like an NSA dream come true.

 

[ColdWetDog]

Chips last a lot longer if they're kept cool - all the failure mechanisms go up non-linearly with temperature. Running cooler might also be contributing to the better reliability here. If these pods could be placed in even deeper water, they'll be cooler still.

But I wonder about the effect on sea life. You don't want to disrupt the local ecology and probably the fishing by putting these warm pods all over the sea floor. If this idea is as attractive as it sounds, there might be thousands of them off the coasts of Seattle, New York, and Shanghai. Maybe that just means they have to be spaced out more.

As chips get more efficient, they produce less heat per unit of computational work. But that also means that more of them can be packed into these pods. Maybe if Moore's Law beats the increasing demands on servers, the pods can run cooler, but it could just as well go the other way.

Read about Hydrothermal vents. If anything, the pods will help.

TL;DR - heat is good down in the cold and dark. Also read the above posts about how absolutely insignificant this heat source would be. Not even a rounding error.

 

[Skyfire77]

So Skynet or Shodan or wheover has already tricked us into testing computing nodes in remote locations and with no habitable environment within.

♫This was a triumph
I'm making a note here: "HUGE SUCCESS"
It's hard to overstate
My satisfaction♫

 

[SixDegrees]

Undersea pods that could be placed near major undersea cables - Sounds like an NSA dream come true.

Meh. The Navy has had that capability for many decades. Commercial deployment doesn't bring anything new to the table.

 

[SixDegrees]

Chips last a lot longer if they're kept cool - all the failure mechanisms go up non-linearly with temperature. Running cooler might also be contributing to the better reliability here. If these pods could be placed in even deeper water, they'll be cooler still.

But I wonder about the effect on sea life. You don't want to disrupt the local ecology and probably the fishing by putting these warm pods all over the sea floor. If this idea is as attractive as it sounds, there might be thousands of them off the coasts of Seattle, New York, and Shanghai. Maybe that just means they have to be spaced out more.

As chips get more efficient, they produce less heat per unit of computational work. But that also means that more of them can be packed into these pods. Maybe if Moore's Law beats the increasing demands on servers, the pods can run cooler, but it could just as well go the other way.

Read about Hydrothermal vents. If anything, the pods will help.

TL;DR - heat is good down in the cold and dark. Also read the above posts about how absolutely insignificant this heat source would be. Not even a rounding error.

After a couple of years, I'm actually surprised at how little marine life took up residence on the container. I would have expected more encrustation.