The European forecast model already kicking America’s butt just improved

Better resolution will allow the world's best model to improve local forecasts.

Read the whole story

 

[bthylafh]

Thanks to once having used a certain Chrome extension, I wanted to read the headline as "kicking America's cloud" and it still kind of makes sense that way.

 

[Cumulonimbus Maximus]

My understanding is that American models are getting upgrades computationally speaking, and models like the NAM and GFS are seeing the benefits, but the aging satellite hardware that desperately needs to be upgraded or replaced will continue to hold back bigger improvements.

 

[Faanchou]

<old-man-whine>
When I was a child, it was enough to look at the calendar and you knew what the weather would be like today. Now, you need a stop watch.
</old-man-whine>

On a more serious note, it's nice that the models keep up and keep improving. CPU cycles have never been this cheap.

 

[Deleted member 46272]

Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

 

[grumpy2]

cslawren[/url]":2fdm5u95]My understanding is that American models are getting upgrades computationally speaking, and models like the NAM and GFS are seeing the benefits, but the aging satellite hardware that desperately needs to be upgraded or replaced will continue to hold back bigger improvements.

Isn't the satellite data shared between the different models, though? If the European model is more accurate (which it is), and it doesn't have better, more accurate satellite data (does it? I don't see why it would), then you can't really pin the GFS shortcomings on aging satellites, can you?

 

[metaldark]

Xelas[/url]":1tsd1tuw]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

That's kinda, literally, what the article is about....

 

[KiwiPhred]

Xelas[/url]":ttua2bup]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

It's because America is not great anymore but don't worry Donald will fix it.

 

[Aykernar]

Xelas[/url]":28h8oeso]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Looking at the Top-500 list linked in the article, it appears that the NOAA (the super-agency of the US National Weather Service) actually has access to more supercomputing power than its European counterpart, whose systems seem to be in the low end for major meteorology services in general. As the saying goes, it's not the number of cycles you have that matters, it's how you use them.

 

[Danrarbc]

Can't help but think we're falling behind partially because of climate change denial.

 

[bdp]

Maybe put the word "weather" in the article title or subheading? I had to click through to find out what was being forecast.

 

[Faanchou]

Aykernar[/url]":1aaqlhq3]

Xelas[/url]":1aaqlhq3]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Looking at the Top-500 list linked in the article, it appears that the NOAA (the super-agency of the US National Weather Service) actually has access to more supercomputing power than its European counterpart, whose systems seem to be in the low end for major meteorology services in general. As the saying goes, it's not the number of cycles you have that matters, it's how you use them.

Yeah, the NOAA supercomputers are currently busy indexing emails for Lamar Smith.

 

[Gasaraki]

The US is upgrading their system too. I think it comes online at the end of this year. I'll will look and edit this post.

OK here it is, I guess it came online in January.

http://www.noaanews.noaa.gov/stories201 ... puter.html

 

[Aykernar]

Faanchou[/url]":q4m34lrj]

Aykernar[/url]":q4m34lrj]

Xelas[/url]":q4m34lrj]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Looking at the Top-500 list linked in the article, it appears that the NOAA (the super-agency of the US National Weather Service) actually has access to more supercomputing power than its European counterpart, whose systems seem to be in the low end for major meteorology services in general. As the saying goes, it's not the number of cycles you have that matters, it's how you use them.

Yeah, the NOAA supercomputers are currently busy indexing emails for Lamar Smith.

I'll admit that got a laugh out of me. But it ended quickly when I realized it might actually be true.

 

[Callitrax]

Xelas[/url]":3dv1kzh5]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

The algorithm mostly.
They use a lot of NOAA/NASA generated data (Then make us pay for the output, which is BS)

 

[phizch]

More info about the new grid layout : Changes to ECMWF's grids in 2016 (pdf).

 

[dio82]

Callitrax[/url]":232khzaw]

Xelas[/url]":232khzaw]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

The algorithm mostly.
They use a lot of NOAA/NASA generated data (Then make us pay for the output, which is BS)

Coming from CFD, I will attest that the octahedral mesh will produce significantly better results compared to a square mesh of same spacing/size. So that alone accounts for a lot in the relative gap. Never mind an apparently much better Satellite data interpolation method (remember SiSo).

Even better would be actually an unstructured polyhedral mesh. Edit: link to polyhedral mesh example: https://i.ytimg.com/vi/79Au5vCTkes/maxresdefault.jpg

 

[EricBerger]

bdp[/url]":3z7ufbus]Maybe put the word "weather" in the article title or subheading? I had to click through to find out what was being forecast.

Good suggestion, thanks!

 

[JohnDeL]

How is "overall forecast accuracy" determined? What all is included and how is it weighted?

The way that the European and American GFS models parallel each other in their accuracy suggests that there is some factor that is left out of both of the models that might improve the accuracy (that's the "better physics" you mentioned). Any idea what that factor might be?

 

[EricBerger]

JohnDeL[/url]":76wd379u]How is "overall forecast accuracy" determined? What all is included and how is it weighted?

First of all, hello John!

Second of all, my layman's understanding is that the models are scored based on anomaly correlations at each grid point between observed and the simulated anomalies. Beyond that the mathematics quickly supersedes the four calculus classes that I took 20 years ago and have since forgotten.

 

[Deleted member 1]

Aykernar[/url]":18rcccev]

Xelas[/url]":18rcccev]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Looking at the Top-500 list linked in the article, it appears that the NOAA (the super-agency of the US National Weather Service) actually has access to more supercomputing power than its European counterpart, whose systems seem to be in the low end for major meteorology services in general

OK, this is getting depressing...
We spend more per capita on health care - and the Europeans beat us in most health stats.
We spend more per capita on education - and the Europeans beat us in math/science competitions
And now, this?

 

[Bernardo Verda]

ReadandShare[/url]":2e1vu1gx]

Aykernar[/url]":2e1vu1gx]

Xelas[/url]":2e1vu1gx]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Looking at the Top-500 list linked in the article, it appears that the NOAA (the super-agency of the US National Weather Service) actually has access to more supercomputing power than its European counterpart, whose systems seem to be in the low end for major meteorology services in general

OK, this is getting depressing...
We spend more per capita on health care - and the Europeans beat us in most health stats.
We spend more per capita on education - and the Europeans beat us in math/science competitions
And now, this?

I guess that just goes to show why "socialism" is so dangerous?

 

[alastairmayer]

OT, but I have a soft spot for ECMRWF ... they were my first software sale, back in the very early '80s. Nothing to do with weather forecasting per se, it was a text processing/formatting package for CDC mainframes. (Anyone remembering the funky character sets they used -- 6 bit or 6/12 bit -- will appreciate the challenge.)

Nice to see they've moved on from there...

 

[Justin Credible]

EricBerger[/url]":1bqv6k56]

JohnDeL[/url]":1bqv6k56]How is "overall forecast accuracy" determined? What all is included and how is it weighted?

First of all, hello John!

Second of all, my layman's understanding is that the models are scored based on anomaly correlations at each grid point between observed and the simulated anomalies. Beyond that the mathematics quickly supersedes the four calculus classes that I took 20 years ago and have since forgotten.

Am i missing something?

How does one get a bro hug "First of all, hello John!" from the story author?

Who are you JohnDeL?

Reveal yourself!

 

[mmtroute]

I suspect, the biggest improvement comes from optimizing the codes for parallel systems. The majority of the codes in this space are still very serial in nature. Vectorization has been difficult due to the math (as noted in the article, lots of dependency that requires unique matrix methods). Additionally scalability across nodes has been limited for reasons that include more than just processing power. Very few codes are compute bound anymore. Memory bound code and communication bound code have become the two driving factors in code modernization. More and more codes will begin to optimize for parallel processing as time goes by and as new fabric and memory technology becomes available.

Todays CPU's (single core / serial programming is dead) really require a parallel programming model. Rethinking the structure of the code, not just relying on the compiler to vectorize the code, which is only going to catch low hanging fruit of parallel potential.

GFS will soon have optimized codes that will bring parity to the US system...It's happening now...

Source: I have an interesting job.

edited: words and stuff

 

[mogbert]

Which weather forecasting sites use which model? How can I tell if the site I use is using the European model? OR is it that their model is used to predict their weather and our model is used to predict ours? The story seemed to indicate that their model was available for our side of the 'pond'.

 

[Morris von Habsburg]

ReadandShare[/url]":3kuogdxv]

Aykernar[/url]":3kuogdxv]

Xelas[/url]":3kuogdxv]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Looking at the Top-500 list linked in the article, it appears that the NOAA (the super-agency of the US National Weather Service) actually has access to more supercomputing power than its European counterpart, whose systems seem to be in the low end for major meteorology services in general

OK, this is getting depressing...
We spend more per capita on health care - and the Europeans beat us in most health stats.
We spend more per capita on education - and the Europeans beat us in math/science competitions
And now, this?

Because one needs to spend more if a cut of it needs to go towards profits?

 

[isparavanje]

Morris von Habsburg[/url]":1zrdjie7]

ReadandShare[/url]":1zrdjie7]

Aykernar[/url]":1zrdjie7]

Xelas[/url]":1zrdjie7]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Looking at the Top-500 list linked in the article, it appears that the NOAA (the super-agency of the US National Weather Service) actually has access to more supercomputing power than its European counterpart, whose systems seem to be in the low end for major meteorology services in general

OK, this is getting depressing...
We spend more per capita on health care - and the Europeans beat us in most health stats.
We spend more per capita on education - and the Europeans beat us in math/science competitions
And now, this?

Because one needs to spend more if a cut of it needs to go towards profits?

Well US natural sciences academia still seems to be leading somewhat, but the government is doing all they can to rectify that.

 

[VidasDuday]

Xelas[/url]":14mjrmcr]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Part of the problem is the resolution of the initial conditions is pretty sucky. Here in Houston our wx forecast is based on Corpus way to our SW and Lake Charles way to our E.

Pathetic.

 

[tuxRoller]

Morris von Habsburg[/url]":nook12l7]

ReadandShare[/url]":nook12l7]

Aykernar[/url]":nook12l7]

Xelas[/url]":nook12l7]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Looking at the Top-500 list linked in the article, it appears that the NOAA (the super-agency of the US National Weather Service) actually has access to more supercomputing power than its European counterpart, whose systems seem to be in the low end for major meteorology services in general

OK, this is getting depressing...
We spend more per capita on health care - and the Europeans beat us in most health stats.
We spend more per capita on education - and the Europeans beat us in math/science competitions
And now, this?

Because one needs to spend more if a cut of it needs to go towards profits?

Iirc, health care administrative overhead in the usa is roughly $200billion.

 

[The80sCalled]

Xelas[/url]":3f6vsbpp]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Here's an in-depth evaluation from 2014 by Cliff Mass, local-famous Seattle meteorologist:

http://cliffmass.blogspot.com/2014/02/w ... s-new.html

The gist is that the National Weather Service lacks compute power (apparently despite what the Top-500 list says about NOAA), and congress has been playing politics instead of fixing it in a timely manner.

Edit: if anyone is in the Seattle area, March 16th Cliff will be giving a talk at UW about the history of weather forecasting. He's a character; I'd totally go if I were in the area!

 

[mr_nobody]

Get with it NOAA -- Make America's Weather Great Again

;-)

 

[not trending]

Faanchou[/url]":gwkt4sjx]<old-man-whine>
When I was a child, it was enough to look at the calendar and you knew what the weather would be like today. Now, you need a stop watch.
</old-man-whine>

On a more serious note, it's nice that the models keep up and keep improving. CPU cycles have never been this cheap.

Yep, the almanac was good enough for us, and a look skyward. Them young whipper-snippers can't tell sun from rain without their iPhones.
</also-old-man-whine>

 

[kilkenny_cat]

KiwiPhred[/url]":2rdx7fqw]

Xelas[/url]":2rdx7fqw]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

It's because America is not great anymore but don't worry Donald will fix it.

America is just fine. It is the political machinations of idiots like Lamar Smith continuously endeavoring to cut off NOAA funding. At least the European meteorologists do not have to put up with that sort of political interference with science. Unfortunately Donald seems to be of the same mind as Lamar... might as well cut the funding of all far-reaching government-financed science endeavors, at least those that take longer to achieve results than the time taken to construct one of Donald's casino-hotels.

 

[snowman<ca>]

mmtroute[/url]":19f3f8u1]I suspect, the biggest improvement comes from optimizing the codes for parallel systems. The majority of the codes in this space are still very serial in nature. Vectorization has been difficult due to the math (as noted in the article, lots of dependency that requires unique matrix methods). Additionally scalability across nodes has been limited for reasons that include more than just processing power. Very few codes are compute bound anymore. Memory bound code and communication bound code have become the two driving factors in code modernization. More and more codes will begin to optimize for parallel processing as time goes by and as new fabric and memory technology becomes available.

Todays CPU's (single core / serial programming is dead) really require a parallel programming model. Rethinking the structure of the code, not just relying on the compiler to vectorize the code, which is only going to catch low hanging fruit of parallel potential.

GFS will soon have optimized codes that will bring parity to the US system...It's happening now...

Source: I have an interesting job.

edited: words and stuff

It is not like they have been running this on a single PC, the systems they use have been optimized around being massively parallel for a long time.

 

[betam4x]

KiwiPhred[/url]":1sl59rp3]

Xelas[/url]":1sl59rp3]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

It's because America is not great anymore but don't worry Donald will fix it.

He'd have to be president for that to happen. As long as I'm living/breathing...that's not going to happen.

 

[captainlightbeam]

Read more about this at the Cliff Mass Weather Blog: The National Weather Service Selects A New Global Weather Prediction Model: Will It Make the Right Choice?

 

[Potatoswatter]

KiwiPhred[/url]":27dduko2]It's because America is not great anymore but don't worry Donald will fix it.

The very first thing he'll do is to import European models. They'll be sent straight to the White House.

 

[franky54839]

President Trump will make USA weather service great again.

 

[AxMi-24]

isparavanje[/url]":36qcwa4z]

Morris von Habsburg[/url]":36qcwa4z]

ReadandShare[/url]":36qcwa4z]

Aykernar[/url]":36qcwa4z]

Xelas[/url]":36qcwa4z]Why is the American GFS model consistently behind? Is it just the algorithm, or raw computational power (resources), source data (satellites, sensors), etc?

Looking at the Top-500 list linked in the article, it appears that the NOAA (the super-agency of the US National Weather Service) actually has access to more supercomputing power than its European counterpart, whose systems seem to be in the low end for major meteorology services in general

OK, this is getting depressing...
We spend more per capita on health care - and the Europeans beat us in most health stats.
We spend more per capita on education - and the Europeans beat us in math/science competitions
And now, this?

Because one needs to spend more if a cut of it needs to go towards profits?

Well US natural sciences academia still seems to be leading somewhat, but the government is doing all they can to rectify that.

That part should be safe as our politicians (EU) seem to be utterly unable to comprehend that science can't be run as a business where you buy results in advance and that nothing new comes out of that kind of predicted science. Not to mention the new fad (at least in Germany) where you can get 20 M€ for a building and equipment but not half a cent to actually hire people to use the said building and equipment...