US weather forecasts get a software update

It’s been almost 40 years since the model got a new core.

Read the whole story

 

[whiteknave]

Wow. It's amazing how the observed snowfall was only within the US borders. Further prrof that Canada is manipulating the US weather.*


*'tis a joke

 

[dramamoose]

It's odd to me that there are two separate systems being used by the US and the Europeans. I know that both models are used by meteorologists to provide the actual figures, but I'm surprised that there isn't some effort to develop joint models. Obviously a model that works well for the US may not work well for Europe or Australia, much less Russia/China/India/Japan, but this seems like the sort of thing that a combined international effort would be useful for, particularly considering that the data is all released publicly anyway.

I'm also curious how the model itself works in terms of continuous self-improvement. Weather modeling seems like the perfect thing to target some machine learning towards, considering that you have a massive number of incredibly accurate data points.

 

[jandrese]

Interesting that the examples were of events where more precipitation fell than was expected under the old model. Most of the time in my experience the error is in the other direction. The weather forecast calls for rain that never appears. I see this all the time where the 3 day forecast calls for a 60% chance of 0.5" of rainfall on a specific day, but as that day approaches those rainfall estimates evaporate until you're left with a dry but maybe overcast day.

It's much less common for it to be wrong in the opposite direction, calling for a clear day or a low chance of rain when the truth is a heavy rainstorm.

I do pay attention to this because I plan out my bike commutes to work based on the weather report, and have skipped many days due to rain forecasts that never panned out, but only been unexpectedly rained on once or twice.

This could be an artifact of the weather patterns where I live. There are mountains to my west that do a lot to shape the weather patterns and make life difficult for forecasters. Lots of chaotic swirling in the middle layers to mess up nice pretty mathematical models.

 

[muckz]

Does the new software come with more code red?

edit: link to story https://www.cnn.com/2019/06/10/media/we ... index.html

 

[arcite]

More cores, can’t argue with that.

 

[Richard Weiss]

"All models are wrong, but some are useful." -- George E.P. Box (I think)

And some models are more useful than others. I suspect this one will be refined and improved over time.

 

[Bad Monkey!]

It's odd to me that there are two separate systems being used by the US and the Europeans. I know that both models are used by meteorologists to provide the actual figures, but I'm surprised that there isn't some effort to develop joint models. Obviously a model that works well for the US may not work well for Europe or Australia, much less Russia/China/India/Japan, but this seems like the sort of thing that a combined international effort would be useful for, particularly considering that the data is all released publicly anyway.

I'm also curious how the model itself works in terms of continuous self-improvement. Weather modeling seems like the perfect thing to target some machine learning towards, considering that you have a massive number of incredibly accurate data points.

Not all the data is released publicly. Access to the most accurate ECMWF data requires a subscription. The US GFS data has always been public, well, at least it will be unless Accuweather gets their way.

 

[Bonobob]

Wow. It's amazing how the observed snowfall was only within the US borders. Further prrof that Canada is manipulating the US weather.*


*'tis a joke

Clearly, the country to our north is so poor that they can't afford weather tracking stations.

 

[terrydactyl]

Keep in mind that former AccuWeather CEO Barry Myers is nominated to head NOAA. He's long been opposed to NWS releasing these weather forecasts directly to the public.

 

[Veritas super omens]

Wow. It's amazing how the observed snowfall was only within the US borders. Further prrof that Canada is manipulating the US weather.*


*'tis a joke

Clearly, the country to our north is so poor that they can't afford weather tracking stations.

Weather tracking stations? In a fictional country? /s

 

[eathdemon]

dumb question how scalable are those models? would it be possible in theory to add time on aws/google cloud/azure to make it more proditive, or is there a hard limit to its scaling?

 

[DanNeely]

dumb question how scalable are those models? would it be possible in theory to add time on aws/google cloud/azure to make it more proditive, or is there a hard limit to its scaling?

AFAIK the models can only run effectively at scale on super computers because they need to communicate large amounts of data between adjacent cells of atmosphere and ethernet is too slow and high latency of an interconnect.

 

[terrydactyl]

Wow. It's amazing how the observed snowfall was only within the US borders. Further prrof that Canada is manipulating the US weather.*


*'tis a joke

Clearly, the country to our north is so poor that they can't afford weather tracking stations.

Well there appears to be no weather in the country to our south either.
https://graphical.weather.gov/sectors/p ... ement=MaxT

 

[yoshi12]

dumb question how scalable are those models? would it be possible in theory to add time on aws/google cloud/azure to make it more proditive, or is there a hard limit to its scaling?

There is a limit. At its heart weather prediction models are applied linear algebra at a massive scale, so the communication latency between nodes is actually a key contributor to performance. Cloud computing is great at many things, but low-latency internode communication isn't one of them.

 

[john-whorfin]

Will the new models make any difference once 5G interference trashes the satellite data?

 

[Insightfill]

Interesting that the examples were of events where more precipitation fell than was expected under the old model. Most of the time in my experience the error is in the other direction. The weather forecast calls for rain that never appears.

Nate Silver's "Signal and the Noise" discusses this at some depth. There's a bias for local forecasts/forecasters to over-predict rain so as to avoid blame for predicting clear skies and then getting peoples' recreational plans ruined. A forecast of 50% chance of rain actually correlates to 30%, while a "100% chance of rain" actually pans out to about 70%.

 

[duncan99]

...

Not all the data is released publicly. Access to the most accurate ECMWF data requires a subscription. The US GFS data has always been public, well, at least it will be unless Accuweather gets their way.

A lot of public data can be found here:

NWS Model Analyses and Guidance pages

Edit: phrasing

 

[Cliffy]

Clearly, the country to our north is so poor that they can't afford weather tracking stations.[/quote]Weather tracking stations? In a fictional country? /s[/quote]


So...I guess I really do live in the "middle of nowhere"...

 

[graylshaped]

Wow. It's amazing how the observed snowfall was only within the US borders. Further prrof that Canada is manipulating the US weather.*


*'tis a joke

Mythical results are discarded. It was part of the upgrade.

 

[Fatesrider]

It's odd to me that there are two separate systems being used by the US and the Europeans. I know that both models are used by meteorologists to provide the actual figures, but I'm surprised that there isn't some effort to develop joint models. Obviously a model that works well for the US may not work well for Europe or Australia, much less Russia/China/India/Japan, but this seems like the sort of thing that a combined international effort would be useful for, particularly considering that the data is all released publicly anyway.

I'm also curious how the model itself works in terms of continuous self-improvement. Weather modeling seems like the perfect thing to target some machine learning towards, considering that you have a massive number of incredibly accurate data points.

I MIGHT be mistaken here, but one could reasonably argue the reason we don't use the European model is that the variables are a lot different with respect to what aspects of atmosphere impact where.

Contrasting the European model, you'd have a lot more warm water coming in from the northwest, where in the U.S. that water in the northwest is quite cold (using west coasts of each place, obviously).

The thing that I noticed is that in the U.S. model, the temperatures are biased low, which might cause major issues in forecasting heat waves or other such things.

I don't pretend to know how these things are put together, but at the same time, I can't imagine there's a one-size-fits-all solution to the core models. given how oceanic currents and geography can cause radically different weather, even if you have the same flows (such as in Western Europe and the Western U.S., I'd expect that any overlap in similarity would be on the periphery of the zones, where weather moves away, or comes in.

Europe has a vested interest in tracking the storms that develop in the Atlantic, since they follow the trade winds over to them and potentially deliver another blow. Since those storms tend to give a LOT more warning than they do on the eastern Atlantic, their models may be superior since they'd, like us, would want to know as early as possible what risks are coming and preparations that need to be made. I heard that the European models are better at figuring out where hurricanes will go and their intensity (at least before this change - which has yet to see how well it does there), and that need to track early may be why they're more accurate.

Predicting the weather for one's NEIGHBORHOOD is often inaccurate. So I have to think that if they're going to model things, it'll be on a wider scale, and fairly specific to the unique weather factors of the regions being modeled.

 

[Davide_sd]

It's odd to me that there are two separate systems being used by the US and the Europeans. I know that both models are used by meteorologists to provide the actual figures, but I'm surprised that there isn't some effort to develop joint models. Obviously a model that works well for the US may not work well for Europe or Australia, much less Russia/China/India/Japan, but this seems like the sort of thing that a combined international effort would be useful for, particularly considering that the data is all released publicly anyway.

I'm also curious how the model itself works in terms of continuous self-improvement. Weather modeling seems like the perfect thing to target some machine learning towards, considering that you have a massive number of incredibly accurate data points.

Weather prediction is still a very hard work today, both in terms of modeling and computational effort. I believe that the use of different models for different parts of the world has its advantages: this gives research groups wide "study area", where they can develop and test new models (or sub-models) without being too constrained to what other research groups are doing.
It takes years to develop something reliable, so you wouldn't like to invest everything you have over a single model, which may one day become too limited.

Regarding machine learning, I believe it is already employed in data acquisition. Weather stations may not be very reliable, and some forms of blending between data extrapolation and data prediction is needed. Also, machine learning works well at finding patterns, maybe it could be used to improve weather prediction on local areas without having to rely on heavily physical computations.

 

[EagerEyes]

I MIGHT be mistaken here, but one could reasonably argue the reason we don't use the European model is that the variables are a lot different with respect to what aspects of atmosphere impact where.

AFAIK they all model the entire planet. Cliff Mass often refers to the European Model to get a second opinion in his postings about PNW (or other U.S.) weather. It seems a bit myopic to model only a part of what's obviously a global system.

 

[awshucks]

It's hot. Damn hot!

 

[DanNeely]

I MIGHT be mistaken here, but one could reasonably argue the reason we don't use the European model is that the variables are a lot different with respect to what aspects of atmosphere impact where.

AFAIK they all model the entire planet. Cliff Mass often refers to the European Model to get a second opinion in his postings about PNW (or other U.S.) weather. It seems a bit myopic to model only a part of what's obviously a global system.

They do both global and regional models. High detail forecasts need regional models that can be ran at higher fidelity (smaller cell sizes and shorter time steps) than a global model. However you need to provide input to a regional model at its edges (weather moving into the area); so they run global models too. AFAIK it's not just a 2-step situation either, with shortest term hugest fidelity models used to generate storm warnings/hourly forecasts, and intermediate ones used to do weekly/etc ones.

 

[graylshaped]

Interesting that the examples were of events where more precipitation fell than was expected under the old model. Most of the time in my experience the error is in the other direction. The weather forecast calls for rain that never appears.

Nate Silver's "Signal and the Noise" discusses this at some depth. There's a bias for local forecasts/forecasters to over-predict rain so as to avoid blame for predicting clear skies and then getting peoples' recreational plans ruined. A forecast of 50% chance of rain actually correlates to 30%, while a "100% chance of rain" actually pans out to about 70%.

I've had that one on my kindle for years. You've tempted me to make sure it's downloaded for an overseas plane ride in the near future.

 

[adespoton]

Wow. It's amazing how the observed snowfall was only within the US borders. Further prrof that Canada is manipulating the US weather.*


*'tis a joke

This is due to the fact that on the other side of the border, the permanent snow and ice is so heavy as to not be plottable on that map.*


*'tis also a joke

 

[Navalia Vigilate]

Interesting that the examples were of events where more precipitation fell than was expected under the old model. Most of the time in my experience the error is in the other direction. The weather forecast calls for rain that never appears. I see this all the time where the 3 day forecast calls for a 60% chance of 0.5" of rainfall on a specific day, but as that day approaches those rainfall estimates evaporate until you're left with a dry but maybe overcast day.

It's much less common for it to be wrong in the opposite direction, calling for a clear day or a low chance of rain when the truth is a heavy rainstorm.

I do pay attention to this because I plan out my bike commutes to work based on the weather report, and have skipped many days due to rain forecasts that never panned out, but only been unexpectedly rained on once or twice.

This could be an artifact of the weather patterns where I live. There are mountains to my west that do a lot to shape the weather patterns and make life difficult for forecasters. Lots of chaotic swirling in the middle layers to mess up nice pretty mathematical models.

What happens when a forecast over predicts adverse weather?

What happens when a forecast under predicts adverse weather?

The model looks a lot better and I've been following this on weather forums. There are imperfections but the improved reliability of the end forecast versus the observed weather is much sharper and more local/regional events are in better focus.

No butterfly in Japan observations, but weather events from localized low pressure ridges that pick up water from passing a moderately sized body of water, that become a small but very violent storm cell, are much more reliable. This saves lives and as someone that spends a percentage of every week sailboat racing, I appreciate these improvements by NOAA.

 

[Northbynorth]

It's odd to me that there are two separate systems being used by the US and the Europeans. I know that both models are used by meteorologists to provide the actual figures, but I'm surprised that there isn't some effort to develop joint models. Obviously a model that works well for the US may not work well for Europe or Australia, much less Russia/China/India/Japan, but this seems like the sort of thing that a combined international effort would be useful for, particularly considering that the data is all released publicly anyway.

There are actual benefits having several different model systems. Even if each system may run several (10-50) slightly different models to get an estimate of the uncertainty of the forecast, comparing different systems adds to the probability analysis. I guess that is one reason why the National Hurrican Center also uses ECMWFs model in their forecast .

I'm also curious how the model itself works in terms of continuous self-improvement. Weather modeling seems like the perfect thing to target some machine learning towards, considering that you have a massive number of incredibly accurate data points.

New model releases in recent decades, like this one, have usually been a somewhat mixed bag. Many great improvements, but also some areas where the previous release was better. It has usually taken the model developers a couple of years to iron out these kinks. That fact that a new model has more realistic physics and higher resolution could actually make it more unstable and prone to big errors than a well tuned coarser and less realistic model.

Using machine learning in weather models, or on the output from them, have a huge potential. But as far as I know, it has until now only been used in certain parts of the model and post processing of the output. But there are a lot of development going on all over the world so we will certainly see a lot of results the next decade.

(edit: spelling)

 

[dorkbert]

Wow. It's amazing how the observed snowfall was only within the US borders. Further prrof that Canada is manipulating the US weather.*


*'tis a joke

Reaction from the White House was swift: "It's a conspiracy between China and Canada to destroy the United States! The Chinese spreads fake climate change and Canada manipulates the weather."

 

[erikb]

It's odd to me that there are two separate systems being used by the US and the Europeans.

I get your point that we could theoretically have a more accurate system if we pooled resources & talent, but I don't think it's such a bad thing that there exists more than one system, from more than one agency/development group. I'm thinking along the lines of "competition is good" in that each system might show more accuracy in some particular areas (either geographic areas or meteorological phenomenon), and contrasting that system with the others could help figure out why. Or at least people would know to trust one system more for certain forecasts. As long as there's at least some level of open exchange about what each system is doing*, I'd rather have that scenario than one uber-system designed by a larger design committee. After all, that's usually a recipe for projects that come in late, over budget, and with too many compromises to satisfy everyone.


* Given that science drives the design, I expect this is the case, at least among systems made by publicly-funded agencies. I know there are privately-funded companies with their own private forecast models, that may or may not be willing to share improvement ideas and experience. (I think AccuWeather has one, and there are other less-well known companies that do also.)

 

[nosmadar2016]

Interesting that the examples were of events where more precipitation fell than was expected under the old model. Most of the time in my experience the error is in the other direction. The weather forecast calls for rain that never appears. I see this all the time where the 3 day forecast calls for a 60% chance of 0.5" of rainfall on a specific day, but as that day approaches those rainfall estimates evaporate until you're left with a dry but maybe overcast day.

It's much less common for it to be wrong in the opposite direction, calling for a clear day or a low chance of rain when the truth is a heavy rainstorm.

I do pay attention to this because I plan out my bike commutes to work based on the weather report, and have skipped many days due to rain forecasts that never panned out, but only been unexpectedly rained on once or twice.

This could be an artifact of the weather patterns where I live. There are mountains to my west that do a lot to shape the weather patterns and make life difficult for forecasters. Lots of chaotic swirling in the middle layers to mess up nice pretty mathematical models.

Mostly the latter. I live near one of the Great Lakes and that, plus a large urban heat island (Gave it away, didn't I) contribute to some really 'interesting' weather forecasts. There are some days were they just say: "It's going to depend, check back with us during the next N hours". And days where we are overcast and it is raining but rain does not reach the ground, or it rains on some other parts of the area but my particular suburb is skipped.

 

[jt122333221]

dumb question how scalable are those models? would it be possible in theory to add time on aws/google cloud/azure to make it more proditive, or is there a hard limit to its scaling?

AFAIK the models can only run effectively at scale on super computers because they need to communicate large amounts of data between adjacent cells of atmosphere and ethernet is too slow and high latency of an interconnect.

So, I can't play around with this on my chromebook?

/s

Seriously though, this area has a lot of opportunity for neural nets/task-driven AIs to improve how well weather is forecasted. I'd be awesome to know how next week Thursday is going to be and have a 99% certainty that it's an accurate forecast.

 

[Tofystedeth]

The upgrade brings us a new “Finite-Volume Cubed-Sphere” (or FV3) dynamical core

Huh, who would have thought they'd solve how to cube the sphere before squaring the circle.

 

[ScottJohnson]

The upgrade brings us a new “Finite-Volume Cubed-Sphere” (or FV3) dynamical core

Huh, who would have thought they'd solve how to cube the sphere before squaring the circle.

I guess you're not a professional wrestling fan.

 

[erikb]

dumb question how scalable are those models? would it be possible in theory to add time on aws/google cloud/azure to make it more proditive, or is there a hard limit to its scaling?

There is a limit. At its heart weather prediction models are applied linear algebra at a massive scale, so the communication latency between nodes is actually a key contributor to performance. Cloud computing is great at many things, but low-latency internode communication isn't one of them.

Right. It may be a misconception, but cloud computing centers != supercomputers. They're very different animals. The term "supercomputer" often gets thrown around casually, but in a true-er sense, modern supercomputers are set apart from other types of computers not because they have a lot of fast CPUs, but because they have very low-latency, often specialized, links between those CPUs (and memory). Cloud computing centers may have lots of fast CPUs, but they don't have the fast, low-latency links that supercomputers do. They are best suited for workloads can scale up in somewhat smaller, but yet somewhat independent pieces. Having lots of separate nodes serving lots of different clients over the internet is just this sort of thing, as one example.

 

[Snark218]

This could be an artifact of the weather patterns where I live. There are mountains to my west that do a lot to shape the weather patterns and make life difficult for forecasters. Lots of chaotic swirling in the middle layers to mess up nice pretty mathematical models.

I suspect we live in the same reason, and yeah...nothing quite like a 14,000 foot mountain on the outskirts of town to ruin a good forecast. They do a great job with the broad outlines, but my house (for example) is in a weird little microclimate that gets a few drops while downtown - walking distance away! - is getting righteously shithammered by a giant thunderstorm. But if it comes from the other direction? Strap down the dog, honey.

 

[Tipped]

It's odd to me that there are two separate systems being used by the US and the Europeans. I know that both models are used by meteorologists to provide the actual figures, but I'm surprised that there isn't some effort to develop joint models. Obviously a model that works well for the US may not work well for Europe or Australia, much less Russia/China/India/Japan, but this seems like the sort of thing that a combined international effort would be useful for, particularly considering that the data is all released publicly anyway.

I'm also curious how the model itself works in terms of continuous self-improvement. Weather modeling seems like the perfect thing to target some machine learning towards, considering that you have a massive number of incredibly accurate data points.

I MIGHT be mistaken here, but one could reasonably argue the reason we don't use the European model is that the variables are a lot different with respect to what aspects of atmosphere impact where.

Contrasting the European model, you'd have a lot more warm water coming in from the northwest, where in the U.S. that water in the northwest is quite cold (using west coasts of each place, obviously).

The thing that I noticed is that in the U.S. model, the temperatures are biased low, which might cause major issues in forecasting heat waves or other such things.

I don't pretend to know how these things are put together, but at the same time, I can't imagine there's a one-size-fits-all solution to the core models. given how oceanic currents and geography can cause radically different weather, even if you have the same flows (such as in Western Europe and the Western U.S., I'd expect that any overlap in similarity would be on the periphery of the zones, where weather moves away, or comes in.

That's related, but not quite right. All of these models are based on solving essentially the same equations (it's just physics after all), they just do so in different ways. eg, FV-3 uses a finite volume formulation, whereas IFS (ECMWF's atmosphere model) uses a finite element formulation (details here: https://www.ecmwf.int/en/forecasts/docu ... umentation).

Differences in the software implementations (and their respective bugs, though some of those are shared...) lead to some differences in how the models behave, but that's not that big of a deal since the models have parameters which need to be tuned for them to produce reasonable results for the specific regions. This was done for FV-3, and could almost certainly (I don't know how ECMWF's model does things, but I suspect its not that different from NOAA) have been done for IFS.

BTW, another model considered for the replacement was NCAR's MPAS model; its also a finite volume model, similar to the WRF model: https://www.climatescience.org.au/sites ... 5B1%5D.pdf

Europe has a vested interest in tracking the storms that develop in the Atlantic, since they follow the trade winds over to them and potentially deliver another blow. Since those storms tend to give a LOT more warning than they do on the eastern Atlantic, their models may be superior since they'd, like us, would want to know as early as possible what risks are coming and preparations that need to be made. I heard that the European models are better at figuring out where hurricanes will go and their intensity (at least before this change - which has yet to see how well it does there), and that need to track early may be why they're more accurate.

Predicting the weather for one's NEIGHBORHOOD is often inaccurate. So I have to think that if they're going to model things, it'll be on a wider scale, and fairly specific to the unique weather factors of the regions being modeled.

Yes, they do seem to be aiming for better accuracy around the globe, while NOAA aims for regional accuracy, see https://blogs.agu.org/wildwildscience/2 ... sy-likely/

 

[gallahad]

It's odd to me that there are two separate systems being used by the US and the Europeans. I know that both models are used by meteorologists to provide the actual figures, but I'm surprised that there isn't some effort to develop joint models. Obviously a model that works well for the US may not work well for Europe or Australia, much less Russia/China/India/Japan, but this seems like the sort of thing that a combined international effort would be useful for, particularly considering that the data is all released publicly anyway.

I'm also curious how the model itself works in terms of continuous self-improvement. Weather modeling seems like the perfect thing to target some machine learning towards, considering that you have a massive number of incredibly accurate data points.

Among other reasons people have given, having multiple models that do things their own way means that you can cross-reference them. When you get The Forecast from a model, what you're getting is an melding of some dozens of runs with slightly different inputs; this helps accuracy, but since it's all the same model the result is still susceptible to any biases or flaws in the core model. But with multiple independent models, you now have independent data sets that you can use to increase the accuracy of your forecast. If you read the forecast discussion products from the National Hurricane Center, you'll notice that their forecast comes from analyzing three or four different models, and they'll talk about how they weight the different models to get their forecast.