Google researchers presented a short period of weather forecasting future use of machine learning methods. This method, although at an early stage of development, but the results have been better than the traditional model.
Foreword
Weather or light or powerhouse will always affect people's daily lives, and the accuracy of weather forecasting will greatly affect people's way of coping weather. The weather forecast can tell whether people should choose a different route to work, whether to reschedule plans for the weekend picnic, whether because of a storm struck and evacuation shelter. But for local storm or thunderstorms hourly time scale evolution of weather events, make accurate predictions is challenging.
In the paper "Machine Learning for Precipitation Nowcasting from Radar Images", the researchers on short-term rainfall prediction problem raised new research based on machine learning model The model for the future in a short time weather be highly localized " non-physical "forecast.
Machine learning is a very significant advantage is that if the offer has been trained model, then calculate the cost of inference process will be very small. This makes the prediction data after the input is almost in real time, and as a result the original having a high resolution. This focused on short-term 0-6 hours of precipitation method may generate a prediction result of the prediction in the case where the resolution of 1km total delay of only 5-10 minutes, which also includes the data acquisition delay.
Although this method is still in the early stages of development, but has been better than the traditional model.
Beyond the traditional weather forecasting methods
Meteorological agencies around the world have a lot of monitoring facilities, for example, can be measured in real-time Doppler radar rainfall; weather satellites can provide multispectral imaging; ground weather station can directly measure wind speed, wind direction and precipitation and so on. The chart below compares the continental United States indicate precipitation cloud imaging false color imaging radar and geostationary satellites provided to explain the importance of weather information from multiple sources. There is the presence of precipitation and cloud related, but not perfectly correlated, so simply infer rainfall situation is very challenging from satellite remote sensing images.
Top: the image display position of the observed cloud geostationary satellite. Bottom: an image display Doppler radar stations observed the precipitation position.
Unfortunately, these methods are not measured on a global scale are applicable. For example, most of the radar data from the ground stations, which is usually not feasible in the ocean. In addition, the coverage varies by location measurement, in some places even with good coverage satellites, radar coverage may be very small.
Even so, there are such vast amounts of observational data, as well as many types of data, weather forecasting system that is difficult to incorporate them all. In the United States, the growth rate of the scale of remote sensing data collected by the National Oceanic and Atmospheric Administration (https://www.noaa.gov/)(NOAA) has reached a daily 100TB. These data will provide NOAA weather prediction engine to run on supercomputers to provide the next 1-10 days of global weather forecasting. The development of engines over the past half century, are based on numerical methods, it can be directly simulate physical processes, including atmospheric dynamics and a large number of effects, such as thermal radiation, vegetation, lake, sea etc. effect.
However, the availability of computing resources at multiple levels limits the ability of the numerical method based on the weather forecast. For example, the demand for computing power will limit the spatial resolution in the range of about 5km, which is not enough to analyze weather patterns in urban areas and farmland. Numerical methods take several hours to run. If six hours to complete the calculation of a predictable, only 3-4 times per day forecasts, resulting in each prediction is based on the six hours before the old data, which limits understanding of the current situation is taking place. By contrast, short-term forecast for traffic routing and evacuation plans and other instant decision-making scene is more applicable.
Radar radar forecast
As a typical example the system can generate a type of prediction, researchers consider a radar radar prediction problem: given the last hour radar FIG sequence, predicted radar chart within hours from now N, where N is typically 0-6 between the hours. Since the radar data has been converted into an image, this prediction can be viewed as a computer vision problems, from the input image sequence, to predict the evolution of weather. In such a short time scale, the evolution of the primary control of two physical processes: advection causes movement of clouds, resulting in convective cloud formation, influenced by both local topography and geographical conditions.
The row (left to right): the former figure shows three before the current time of 60 minutes, 30 minutes and 0 minutes (i.e., time required point predicted) radar FIG. Far right shows the radar chart 60 minutes after the current time, that is the true value of the short-term prediction. Below left: on the first three rows of FIG optical flow vector field application (OF) algorithm (as a comparison). Optical flow method was developed in the 1940s a computer vision method, commonly used to predict short-term changes in the weather. Bottom right: Example result predicted by the optical flow method. It may be noted that it is well to track the movement of the figure the lower left corner of precipitation, but did not consider the attenuation of storm intensity.
The researchers are using a non-physical method of data-driven, which means that the neural network is just learning how to fit physical changes in the atmosphere from the training sample, without introducing any prior knowledge of how the atmosphere works. The weather forecasting problem as a problem of converting the image to image, and image analysis using the currently most advanced convolution neural network (CNNs) technology to solve.
CNNs layer usually consists of a linear sequence, where each input image are converted into a certain set of operating new output image. Typically, layers other convolutional neural network uses a set of convolution collation image convolution, the number of channels and also changes the overall resolution of the image. The convolution kernel itself is a small image (for us small image, usually 3x3 or 5x5). CNN convolution kernel to provide most of the power, and brings the edge detection, to identify meaningful patterns and other operations.
U-Net is a particularly effective CNN. U-Net, as a group are arranged in the layer sequence coding phase, wherein the reduced resolution input image, layer by layer; decoding stage is followed, at this stage, the encoder generates a low-dimensional representation of the image is expanded back to a higher resolution. The following figure shows all the layers of the specific U-Net structure.
(A) an overall configuration of U-Net. CNN blue block corresponding base layer; pink layer block corresponding to the sample; green block corresponding to the sample layer. The solid line indicates the connection relation between the input layer; broken line indicates the long jump across the connection of the encoding and decoding stages; the dotted line shows the respective layers of the short hop connection. (B) operating in the base layer. (C) sampling on the operation layer.
U-Net input is an image, the last hour of the observation sequence in multispectral images each satellite occupies one of the channels. For example, if the last hour of the 10 satellite image acquisition, each image multispectral imaging to 10 are different wavelengths, then the input image will be a model of the channel 100 thereof. For prediction to FIGS radar radar chart, input 30 is a sequence of radar observations in the last hour of the composition, a 2 minutes apart; comprising prediction result output from now N hours. Preliminary work for the United States, the researchers used historical data observed in the continental United States 2017-2019 training network. Data period is divided into four weeks, the first three weeks of each cycle, as training data, evaluation results for the fourth week.
result
Researchers will result with three widely used models were compared. First, high-resolution NOAA's fast refresh (HRRR) numerical prediction method. HRRR contains many different climate forecast for the amount, where researchers with the results of one hour ground cumulative total precipitation forecast comparison, because this is the highest quality of one hour precipitation forecast indicators. The second algorithm is based on the optical flow (OF), the method attempts to track the series of images of moving objects. This is a method often used for weather forecasting, even if it made an assumption that is obviously not true - a total precipitation within a larger area in the forecast period is constant. Third, the so-called persistent model, which is an ordinary model, it is assumed that a place will be like in the future as the current time in the same level of precipitation, that is precipitation pattern does not change. This may seem an over-simplified model, but given the difficulty of forecasting the weather, which is a common practice.
Within approximately one day for doing predicted visualization. Left: 1 hour HRRR prediction made at the beginning of every hour, which limitation is provided HRRR prediction interval. Middle: the real situation, that is the situation you want to predict. Right: forecasting model made by the researchers. Every two minutes can be predicted (here shows the results of every 15 minutes), the resolution is about 10 times the prediction region HRRR method. It may be noted this method to capture the overall movement of the storm and morphology.
The researchers used accuracy - the recall rate (PR) images to compare models. Because the model can be directly to the classifier results and therefore provides full profile PR (the blue line in the figure). However, the researchers can not directly get HRRR model, and the persistence model and model-based optical flow are not the ability to weigh the trade-off between precision and recall rates, these models can only be represented by a single point. Can be seen, the quality of the neural network prediction method researchers than three other models (since the other points in the blue line represents the model above). It is noteworthy that, when the forecast range of 5-6 hours, the result HRRR model began to exceed the results of the moment.
Accuracy - recall rate (PR) curve (https://en.wikipedia.org/wiki/Precision_and_recall) The results of the method (blue line) for comparison, and the following three methods: Method optical flow (OF) based on continuous model, HRRR prediction for 1 hour. Because you can not get directly to their classification, and therefore can not provide a complete PR curve. Left: predictions for rain. Right: predictions for moderate rain.
Based on the advantages of a large machine learning methods is to predict the result is effective real-time, which means that real-time prediction may be based on new data, while HRRR be affected by a delay of 1-3 hours of computing. This method is based on computer vision that may provide better prediction result is short. In contrast HRRR using numerical model may provide a more long-term prediction, in part because it uses the full 3D physical model - cloud formation is difficult to observe from the 2D image, it is more difficult to machine learning based learning process to convection. The combination of the two systems - the use of machine learning models for fast prediction, using HRRR long-term forecasts - could produce better results on the whole, this is a focus of future work may concern. The researchers are also considering 3D for machine learning directly observed data. In any case, real-time forecasting in real time planning, decision support and improve the lives of the key tools.
Original link: https://ai.googleblog.com/2020/01/using-machine-learning-to-nowcast.html
