Click the card below to follow the " CVer " official account
AI/CV heavy dry goods, delivered in the first time
Click to enter —> [Target Detection and Transformer] Exchange Group
Yun Zhong sent from Aofei Temple
and reprinted from: Qubit (QbitAI)
It is 10,000 times faster than traditional methods and only takes 1.4 seconds to complete a 24-hour global weather forecast— it is the Pangu weather model
from Huawei Cloud .
Today, it was published in Nature, which is said to be the first paper published in Nature in recent years with a Chinese technology company as the sole author (that is, Huawei Cloud's exclusive work).
Reviewers gave it high marks, and this model allows humans to re-examine the future of weather forecasting models.
The implication is that with it, the original traditional method is not fragrant.
So how exactly was it developed? What key problems were solved? What are the specific effects and applications?
Follow this paper to take you through it.
Solve the problem of insufficient accuracy of existing AI weather forecast models
Since the 1920s, especially in the past 30 years, with the rapid development of computing power, traditional numerical weather prediction has achieved great success in daily weather forecasting, extreme disaster warning, climate change prediction and other fields.
However, as the increase in computing power slows down and the physical model becomes more complex, the bottleneck of this approach becomes increasingly prominent.
So researchers began to mine new weather forecasting paradigms such as using deep learning methods to predict future weather.
The Huawei Cloud R&D team started research in this area two years ago.
They found that in the field where numerical methods are most widely used, such as medium and long-term forecasting, the accuracy of existing AI forecasting methods is still significantly lower than numerical forecasting methods, and is constrained by problems such as lack of interpretability and inaccurate prediction of extreme weather.
There are two main reasons for the lack of accuracy of the AI weather forecast model :
First, the existing AI weather forecast models are all based on 2D neural networks, which cannot handle uneven 3D weather data well;
Second, AI methods lack mathematical and physical mechanism constraints, so iteration errors will continue to accumulate during the iterative process.
Here, researchers at HUAWEI CLOUD proposed 3D Earth-Specific Transformer (3DEST) to process complex and uneven 3D meteorological data, thus creating a large Pangu meteorological model.
The main idea is to use a 3D variant of a visual transformer to deal with complex and uneven meteorological elements, and use a hierarchical time-domain aggregation strategy to train models with 4 different forecast intervals (1-hour interval, 3-hour interval, 6-hour interval, 24-hour interval), which minimizes the number of iterations for predicting weather conditions at a specific time, thereby reducing iteration errors and avoiding the consumption of training resources caused by recursive training.
To train each model, the researchers trained for 100 epochs using weather data from 1979-2021, sampled hourly.
Each model needs to be trained for 16 days on 192 V100 graphics cards. In fact, even after 100 epochs, these models still do not fully converge.
In other words, with more sufficient computing resources, the accuracy of AI forecasts can be further improved.
In the final reasoning, the large Pangu meteorological model only needs to run for 1.4 seconds on a V100 graphics card to complete a 24-hour global weather forecast, including geopotential, humidity, wind speed, temperature, sea level pressure, etc., and the horizontal spatial resolution reaches 0.25∘×0.25∘ , with a time resolution of 1 hour and covering 13 layers of vertical height, it can accurately predict fine-grained meteorological features.
As the first AI method whose accuracy exceeds traditional numerical forecasting methods, its calculation speed is more than 10,000 times faster than traditional numerical forecasting.
Can be directly applied to multiple downstream scenarios
In May of this year, the direction of typhoon "Mawar" received widespread attention.
The Central Meteorological Administration stated that the Huawei Cloud Pangu large model performed well in the path forecast of "Mawa", and predicted that it would turn to the east coast of Taiwan Island five days in advance.
At the 19th World Meteorological Congress, the European Meteorological Agency also pointed out that Huawei Cloud's Pangea weather model has undeniable capabilities in accuracy, and the pure data-driven AI weather forecast model has shown that it can compete with the European medium-range weather forecast center. Forecasting strength comparable to numerical models.
Florence Habier, director of the European Center for Medium-Range Weather Forecasting, showed in detail the real-time operation test comparison between the Huawei Cloud Pangu Meteorological Model and the European Center for Medium-Range Weather Forecasting:
In order to explore the ability of AI to capture extreme weather, we studied a case in Finland in February this year, when a cold wave of -29°C was observed, and we found that Pan Gu recognized the seriousness of this event earlier.
Florence Harbier also emphasized that the AI forecasting method consumes less resources and provides important opportunities for developing countries, because it no longer requires large-scale supercomputing resources, and also provides a rare opportunity to improve global forecasting capabilities .
As for HUAWEI CLOUD’s choice of the field of AI weather forecasting as a “breakthrough”, on the one hand, weather forecasting, especially the accurate prediction of extreme weather such as heavy rain, typhoon, drought, and cold wave, is related to international people’s livelihood. On the other hand, weather forecasting is very complicated. AI New laws of atmospheric evolution can be mined from massive amounts of data, which has huge potential for improvement in accuracy and speed.
It is understood that the WMO 2024-2027 strategic plan to be released by the World Meteorological Organization (WMO) absorbs artificial intelligence elements, making it an important force to promote the development of meteorological science and technology.
WMO will also actively promote the demonstration and application of AI in the fields of nowcasting and numerical weather forecasting, create an international comparison platform for the application of artificial intelligence products, formulate AI meteorological application standards and guidelines, promote the sharing of artificial intelligence data sets and other related work, explore and develop The application potential of AI in the field of meteorology effectively supports the early warning initiative for the whole people.
Three keys to the future
Finally, how does the HUAWEI CLOUD Pangu meteorological large-scale model team view the future of AI weather forecasting?
The answer lies in three keys:
First, big data . Huge meteorological data is the cornerstone of the AI model. The current Pangu meteorological model only uses part of the ERA5 reanalysis data. The future AI model will be based on massive and more refined global observation data.
Second, large computing power . The ultra-high resolution of meteorological data poses a huge challenge to the training of AI models. The current input resolution of the Pangu meteorological large model is 1440×720×14×5, compared with the commonly used resolution of 224×224×3 for computational vision tasks. About 500 times, as the resolution further increases and the model increases, the required computing resources will also increase rapidly.
Finally, the big model . Complex meteorological laws, ultra-high resolution and huge amount of data all determine that AI weather forecasting requires the use of extremely computationally intensive AI models.
At the same time, if you want to continuously iterate the leading AI weather forecast model, a stable cloud environment, work suite and corresponding operation and maintenance are also essential.
Paper address:
https://www.nature.com/articles/s41586-023-06185-3
Click to enter —> [Target Detection and Transformer] Exchange Group
The latest CVPR 2023 papers and code download
Background reply: CVPR2023, you can download the collection of CVPR 2023 papers and code open source papers
Background reply: Transformer review, you can download the latest 3 Transformer review PDFs
目标检测和Transformer交流群成立
扫描下方二维码，或者添加微信：CVer333，即可添加CVer小助手微信，便可申请加入CVer-目标检测或者Transformer 微信交流群。另外其他垂直方向已涵盖：目标检测、图像分割、目标跟踪、人脸检测&识别、OCR、姿态估计、超分辨率、SLAM、医疗影像、Re-ID、GAN、NAS、深度估计、自动驾驶、强化学习、车道线检测、模型剪枝&压缩、去噪、去雾、去雨、风格迁移、遥感图像、行为识别、视频理解、图像融合、图像检索、论文投稿&交流、PyTorch、TensorFlow和Transformer等。
一定要备注：研究方向+地点+学校/公司+昵称（如目标检测或者Transformer+上海+上交+卡卡），根据格式备注，可更快被通过且邀请进群
▲扫码或加微信号: CVer333，进交流群
CVer计算机视觉（知识星球）来了！想要了解最新最快最好的CV/DL/AI论文速递、优质实战项目、AI行业前沿、从入门到精通学习教程等资料，欢迎扫描下方二维码，加入CVer计算机视觉，已汇集数千人！
▲扫码进星球
▲点击上方卡片，关注CVer公众号It's not easy to organize, please like and watch