According to some media reports , Huawei will release a multi-modal 100-billion-scale large-scale model product called "Pangu Chat" that directly targets ChatGPT.
According to reports, the Pangu model was successfully approved in Huawei Cloud in November 2020. This "Pangu Chat" is expected to be released and internally tested at the Huawei Cloud Developer Conference (HDC.Cloud 2023) held on July 7 this year. The product is mainly for To B / G government and enterprise customers.
According to a paper released by Huawei, the parameters of Huawei's Pangu-Σ large model are at most 1.085 trillion, which is developed based on Huawei's self-developed MindSpore framework. Overall, the PanGu-Σ large model may be close to the level of GPT-3.5 in terms of dialogue.
IT Home inquired about public information and learned that the Huawei Pangu model was officially released in April 2021, and was later upgraded to version 2.0 in April 2022. At present, the NLP large model, CV large model, and scientific computing large model (meteorological large model) in the AI large model have been marked as coming online.
According to reports, this is the first Chinese pre-training large model with 100 billion parameters, and the CV large model has reached 3 billion parameters for the first time. Pangu CV Large Model The largest CV large model in the industry, it is the first to achieve both discrimination and generation capabilities, and it is the industry's first in the small sample learning ability on ImageNet; the Pangu meteorological large model provides second-level weather forecasts; Zidong Taichu is the world's first map , text, and audio three-modal large model.
For the positioning of the Pangu large model, Huawei's internal team has established three key core design principles: first, the model must be large enough to absorb massive amounts of data; The generalization ability can truly be applied to work scenarios in all walks of life.
According to the PPT information of Huawei Cloud executives' speeches, at present, the basic layer of Huawei's "Pangu series AI large model" mainly includes NLP large model, CV large model, and scientific computing large model, etc., and the upper layer is Huawei's industry large model developed with partners .
The official website of Huawei Cloud shows that the Pangu large model is composed of multiple large models such as NLP large model, CV large model, multimodal large model, and scientific computing large model. AI scale, industrialization problems, can support a variety of natural language processing tasks, including text generation, text classification, question answering system, etc.
Specifically, the Pangu NLP large model uses the Encoder-Decoder architecture for the first time, taking into account the comprehension and generation capabilities of the NLP large model, ensuring the flexibility of embedding the model in different systems. In downstream applications, only a small number of samples and learnable parameters are needed to complete the rapid fine-tuning and downstream adaptation of a large-scale model of 100 billion. This model has a good performance in intelligent public opinion and intelligent marketing.
The Pangu CV large model is the industry's largest CV large model that realizes on-demand model extraction for the first time. For the first time, it realizes both discrimination and generation capabilities. Based on model size and operating speed requirements, it adaptively extracts models of different scales, and AI application development is quickly implemented. Using hierarchical semantic alignment and semantic adjustment algorithms, better separability of shallow features has been obtained, and the ability of small-sample learning has been significantly improved, ranking first in the industry. Good performance in logistics.
The Pangu Meteorological Large Model provides second-level weather forecasts. With the help of the innovative 3DEST network structure and layered time aggregation algorithm, the accuracy of the key elements of weather forecasts and common time ranges exceeds the current most advanced forecasting methods, and the speed is faster than traditional methods. More than 1000 times. At the same time, the Pangu meteorological large model supports a wide range of downstream forecasting schemes. For example, in the typhoon track prediction task, compared with the traditional numerical weather forecasting method, the Pangu meteorological large model can reduce the position error by more than 20%.
The information previously disclosed by Zheshang Securities shows that Huawei used more than 2,000 Shengteng 910 chips when training the Pangu large model with 100 billion parameters, and carried out data training capabilities for more than 2 months. According to Huawei internally, more than 4,000 GPU/TPU cards are called for large-scale model training each year, and the computing power cost of large-scale models in three years is as high as 960 million yuan.
Soochow Securities pointed out in the research report on Huawei’s Pangu large-scale model industry chain that the advantage of Huawei’s Pangu large-scale model lies in its talent reserve and independent controllable computing power. , including Tuowei Information, Sichuan Changhong, Kylin Software (China Software), Tongxin Software (ArcherMind Technology), Kylin Principal and other Huawei ecological companies. Guosheng Securities believes that Huawei Pangu is the first multi-modal 100 billion-level large-scale model, which is expected to empower all industries.
