"Has the large model you trained emerged?" "Not yet. It's so uncomfortable." For a while, it became a hot topic among AI track players recently.
Whether you admit it or not, I believe that every player is unwilling to lose this fierce competition. Since Baidu became the "first person to eat crabs" in China, two more Chinese technology giants have made preparations - Huawei and Alibaba's newly developed "big model" and "ChatGPT" will also be launched in the near future. come out.
The first is the news that the new version of Huawei's Pangu model will be unveiled soon! It is said that on April 8, Tian Qi, Chief Scientist of Huawei Cloud Artificial Intelligence, will introduce "the progress and application of the Pangu Large Model" through a technology sharing at the Hangzhou Artificial Intelligence Large Model Technology Summit Forum.
At the same time, a meeting invitation letter from Ali also brought important news to the industry: at the Alibaba Cloud Summit on April 11, the latest self-developed large model of Ali will be officially launched, and the internal testing of the model is currently It is already in progress, and various industry application models will come out in the future. Just at noon today, Tongyi Large Model has officially announced the start of enterprise invitation testing!
If it is said that the giants from "refining the model" to "refining the large model" were showing their muscles and laying the foundation, then the significance of releasing the model this time is more about entering the ChatGPT product track!
1. Huawei: Deeply cultivating the B-end and opening up multiple business scenarios
In fact, Huawei has launched the layout of AI large models since 2020. According to relevant information, the Pangu NLP large model adopts deep learning and natural language processing technology, and uses a large number of Chinese corpora for training. Based on the industrialized AI development model of "pre-training + downstream fine-tuning", it has over 100 billion parameters and supports a variety of Natural language processing tasks, including text generation, text classification, system question answering, etc.
1. Huawei ChatGPT: close to GPT3.5 level, focusing on Chinese
optimization Large models and scientific computing large models. According to reports, the NLP large model has learned over 40TB of Chinese text data in the pre-training stage; the CV large model has achieved both image discrimination and generation capabilities for the first time; and the scientific computing large model can be applied to meteorology, biomedicine and other fields.
Compared with ChatGPT, the Pangu NLP large model is closer to the level of GPT3.5 in terms of data, but it pays more attention to Chinese language optimization, has greater advantages in understanding Chinese grammar and semantics, and has strong generalization ability and small sample learning And the three characteristics of high precision of the model can be applied to a large number of complex industry scenarios, and even a small number of samples can achieve high precision.
According to Huawei's official introduction, the Pangu NLP model ranks first in the general ranking list, classification, and reading comprehension items, setting a new world record in three rankings. One of the pretrained models close to the level of human understanding (85.61).
2. Huawei's three unique advantages
In terms of large-scale model research and development, Huawei has a relatively unique advantage compared to other domestic companies-it has a complete industrial chain and strong computing power matching capabilities. According to reports, the large models of the Pangu series are all based on Huawei Model Arts, and the computing power base is provided by the second phase of Pengcheng Cloud Brain, which will continue to rank first in the world's artificial intelligence computing power in 2022. When training the Pangu model, the Huawei team used more than 2,000 Ascend 910 chips for more than two months of training.
In terms of industrial ecology, Huawei has continued its corporate genes that focus on the B-side. The original intention of the industrialization of large models is to open up more B-side business scenarios. on a higher priority". In addition, the Pangu model also incorporates Huawei's technological advantages in 5G, cloud computing, and the Internet of Things. It can be applied to intelligent customer service, machine translation, voice recognition, and other fields to provide enterprises with intelligent services.
According to the research report of CITIC Construction Investment, Pangu’s pre-training large model capabilities have been verified in fields including energy, retail, finance, and industry. At the same time, the Pangu NLP large model achieves the learning goal of few samples through transfer learning, and adopts distributed computing technology, which can realize online training and incremental learning of the model, continuously optimize the model with the increase of data volume, and improve the accuracy and quality of the model , making it more suitable for complex commercial scenarios, and its performance has surpassed GPT-3.5 in some applications.
2. Ali: The C-end is making efforts, and the evaluation results are gratifying
Let’s look at Ali again. Coincidentally, Ali’s research and development of Chinese large models also began around 2020. In 2021, Ali successively released the first domestic multimodal large model "Tongyi-M6" with over 10 billion parameters, and the language model PLUG known as "Chinese version of GPT-3". Although the number of parameters is only 27 billion, PLUG has the same powerful text generation ability as GPT-3.
1. Integration and upgrade of the Tongyi large model
According to reports, the upcoming ChatGPT version of Ali Dharma Academy is based on the integration and upgrading of the Tongyi large model system. The Tongyi large model base is built based on the underlying technologies such as the unified learning paradigm OFA, and can simultaneously process multiple single-modal and multi-modal tasks such as texts, graphs, picture descriptions, and content summaries without introducing new structures . After upgrading, it can handle a variety of cross-modal tasks including voice and motion.
Among them, "Tongyi-M6" mainly solves the related tasks of text image generation and construction, including image-text understanding, image-text generation, speech understanding, and speech generation, and the model version released by Ali in 2021 has reached 100 billion parameters; while Tongyi-M6 AliceMind is an NLP pre-training model, and its application scenarios include Wenshengwen, text understanding, question-and-answer dialogue, etc.; general-vision application scenarios cover video representation, image detection, video editing, etc.
2. Ali’s internal revelations
In a previous interview about Ali’s large-scale model, Ali’s internal personnel said: There are two main solutions that Ali can provide in terms of large-scale models. One is that the models launched by Ali’s internal The combination of a wide range of C-end products within the group, such as the search business of Tmall, Taobao and AutoNavi, will bring a new business model to the search engine, and will also inject more vitality into Ali’s already thriving C-end business , and the data accumulation at the C-end will also well feed back the research and development of Ali's large model and related applications. In addition, Ali can also export its own API and charge partners or channel providers. For API, it may be helpful to partners in specific fields, such as e-commerce, search recommendation, etc.
In terms of computing power, Alibaba Cloud has at least tens of thousands of A100s on the cloud, and the overall number can reach at least 100,000. For the group, it should be 5 times that of Alibaba Cloud. The computing resources of Bodhidharma Academy, Tmall, and Taobao are all used within the group. Due to the demand for large-scale model and derivative application research and development, Alibaba Cloud will grow by 30-50% this year. Some customers will have the need to reproduce GPT, and put forward large-scale AI computing power requirements. Alibaba will provide cloud support.
In addition, there are also revelations that Ali's upcoming ChatGPT-like dialogue robot product may also be combined with DingTalk productivity tools, which will also be confirmed by Alibaba later.
3. Evaluation of bloggers at station B: the results are gratifying What’s
more noteworthy is that in recent days, bloggers at station B have tested Ali’s Tmall Genie and found that it has launched the voice assistant of Ali’s version of ChatGPT. Ali also responded to this, saying that what Tmall Genie is currently integrating is a large-scale model technology demo, "Tmall Genie and Dharma Academy have been working closely together, including promoting comprehensive applications such as large language models, acoustic models, and voice AI. ".
In the evaluation video released by the blogger at station B, the blogger had 15 rounds of question-and-answer dialogues for more than 3 minutes with the Tmall Elf who integrated the demo version of the large model, and the answers to 10 questions were significantly better than those of the same type published in China. product. And after learning about the user's ordering needs, Tmall Genie agreed to the user's request and said that it had helped the user complete the order. Although the demo version of Tmall Genie did not complete the task as it said because it did not integrate the corresponding interface, it is believed that after the official version is released, functions such as ordering food, taxis, and ticket purchases may be realized.
3. Experts commented on Huawei's solid ecological layout and Ali's data and computing power are better
So, what are the comments and opinions of industry experts on the upcoming large models of Ali and Huawei? To this end, 51CTO interviewed Mr. Liu Zhanliang, an expert in artificial intelligence technology and former technical director of the Government Innovation Center of Zhiyuan Research Institute.
According to Liu Zhanliang's analysis, Huawei's main advantage in this area is: Huawei's long-term accumulation in the toB field enables it to have a very solid industrial ecology, and Huawei's full-stack layout in artificial intelligence will help enterprises in various fields, Long-term development in various businesses. But from a short-term perspective, the commercialization progress of Huawei's large-scale models may be affected by the maturity of the underlying infrastructure.
For Ali, it has an ecosystem covering multiple industries. The accumulation of a large amount of C-end and B-end data will provide a very important impetus for the research and development of Ali’s large model. In terms of computing power, compared with most technology companies, Ali It also has huge advantages. With the blessing of data and computing power advantages, Ali should be able to reach a relatively high level of efficiency in large-scale model development and production applications.
Although from the perspective of application fields and target users, Huawei and Ali may be different. But at the end of the day, both companies have the potential to push China's AI scene forward.
But before that, Liu Zhanliang also pointed out that there are still three major challenges facing the two giants. One is to deal with the fierce market competition at home and abroad; the other is to be precise about potential political risks with zero mistakes The third is the clichéd technical challenge. Both Huawei and Alibaba have accumulated rich experience in their own business, so can the large model and a series of derivative products developed by them play a higher role in actual production? Efficiency, which tests the enterprise's application and understanding of technology in a specific field.
4. Localization of ChatGPT is the way out
Recently, ChatGPT has been caught in a series of disputes. Due to the illegal collection of user data, Italy has banned ChatGPT completely. Germany is also planning to ban ChatGPT nationwide due to data protection considerations. use. Not long ago, ChatGPT also carried out a large-scale silent ban on the Asian region. From the above incidents, relying on ChatGPT, Microsoft and OpenAI have become the de facto number one players, and this is not a good momentum for the development of similar products of domestic companies, whether it is banned or banned.
It is true that due to the natural complexity of the Chinese language, it will become more difficult to realize the Chinese version of ChatGPT, but domestic technology companies can use their own advantages to develop the same type of models and applications for different business scenarios during this period. Whether it is out of the consideration of self-interest or the pursuit of a higher ideal, this in itself is something to be encouraged. While the development of artificial intelligence brings about a technological revolution, it is also destined to bring about a hundred schools of thought on the same track, whether it is at the corporate level or at the global level, it is inevitable.
5. Write at the end
Today, the "narrative" of ChatGPT becoming the next-generation operating system has just begun, and the powerful productivity improvement it brings will surely empower thousands of industries and bring huge business opportunities. Under such circumstances, independent controllability and reasonable supervision are issues that every country, every technology company, and even every technology practitioner should pay attention to and think about.
Although at present, no matter Baidu, Huawei, Ali or other technology companies that are deeply cultivating this field, they are temporarily unable to launch AI large models that can compete with GPT-4. But sometimes late development is not necessarily a bad thing. Let us give them some confidence and time. I believe that with the continuous efforts of these companies and technology practitioners, the pace of domestic AIGC ecological construction autonomy will be faster and faster.